Reference · Glossary
Mixture of experts (MoE)
Last updated
A model architecture built from many smaller "expert" sub-networks, where a router picks only a handful of experts to run for each token — giving a model a very large total parameter count while only using a fraction of it per request, which keeps inference faster and cheaper than a same-sized "dense" model.
#When to use
You don't choose this directly as a user — it matters when comparing models: an MoE model can offer strong quality at lower latency/cost than a dense model with a similar total size.
#When not to
N/A as a usage decision — it's an architecture choice made by model builders. As a learner, just know why some "huge" models still respond quickly and cheaply.
#Example
A 200B-parameter MoE model that only activates ~20B parameters per token behaves, cost- and speed-wise, much closer to a 20B dense model — while still drawing on the knowledge spread across all its experts.