Reference · Glossary
Mixture of experts
Architecture that routes each token to a subset of expert sub-networks — scales capacity without running every parameter.
Reference · Glossary
Architecture that routes each token to a subset of expert sub-networks — scales capacity without running every parameter.