What Does This Mean?
Short for mixture of experts: a neural-network design in which several specialized sub-networks (experts) sit behind a gating mechanism that routes each input to only some of them, so only a fraction of the model's parameters is used for each input.
EXPLAINER
Jia-Bin Huang · original source ↗
EXPLAINER
AIChronicles_JK · original source ↗
The preview image is loaded from YouTube when this page opens, so YouTube receives your IP address and browser details, as with any request to another site. The video itself loads only after you click play. Details in our Terms
REAL-WORLD EXAMPLE
""The model is an MoE, so only a small slice of its parameters runs for each token.""
LORE & ORIGIN
Wikipedia dates the classic form to Jacobs, Jordan, Nowlan and Hinton's February 1991 paper "Adaptive Mixtures of Local Experts". Sparsely-gated layers appeared at Google Brain in 2017, and Mistral AI's Mixtral 8x7B (December 2023, 8 experts, 2 active) brought the design into wide open-weight use.
MENTIONS OVER TIME
Radar observation data not yet available for this term.
RELATED SEARCHES
TOP PLATFORMS
Not enough Radar data yet to show a platform breakdown.
WHEN DID YOU FIRST HEAR THIS?
CLICK AN OPTION BELOW TO CAST YOUR VOTE.
[ VOTE TO REVEAL COMMUNITY RESULTS ]