What Does This Mean?
A machine learning design that uses several specialised expert networks plus a gating function deciding how much each contributes, so only some parameters are active per input; widely used in large language models, often shortened to MoE.
EXPLAINER
Jia-Bin Huang · original source ↗
EXPLAINER
AIChronicles_JK · original source ↗
The preview image is loaded from YouTube when this page opens, so YouTube receives your IP address and browser details, as with any request to another site. The video itself loads only after you click play. Details in our Terms
REAL-WORLD EXAMPLE
""The mixture of experts model only fires two experts for each token.""
LORE & ORIGIN
Wikipedia cites the 1991 paper Adaptive Mixtures of Local Experts by Jacobs, Jordan, Nowlan and Hinton, with Mixtral 8x7B (December 2023) and DBRX (March 2024) as language-model examples.
MENTIONS OVER TIME
Radar observation data not yet available for this term.
RELATED SEARCHES
TOP PLATFORMS
Not enough Radar data yet to show a platform breakdown.
WHEN DID YOU FIRST HEAR THIS?
CLICK AN OPTION BELOW TO CAST YOUR VOTE.
[ VOTE TO REVEAL COMMUNITY RESULTS ]