What Does This Mean?
When items from a test or benchmark set end up in a model's training data, so the model has effectively seen the exam in advance and its scores overstate what it can really do. Often discussed as benchmark leakage.
REAL-WORLD EXAMPLE
""Its test score jumped, but training data contamination could explain why.""
LORE & ORIGIN
A paper on benchmark leakage submitted to arXiv on April 29, 2024 describes the problem as a benchmark example, from the training or test split, appearing in a model's pretraining data. Wikipedia's data leakage article covers the related older machine-learning idea and methods such as Min-K% for detecting pretraining data.
MENTIONS OVER TIME
Radar observation data not yet available for this term.
RELATED SEARCHES
TOP PLATFORMS
Not enough Radar data yet to show a platform breakdown.
WHEN DID YOU FIRST HEAR THIS?
CLICK AN OPTION BELOW TO CAST YOUR VOTE.
[ VOTE TO REVEAL COMMUNITY RESULTS ]