A benchmark is a standardized test or dataset used to measure and compare the performance of AI models on specific tasks. Think of it as the final exam that every model takes under the same conditions. Common benchmarks include MMLU for general knowledge, HumanEval for coding, and GLUE for language understanding. They help researchers and businesses choose the right model, but benchmarks have limits: a model that aces a benchmark may still struggle with real-world messy data.
Fundamentals
Benchmark
Related terms
Learn to use these concepts in practice.
Join the 212AY Academy →