Book a call

Daniel & Friends / Glossary / Model evaluation (evals)

AI and automation

Model evaluation (evals)

Model evaluations are systematic tests that measure the quality of AI answers on a set of typical tasks with a verified correct result. They assess accuracy, consistency, safety or style, and allow models or prompt versions to be compared.

Daniel Votruba ·


In practice

Build your own set of real tasks from your daily work and run it whenever the model or prompt changes. Public leaderboards give only a rough orientation.

Related terms

  • Hallucination: A hallucination is a case where a model states plausible-sounding but false information.
  • Large language model (LLM): A large language model is a model trained on a huge amount of text that predicts the next words and can write, summarise and answer.
  • Prompt engineering: Prompt engineering is the design and testing of instructions for a model: choosing the role, context, examples and answer format so that the output is reliably usable.
  • AI safety and alignment: AI safety is the field concerned with making artificial intelligence systems behave safely and predictably.
  • Human in the loop: Human in the loop is a design for an AI process in which a person reviews, edits or approves the model’s work at chosen points before the result is used.

More terms in the group AI and automation

Multi-agent system · Model bias (AI bias) · Multimodal model · MCP (Model Context Protocol) · Neural network · Machine learning (ML) · NLP (natural language processing) · No-code and low-code · Jailbreak (AI) · Open-weight model · Inference and latency · Predictive analytics · Prompt · Guardrails · Prompt injection · Grounding · RAG (retrieval-augmented generation) · Generative AI · Reasoning model · Function calling (tool use) · RPA (robotic process automation) · Foundation model · Semantic search · Fine-tuning

Full glossary (590 terms)

Terms in practice

Need help putting them to work?

Terms are one thing, putting them to work is another. We go through which of them pay off for you and say so plainly. You get a first opinion for free and we reply within 24 hours.

Book a call

Enjoyed the article? Share it with friends.