Skip to content
CodeWiki
Practice
Paths
Tracks
Cheatsheets
Playground
Glossary
AI era
Search
⌘K
English
English
Chinese
Practice
Paths
Tracks
Cheatsheets
Playground
Glossary
AI era
English
English
Chinese
Practice
/
Quiz
/
Coding in the AI era
/
LLM application evals
A candidate delta is -1 percentage point, the allowed regression is -2 points, and the co…
from LLM application evals
Node 24
advanced
3 min
A candidate delta is -1 percentage point, the allowed regression is -2 points, and the confidence interval is [-4, +2]. What should the gate report?
Inconclusive, because the interval includes both allowed and forbidden changes
Pass, because the point estimate is above -2 points
Fail, because the interval includes -4 points
Pass, because the interval also includes an improvement
Check
Ask AI about this kata
Report an error
next kata
Before an LLM judge blocks releases, what is the most useful calibration step?
Quiz
A new version is available
Reload