Token likelihood, verbal confidence and the probability that a real-world claim is correct are different quantities. Confusing them creates false precision.
What next-token likelihood measures
A generative model estimates the relative likelihood of candidate next pieces of text in a context. That quantity helps determine which continuation is generated. It is not automatically the probability that the full sentence is factually correct. Many fluent sentences are common because they match writing conventions, including sentences about fictional events. Conversely, an uncommon but true technical result may receive lower language-model probability than a familiar misconception.
What a confidence percentage needs
A meaningful estimated probability of correctness requires a precisely defined event, a scoring method, representative labeled examples and calibration testing. If a service displays ‘90% confidence’, ask what was calibrated, on which task, using which population and which model version. A confidence estimate for selecting among four answer options cannot be carried across to an open-ended legal citation task without new validation. Self-reported numbers in model prose are especially unsuitable as standalone statistical guarantees.
A small numerical example
Suppose a support classifier assigns high confidence to 100 cases and 70 classifications are correct. Its observed accuracy in that group is 70%, regardless of whether it printed 95% for every case. You would still need confidence intervals, representative sampling and checks for class imbalance to interpret this number. An open-ended answer with multiple claims demands additional definitions of what counts as correct: all claims, material claims or a weighted review?
Calibration can decay
A calibration chart describes performance under the measured distribution. A model replacement, new document collection, unseen language or altered prompt can change error patterns. Recheck after material changes and during operation. Preserve raw outputs, reference answers and evaluation settings where privacy rules permit. A single aggregated accuracy score can conceal poor outcomes for critical subgroups, rare cases or non-English tasks.
How to use uncertainty responsibly
Use a three-state workflow: verified claim, plausible but unchecked interpretation, and unknown. If an item controls a payment, clinical action or compliance decision, demand traceable records rather than accepting a declared confidence score. In lower-stakes drafting, AI can propose options with uncertainty clearly labeled. Combine abstention metrics with accuracy; a system that refuses everything has few incorrect answers but may be unusable.
What independent assessments contribute
Frameworks such as Stanford HELM and MLCommons publish specified scenarios, metrics and methodologies. Their reports can help readers ask reproducibility questions, but benchmark metrics are not individualized truth probabilities. Read the sampling, grader and language coverage before extending a reported result to your own workflow. Where these details do not match, conduct a task-specific evaluation.
Practical bottom line
Treat impressive precision as a request for evidence, not evidence itself. A reliable statement explains the data, test distribution, evaluation method and domain where an estimate is valid. If these are missing, show an uncertainty label and verify the underlying facts through primary records.
Sources & Further Reading
- NIST AI Risk Management Framework 1.0 — NIST.
- Holistic Evaluation of Language Models — Stanford CRFM.
- AILuminate benchmarks and methodology — MLCommons.
Primary Scope and applicability vary by task and jurisdiction. This article is educational guidance, not legal, medical or financial advice.