A benchmark should not decide an AI purchase
Leaderboards are useful for comparing systems under known rules. They become dangerous when they replace the question that matters: does this system improve our process, with our data, without creating an error we cannot detect or afford?
- Nature
- Velqory perspective supported by technical and regulatory references
- Decision
- Select, pilot, contract, or reject an AI system
- Published and verified
- 4 August 2026
- Limit
- The proposed matrix is editorial; criteria and weights depend on the use case
In brief
Velqory's position is simple: a public result opens the evaluation, it does not close it. NIST calls for metrics connected to the deployment context, testing before and during operation, and documented limitations. HELM structures evaluation around scenarios and metrics. MLCommons itself defines rules, workloads, and conditions to make comparisons reproducible. All of this supports the same conclusion: a number is valid inside the test that produced it. Enterprise procurement must add representative data, cost per accepted outcome, error severity, human review, privacy, security, continuity, and exit.
- Use benchmarks to create a shortlist, never to issue the final decision.
- Evaluate the full system, including prompt, retrieval, tools, guardrails, review, and process, not only the model.
- Measure cost per accepted outcome, not only price per token, isolated latency, or average accuracy.
- Before the pilot, define which errors block procurement, who reviews them, and how the system leaves production.
The number does not know the process that will receive the model
A benchmark is an observation under a set of tasks, data, versions, parameters, hardware, and metrics. Its strength lies in that boundary. The mistake begins when an organisation transports the result to another language, population, document, integration, or risk while keeping the same confidence.
NIST AI RMF recommends that measurement starts from mapped risks, uses conditions similar to deployment, documents uncertainty, and continues in production. HELM seeks coverage across scenarios and metrics while recognising that no evaluation covers the entire space. MLCommons improves comparability by fixing rules and workloads. None of these approaches says that a general ranking replaces local testing.
MLPerf Endpoints v0.7, published on 28 July 2026, is particularly useful to buyers because it adds system, cost, power, and execution context. It remains a comparison base, not validation of a company's specific workflow.
From leaderboard to decision
- 1. ScreeningCompare capabilities
Use public results to narrow the field while preserving the test context, version, and conditions.
- 2. PilotTest real work
Run representative, rare, and adversarial cases against a baseline and criteria fixed before testing.
- 3. DecisionAccept capability and failures
Weigh outcomes, errors, data, security, operations, economics, and exit.
- 4. OperationReassess change
Monitor drift and repeat evaluation after material changes to the model, data, tools, or use.
Procurement needs six proofs that a leaderboard does not provide
Build cases from real work, including difficult, rare, and adversarial examples. Preserve a sample that the supplier and implementation team do not know in advance. Separate average quality from material failure: an imperfect summary and a wrong decision about a person do not carry the same cost.
Then measure the end-to-end system. A model can perform well in isolation and fail because retrieval is incomplete, permissions are excessive, workflow latency is high, the provider is unavailable, or human review is impractical.
Decision matrix for an enterprise pilot
| Dimension | Test question | Acceptance evidence |
|---|---|---|
| Outcome | Does the task finish better than the current baseline? | Representative sample, blind comparison, and criterion set before testing |
| Error | Which failure can harm a person, decision, or obligation? | Severity taxonomy, examples, detection, and authority to block |
| Data | What information enters, remains, returns, or supports improvement? | Data flow, retention, subprocessors, access control, and deletion |
| Security | Does the system resist hostile instructions and tool misuse? | Prompt injection, authorisation, isolation, logging, and rollback tests |
| Operations | Can the team review, monitor, recover, and exit? | SLO, review queue, fallback, export, and decommissioning exercise |
| Economics | What does each genuinely accepted outcome cost? | Tokens, infrastructure, integration, review, rework, incidents, and exit cost |
Close the pilot with a decision that can be repeated
Record model and version, configuration, date, test set, conditions, evaluators, raw results, exclusions, and limitations. Define a possible outcome for each state: proceed, proceed with conditions, repeat the test, or reject. Without this, the same demonstration can produce different decisions depending on who is in the room.
Do not turn a pilot into a permanent licence. Track changes to the model, data, process, and risk. For covered high-risk systems, the AI Act requires metrics appropriate to context and consistent performance throughout the lifecycle. Even outside that scope, the logic is prudent: procurement accepts a capability and a set of failures, not merely a brand.
- Fix task, population, language, volume, and consequence before selecting metrics.
- Include a human or current-process baseline under the same quality criterion.
- Separate detectable failure, silent failure, and material failure.
- Re-evaluate after a material change to model, data, tools, or use.
Sources consulted
Consulted on the stated update date.
Opens in a new window.- 01AI RMF Core, Map, Measure and ManageNIST AI Resource Center · GL-NISTAI
- 02HELM, Holistic Evaluation of Language ModelsStanford Center for Research on Foundation Models · GL-STANFORDCRFM
- 03MLPerf Endpoints v0.7, a foundation releaseMLCommons · GL-MLCOMMONS
- 04MLCommons benchmark principles and suitesMLCommons · GL-MLCOMMONS
- 05AI Act Article 15, accuracy, robustness and cybersecurityAI Act Service Desk · EU-AIACTDESK
Participate
Share the analysis. Add context.
Distribute the article through your channel or leave an observation connected to the topic.
Comments
Loading the conversation.