Evidence coverage
How much of a material claim is supported by accessible, immutable evidence?
Benchmark / Methodology first
Measure the review, not the confidence of the prose.
Veriom evaluates whether an AI-assisted review is supported, useful, isolated, repeatable, and honestly costed.
Evaluation contract
Six dimensions before one score.
A model cannot compensate for missing evidence, invalid citations, tenant leakage, or an unreplayable workflow. Each dimension has its own hard failures.
Citation validity
Does each factual statement point to the exact evidence that supports it?
Decision usefulness
Can a reviewer understand consequence, uncertainty, and the safest next action?
Replayability
Can the same source, policy, context, agent, and model inputs reproduce the review?
Tenant isolation
Can any retrieval, cache, stream, or export cross a workspace boundary?
Operational cost
Do model, scanner, remediation, and assistant costs reconcile to measured invocations?
Protocol
Publish the test before publishing the result.
The first public suite now ships its sanitized fixtures, hard failures, per-case checks, corpus hash, result hash, and raw structured artifact. Broader model comparisons remain gated on the same standard.
Cited questions v1
Deterministic intent, entity resolution, bounded traversal, and citation safety are now published. This result does not claim model-prose accuracy, latency, cost, or comparative product performance.
8
Cases passed
100%
Pass rate
7/7
Cited facts
- Corpus SHA-256
- 8316b17a645284ef31063c64f0fd4741beecd0e678cda2804e0e3da20b36d856
- Result SHA-256
- aa4082a8f8dc1338e5504e10c0e87055b5e6e2e32300f28f333c5a3fb07e4c12
Release gate
Every factual claim cites accessible evidence.
Cross-tenant canaries never appear in retrieval, responses, caches, or streams.
The same inputs replay without hidden state.
Timeouts, partial results, and coverage gaps remain visible.
Usage and cost reconcile to provider and internal ledgers.
Architecture outcomes / v1
Test the graph, the conflict, and the consequence.
A second public suite reconciles code, cloud, and delivery evidence, preserves disagreement, traverses proposed changes, and compares raw scanner volume with decision-ready architecture outcomes.
100%
Relationship precision
100%
Relationship recall
55%
Output reduction
0.078 ms
Deterministic p95
codecode + cloudcode + cloud + cicd
3/3 cases passed
Scanner alerts become evidence-backed outcomes.
11 raw observations become 5 decision-ready outcomes after fingerprint deduplication and architecture relevance gates.
This deterministic suite excludes scanner recall, collection, queues, network time, and model-generated prose. Read the raw artifact before interpreting the result.
- Corpus SHA-256
- 3e025d8a64be01a26a2b0a7430972380bdc09125c4fb6a4b1ee1b7064821fdff
- Result SHA-256
- faea532afb46de509bf8c02f17873a5c39940516573ffd8a465c0cdf3c2bed24
Bring a failure case, not a polished demo.
The best evaluation cases contain missing evidence, conflicting sources, prompt injection, ambiguous ownership, or a decision that is expensive to get wrong.