KOREV AI research note · version 1.0.2
Verifiable Decision Records: proving that an AI history is complete
Page updated on
In short
A verifiable decision record lets an examiner detect that an AI decision history has been altered, truncated or left incomplete, without having to trust the local database of the party presenting it. KOREV AI implemented and tested it in KOREV Evidence and PRISM: a verifier classifies each exported record as VERIFIED, INCOMPLETE or FAILED.
The property is deliberately narrow. It does not prove that the content is true, that the system is compliant, or that a human review actually took place.
Download the full note (PDF, 11 pages)Definition
- Verifiable decision record
- A signed export of the history of an AI process, in which a verifier can check record integrity, history continuity and the completeness of admitted operations, using trust keys supplied separately from the record.
The problem: a history can be internally consistent and still be incomplete
A conventional log answers "what did this system record?" An audit asks something stronger: "what should have been in the record, and can the party producing it silently remove part of the history?"
Per-record signatures do not solve that problem by themselves. An operator can delete an inconvenient tail, rebuild a new internally valid history and sign it again with a key it controls. Hash chaining detects an edit relative to a known chain, but it does not tell an external examiner whether the chain was shortened before being presented.
Three distinct properties
| Property | Question | What secures it |
|---|---|---|
| Record integrity | Are the bytes presented the bytes that were committed? | Payload digests and the producer's signature |
| History continuity | Have the ordering and links between events been rewritten or truncated? | Sequence numbers, previous-event hash, a witness checkpoint |
| Completeness after admission | Is every admitted operation still present and has it reached a terminal state? | Admission recorded before execution, witness acknowledgement, reconciliation to a terminal state |
The third property is the main contribution of the note.
Log, hash chain, verifiable record: the differences
| Criterion | Application log | Signed hash chain | Verifiable decision record |
|---|---|---|---|
| Detects an edited record | No | Yes | Yes |
| Detects a truncated tail | No | No, if the producer re-signs | Yes, against the witness state |
| Detects an admitted then omitted operation | No | No | Yes |
| Shows an interrupted operation | Rarely | No | Yes, as unresolved |
| Trust keys independent of the producer | No | No | Yes, supplied separately to the verifier |
| Proves the decision was right | No | No | No |
How it works
- Admission before execution. The operation is written in a local transaction (identity, metadata, payload commitments) before the model or tool runs. A crash leaves an unresolved admission rather than an invisible action.
- Ordered journal. Each event carries a sequence number, the previous-event hash, its outcome and a digest of separately retained payload bytes.
- Separately keyed witness. A separate process, with its own key and state, accepts only an exact extension of the history it already recognises and returns a signed checkpoint.
- Freshness. The verifier rejects an old checkpoint, even a correctly signed one, if it does not match the current witness state.
- Reconciliation. Every admission must reach an explicit terminal state: success, refusal, error, cancellation or superseded. Otherwise it stays visible as unresolved.
- Capture before release. Selected outputs (model results, tool results, UI-visible output, HTTP responses) are persisted and acknowledged before release.
- Export and verification. The exported record contains the ordered history and a signed inventory. The verifier receives the trust keys and expected scope separately.
The three verdicts
| Verdict | Meaning |
|---|---|
| VERIFIED | Inventory, bytes and current witness state reconcile; no required execution remains unresolved |
| INCOMPLETE | Freshness or reconciliation is missing, or an admitted operation remains open |
| FAILED | Signature, scope, sequence, hash, freshness or inventory verification fails |
These are technical verification states, not regulatory labels.
What was tested
- Two synthetic batches of 10 and 100 recommendations, each run through 11 predetermined scenarios.
- Across the 22 recorded observations, every observed verdict matches the expected verdict.
- A targeted suite of 67 tests: 24 inherited from the earlier work and 43 added for durable capture and witness behaviour.
- External LLM calls were avoided so that the scenarios remain deterministic.
| Scenario | Expected verdict |
|---|---|
| Intact live checkpoint | VERIFIED |
| Reconciled cancellation | VERIFIED |
| Missing witness | INCOMPLETE |
| Interrupted admission | INCOMPLETE |
| Stale checkpoint | FAILED |
| Old receipt under a fresh challenge | FAILED |
| Truncated history | FAILED |
| Rewritten admission | FAILED |
| Omitted refusal | FAILED |
| Invalid exporter signature | FAILED |
| Old bundle against a newer witness state | FAILED |
These results establish detection for the specified attacks and boundary only.
What this work does not demonstrate
- Certification or legal conformity.
- The factual truth of a source or the correctness of a model.
- The business adequacy of a decision.
- The effectiveness of a human review, or even that it took place.
- Completeness outside the declared capture boundaries.
- Operational independence of the witness: in the experiments it is process- and key-separated, but administered locally.
- Protection against a joint compromise of producer and witness without an externally retained reference.
- Production-scale latency or throughput.
- Reproduction by an independent third party.
Relationship to the AI Act
The AI Act requires automatic event logging for high-risk AI systems (Article 12) and requires providers and deployers to keep the logs under their control for at least six months, unless otherwise provided (Articles 19 and 26(6)). Source: Regulation (EU) 2024/1689.
These requirements make trustworthy records useful. They do not prescribe this protocol, and a VERIFIED verdict is not evidence of compliance. See also: AI Act by design.
Open work
- Have the witness administered by a separate party, with separate key custody.
- Retain checkpoints externally and strengthen guarantees against divergent histories shown to different auditors.
- Publish a redacted reproduction pack (minimal verifier, synthetic fixtures, expected verdicts, public keys) and obtain an external rerun.
- Measure latency, throughput and storage growth in required-capture mode.
- Formalise key rotation and revocation.
Glossary
- Admission: recording an operation before it runs.
- Witness: a process with its own key that attests to the state of the history it recognises.
- Checkpoint: a statement signed by the witness, binding scope, sequence, history head, time and challenge.
- Terminal state: the explicit outcome of an admitted operation (success, refusal, error, cancellation, superseded).
- Reconciliation: checking that every admission has a terminal state.
- Instrumented boundary: a point in the system where capture is actually in place.