170 lines
5.8 KiB
Markdown
170 lines
5.8 KiB
Markdown
# One End‑to‑End Trace (User → Retrieval → Decision → Audit Proof)
|
||
|
||
This document answers:
|
||
|
||
“Show me ONE end‑to‑end trace: user request → retrieval → decision → audit query that proves chain‑of‑custody. Walk me through how I would independently verify it.”
|
||
|
||
It uses a real reference trace so every step is reproducible.
|
||
|
||
---
|
||
|
||
## Trace used (reference bundle)
|
||
|
||
- Trace ID: `016cca78-6f9d-4ffe-aec0-99792d383ca1`
|
||
- Evidence bundle (static mirror):
|
||
- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
|
||
- Evidence bundle (Forgejo raw):
|
||
- `https://git.infrafabric.io/danny/hosted/raw/branch/main/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
|
||
- Tarball SHA256:
|
||
- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8`
|
||
- IF.TTT chain record (PQ‑anchored):
|
||
- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.ttt_chain_record.json`
|
||
- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.ttt_chain_ref.json`
|
||
|
||
---
|
||
|
||
## The trace, end‑to‑end (what actually happened)
|
||
|
||
**User request** (from `payload/trace_payload.json`):
|
||
|
||
```
|
||
In English: Summarize what this system can and cannot prove about an LLM answer, in plain language, and include the Trace line. If you cite clinical context, include [Source: ...] tags.
|
||
```
|
||
|
||
**Retrieval** (from `payload/trace_events.jsonl`, event type `retrieval_done`):
|
||
|
||
- `retrieval_event_id`: `c9b3ebf0-15bb-4c80-8c94-574ba5324954`
|
||
- `retrieved_citations`: `if://citation/3862ce4a-bca5-4090-b5e9-5652fee391ae/v1`
|
||
|
||
**Decision** (from `payload/ttt_signed_record.json`, `guard` object):
|
||
|
||
- `decision`: `allow`
|
||
- `enabled`: `true`
|
||
- `vulnerability`: `0.05`
|
||
|
||
**Audit chain head** (from `payload/ttt_signed_record.json`, `trace_chain`):
|
||
|
||
- `event_count`: `6`
|
||
- `head_hash`: `200c83313376e05577e98d59cd13f2441cccb211f9a9a0927c4ceaf8033827f5`
|
||
|
||
This is the “receipt” that binds the final output hash to the chain of events.
|
||
|
||
---
|
||
|
||
## How to independently verify chain‑of‑custody
|
||
|
||
These steps work without access to the backend, just the public artifacts.
|
||
|
||
### 1) Download and verify the bundle hash
|
||
|
||
```bash
|
||
curl -fsSL -o emo.tar.gz \
|
||
'https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz'
|
||
|
||
sha256sum emo.tar.gz
|
||
# expected: 7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8
|
||
```
|
||
|
||
This proves you have the exact bundle the operator claims to have published.
|
||
|
||
### 2) Run the verifier (manifest + chains + signatures)
|
||
|
||
```bash
|
||
python3 -m venv venv
|
||
./venv/bin/pip install canonicaljson pynacl
|
||
curl -fsSL -o iftrace.py 'https://infrafabric.io/static/hosted/iftrace.py'
|
||
|
||
./venv/bin/python iftrace.py verify emo.tar.gz \
|
||
--expected-sha256 7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8
|
||
```
|
||
|
||
Expected result: `OK verify` plus checks for:
|
||
|
||
- `manifest: ok`
|
||
- `trace_events: ok` (hash chain)
|
||
- `req_seen_head: Ed25519 signature OK`
|
||
- `req_seen: ok` (Merkle root)
|
||
- `if_story: ok` (all event_hash references exist)
|
||
|
||
This proves the trace file, the summary, and the completeness ledger are internally consistent and untampered.
|
||
|
||
### 3) Prove completeness for this trace (REQ_SEEN inclusion)
|
||
|
||
```bash
|
||
tar -xzf emo.tar.gz
|
||
./venv/bin/python iftrace.py verify-inclusion payload/req_seen_inclusion_proof.json
|
||
# expected: OK
|
||
```
|
||
|
||
This proves **this trace’s request** is included in the hourly REQ_SEEN ledger, which is the bounded‑completeness claim.
|
||
|
||
### 4) Inspect the chain step‑by‑step (user → retrieval → decision)
|
||
|
||
```bash
|
||
python3 - <<'PY'
|
||
import json
|
||
from pathlib import Path
|
||
base = Path('payload')
|
||
trace_events = [json.loads(l)['event'] for l in base.joinpath('trace_events.jsonl').read_text().splitlines() if l.strip()]
|
||
for ev in trace_events:
|
||
print(ev['idx'], ev['type'], ev['event_hash'], ev['prev_hash'])
|
||
PY
|
||
```
|
||
|
||
Expected sequence (from this bundle):
|
||
|
||
- `0 req_seen` → `09ce8a52ff9070ee…`
|
||
- `1 request_received` → `f9f93f15b8278a4e…`
|
||
- `2 retrieval_done` → `7ec94771dcaed85c…`
|
||
- `3 prompt_built` → `ec84acc6f4df6edd…`
|
||
- `4 model_done` → `94321445f1b3c560…`
|
||
- `5 trace_finalizing` → `200c83313376e055…`
|
||
|
||
You can see the exact retrieval event (`retrieval_done`) and its hash in the chain.
|
||
|
||
### 5) Verify decision + output binding (signed summary)
|
||
|
||
```bash
|
||
python3 - <<'PY'
|
||
import json
|
||
from pathlib import Path
|
||
rec = json.loads(Path('payload/ttt_signed_record.json').read_text())
|
||
print('trace_id', rec['event']['trace_id'])
|
||
print('guard decision', rec['event']['guard']['decision'])
|
||
print('response_sha256', rec['event']['response_sha256'])
|
||
print('trace_chain', rec['event']['trace_chain'])
|
||
PY
|
||
```
|
||
|
||
This proves that the **decision** and the **final output hash** are bound into the signed summary that also contains the chain head.
|
||
|
||
---
|
||
|
||
## What this proves (and what it doesn’t)
|
||
|
||
**Proves:**
|
||
|
||
- The request, retrieval, decision, and output are tied together in a hash‑chained event log.
|
||
- The output hash is bound to a signed summary that includes the chain head.
|
||
- The request is included in the REQ_SEEN ledger for its hour (bounded completeness).
|
||
- The bundle you verified is exactly the published artifact.
|
||
|
||
**Does not prove:**
|
||
|
||
- That the output is factually correct in the outside world.
|
||
- That requests dropped before the backend witness boundary were seen.
|
||
|
||
---
|
||
|
||
## Optional: registry corroboration (IF.TTT)
|
||
|
||
If you have access to the IF.TTT registry, you can verify the PQ‑anchored record by comparing:
|
||
|
||
- `emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.ttt_chain_ref.json`
|
||
- The tarball SHA + bundle content hashes
|
||
|
||
This corroborates that the published bundle hash was anchored in the registry under the cited handle.
|
||
|
||
---
|
||
|
||
If you want the same walkthrough for the newer v3.2 methodology‑hardening bundle, say the word and I’ll generate the equivalent one‑pager for trace `96700e8e-6a83-445e-86f7-06905c500146`.
|