239 lines
8.4 KiB
Markdown
239 lines
8.4 KiB
Markdown
# IF.EMOTION TRACE PROTOCOL v3.0: AUDITABLE DEBUGGING (WITHOUT WISHFUL THINKING)
|
||
|
||
*Alternate title: Debugging Emotion — The Immutable Flight Recorder*
|
||
|
||
**Subject:** End-to-End Traceability, Completeness Witnessing, and PQ-Anchored Evidence Binding
|
||
**Protocol:** IF.TTT (Traceable, Transparent, Trustworthy)
|
||
**Version:** 3.0 (Narrative Annex + Edge Commitments)
|
||
**Date:** 2025-12-21
|
||
**Status:** AUDIT REQUIRED
|
||
**Citation:** `if://whitepaper/emotion/trace-protocol/v3.0`
|
||
|
||
---
|
||
|
||
> Logging is cheap. Evidence is expensive. If your system can’t be verified by someone who doesn’t trust you, it doesn’t have evidence.
|
||
|
||
## Why This Matters (Decision Context)
|
||
|
||
AI incidents are not “debugging problems.” They are dispute problems.
|
||
|
||
If you cannot prove what the system received, what it did, what it returned, and what evidence it claims to have used, you don’t have an incident process. You have a debate club with screenshots.
|
||
|
||
*The goal is not truth. The goal is provenance: what was recorded, when, by which code, under which keys, and whether something was omitted within an explicit boundary.*
|
||
|
||
---
|
||
|
||
## Layered Evidence Stack (Where Guarantees Begin)
|
||
|
||
```mermaid
|
||
flowchart TB
|
||
U[User Browser] -->|HTTPS| E[Edge: TLS + Routing]
|
||
E -->|/oauth2/*| O[oauth2-proxy]
|
||
E -->|/api/*| N[nginx]
|
||
N --> B[Backend Witness Boundary]
|
||
|
||
B --> R[Retrieval / RAG]
|
||
B --> P[Prompt Build]
|
||
B --> M[Model Inference]
|
||
B --> X[Postprocess: citations + trace footer]
|
||
|
||
N --> EDGELOG["Edge attempt log<br/>(JSONL)"]
|
||
B --> T1["REQ_SEEN ledger<br/>(hourly JSONL)"]
|
||
B --> T2["Trace events<br/>(hash chain JSONL)"]
|
||
B --> T3["Signed summary<br/>(output hash + head attestation)"]
|
||
|
||
T1 --> H["Signed Merkle head<br/>(per hour)"]
|
||
T2 --> S["Trace head<br/>(event_hash)"]
|
||
|
||
H --> BUNDLE["Evidence bundle<br/>(tar.gz + manifest)"]
|
||
S --> BUNDLE
|
||
T3 --> BUNDLE
|
||
EDGELOG --> BUNDLE
|
||
|
||
BUNDLE --> REG["IF.TTT registry<br/>(PQ-hybrid anchor)"]
|
||
BUNDLE --> MIRROR["Public static mirror<br/>/static/hosted"]
|
||
```
|
||
|
||
**Interpretation:** integrity begins at the backend witness boundary. Completeness is only meaningful at and after that boundary until edge witnessing is fully cryptographic.
|
||
|
||
---
|
||
|
||
## What This Protocol Guarantees (And What It Refuses To Claim)
|
||
|
||
This system does not try to “prove the model is true.” That is not a meaningful claim for probabilistic generation.
|
||
|
||
This system proves something narrower and more valuable:
|
||
|
||
1) what the system received (as a commitment),
|
||
2) what the system did (trace event chain),
|
||
3) what the system returned (output hash),
|
||
4) what evidence it claims to have used (retrieval IDs + citation handles),
|
||
5) that the resulting artifacts are tamper-evident and portable for external review.
|
||
|
||
If a claim cannot be bound to an artifact, it does not exist.
|
||
|
||
---
|
||
|
||
## Evidence Inventory (What Exists, How You Verify It)
|
||
|
||
| Artifact | What it proves | Verification path |
|
||
|---|---|---|
|
||
| Edge attempt log | request attempts above the backend boundary | inspect JSONL + correlate request id + trace id + body commitment headers |
|
||
| REQ_SEEN ledger | witnessed attempts within backend boundary | Merkle inclusion proof against signed head |
|
||
| REQ_SEEN signed head | integrity of a witnessed batch | verify signature + Merkle root |
|
||
| trace event chain | ordered sequence of system actions | recompute chain (`prev_hash` → `event_hash`) |
|
||
| signed summary | binding between output + trace head + policy decisions | verify signature + output hash |
|
||
| evidence bundle | portable capsule of the above | verify SHA256 + run verifier tool |
|
||
| IF.TTT record | anchored registration (PQ-hybrid) | verify registry chain record + signature |
|
||
|
||
---
|
||
|
||
## The Inversion (Where Teams Usually Get This Wrong)
|
||
|
||
The problem isn’t hallucination.
|
||
|
||
The problem is un-auditable systems pretending to be auditable.
|
||
|
||
Most teams treat tracing as “extra logs.” In probabilistic systems, tracing is the product: it is the only thing that turns a dispute into a verification.
|
||
|
||
---
|
||
|
||
## The Human Constraint (Why This Doesn’t Ship)
|
||
|
||
**The resistance:** “This is too much work; we’ll add observability later.”
|
||
**The psychology:** Shipping dopamine beats audit discipline, and nobody gets promoted for preventing future disputes.
|
||
**The unstick:** Make “no trace, no trust” a deployment gate. If it can’t be verified by an adversarial reviewer, it doesn’t ship.
|
||
|
||
---
|
||
|
||
## Core Protocol (Technical Specification)
|
||
|
||
### Trace ID contract
|
||
|
||
Every request to `/api/chat/completions` receives a Trace ID. This includes denials.
|
||
|
||
- Client request header (optional): `X-IF-Client-Trace: <uuid>` (if present, backend uses it as Trace ID)
|
||
- Response header: `X-IF-Emotion-Trace: <uuid>`
|
||
- Response header: `X-IF-Emotion-Trace-Sig: <sig>` (app-level integrity)
|
||
- User output: final line `Trace: <uuid>`
|
||
|
||
### Body commitment (tamper-evident “what I sent”)
|
||
|
||
The client may send:
|
||
|
||
- `X-IF-Body-Sha256: sha256=<hex>` (or `<hex>`)
|
||
|
||
The backend computes `sha256(request_body_bytes)` and records:
|
||
|
||
- `request_body_sha256` (computed),
|
||
- `claimed_body_sha256` (from header),
|
||
- `body_commit_ok` (match flag).
|
||
|
||
This provides a commitment without storing full request bodies in witness layers.
|
||
|
||
### Edge attempt logging (interim completeness above backend)
|
||
|
||
Because auth/rate limits can deny requests before the backend can write REQ_SEEN, nginx writes an edge attempt log:
|
||
|
||
- File: `/var/log/nginx/edge_req_seen.jsonl`
|
||
- Captures: `req_id`, `status`, `client_trace`, `body_sha256` (header), timing, and (if available) `auth_email`.
|
||
|
||
This is a pragmatic bridge. It is not yet a cryptographic edge ledger.
|
||
|
||
### Backend completeness (REQ_SEEN) and boundary
|
||
|
||
REQ_SEEN records every request attempt that reaches the backend witness boundary as a privacy-preserving commitment:
|
||
|
||
- Ledger: `/opt/if-emotion/data/req_seen/<YYYYMMDDTHH>.jsonl`
|
||
- Signed head: `/opt/if-emotion/data/req_seen/heads/<YYYYMMDDTHH>.json`
|
||
|
||
REQ_SEEN completeness is only valid for requests that reach the backend process. Requests blocked before the backend are out of scope until a cryptographic witness is moved to the edge.
|
||
|
||
### Privacy for witnessed prompts (avoid rainbow tables)
|
||
|
||
REQ_SEEN stores a keyed commitment by default:
|
||
|
||
- `user_text_hash_alg: hmac-sha256`
|
||
- `user_text_hash: hmac_sha256(secret, user_text_bytes)`
|
||
- `user_text_sha256_prefix: <short prefix>` (debug-only correlation)
|
||
|
||
Key source:
|
||
|
||
- `IF_REQ_SEEN_HMAC_KEY` if configured, otherwise falls back to `IF_TTT_SIGNING_SECRET`.
|
||
|
||
### Trace events and signed summary
|
||
|
||
- Trace events: `/opt/if-emotion/data/trace_events.jsonl` (hash chain)
|
||
- Signed summary: `/opt/if-emotion/data/ttt_signed_log.jsonl`
|
||
- Optional payload capture (OAuth-gated): `/opt/if-emotion/data/trace_payloads/<trace_id>.json`
|
||
|
||
Signed completion records include `code_identity` (code SHA + config hash) so “which code ran?” is auditable even without remote attestation.
|
||
|
||
### Post-quantum boundary
|
||
|
||
Evidence bundles are PQ-hybrid signed when registered into IF.TTT:
|
||
|
||
- `pq_status: hybrid-fips204`
|
||
- `pq_algo: ML-DSA-87`
|
||
|
||
Hot-path signatures (Ed25519) are not post-quantum. The correct claim is “PQ-anchored at registry time,” not “PQ everywhere.”
|
||
|
||
---
|
||
|
||
## Verifier Tooling
|
||
|
||
Primary verifier:
|
||
|
||
- `https://infrafabric.io/static/hosted/iftrace.py`
|
||
|
||
Runbook:
|
||
|
||
- `https://infrafabric.io/static/hosted/VERIFY_EMO_TRACE_016cca78-6f9d-4ffe-aec0-99792d383ca1.md`
|
||
|
||
---
|
||
|
||
## Reference Proof Run (Transport + Integrity)
|
||
|
||
Bundle:
|
||
|
||
- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
|
||
|
||
Directory (discovery):
|
||
|
||
- `https://infrafabric.io/static/hosted/`
|
||
|
||
SHA256:
|
||
|
||
- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8`
|
||
|
||
One-line verification:
|
||
|
||
- `curl -fsSL https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz | sha256sum`
|
||
|
||
Expected output:
|
||
|
||
- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8 -`
|
||
|
||
IF.TTT citation handle (PQ-hybrid signed):
|
||
|
||
- `if://citation/c24fe95e-226c-4efc-ba22-5ddcc37ff7d2/v1`
|
||
|
||
---
|
||
|
||
## Narrative Annex (Human-Readable Index)
|
||
|
||
This paper deliberately separates:
|
||
|
||
- Evidence: JSONL + signed summary inside the bundle.
|
||
- Narrative: deterministic projection that points back to evidence by `event_hash`.
|
||
|
||
Annex (public):
|
||
|
||
- `https://infrafabric.io/static/hosted/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md`
|
||
|
||
The narrative is useful during review, but it is not the source of truth.
|
||
|
||
---
|
||
|
||
> The model’s answer is ephemeral. The trace is the product. If you can’t prove what happened, you are not running an AI system — you are running a narrative.
|
||
|