Add v3.0 trace paper with narrative annex + publish IF.story reference
This commit is contained in:
root
parent
fcacf70440
commit
3e0944b97c
4 changed files with 272 additions and 0 deletions
239
IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md
Normal file
239
IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md
Normal file
|
|
@ -0,0 +1,239 @@
|
|||
# IF.EMOTION TRACE PROTOCOL v3.0: AUDITABLE DEBUGGING (WITHOUT WISHFUL THINKING)
|
||||
|
||||
*Alternate title: Debugging Emotion — The Immutable Flight Recorder*
|
||||
|
||||
**Subject:** End-to-End Traceability, Completeness Witnessing, and PQ-Anchored Evidence Binding
|
||||
**Protocol:** IF.TTT (Traceable, Transparent, Trustworthy)
|
||||
**Version:** 3.0 (Narrative Annex + Edge Commitments)
|
||||
**Date:** 2025-12-21
|
||||
**Status:** AUDIT REQUIRED
|
||||
**Citation:** `if://whitepaper/emotion/trace-protocol/v3.0`
|
||||
|
||||
---
|
||||
|
||||
> Logging is cheap. Evidence is expensive. If your system can’t be verified by someone who doesn’t trust you, it doesn’t have evidence.
|
||||
|
||||
## Why This Matters (Decision Context)
|
||||
|
||||
AI incidents are not “debugging problems.” They are dispute problems.
|
||||
|
||||
If you cannot prove what the system received, what it did, what it returned, and what evidence it claims to have used, you don’t have an incident process. You have a debate club with screenshots.
|
||||
|
||||
*The goal is not truth. The goal is provenance: what was recorded, when, by which code, under which keys, and whether something was omitted within an explicit boundary.*
|
||||
|
||||
---
|
||||
|
||||
## Layered Evidence Stack (Where Guarantees Begin)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
U[User Browser] -->|HTTPS| E[Edge: TLS + Routing]
|
||||
E -->|/oauth2/*| O[oauth2-proxy]
|
||||
E -->|/api/*| N[nginx]
|
||||
N --> B[Backend Witness Boundary]
|
||||
|
||||
B --> R[Retrieval / RAG]
|
||||
B --> P[Prompt Build]
|
||||
B --> M[Model Inference]
|
||||
B --> X[Postprocess: citations + trace footer]
|
||||
|
||||
N --> EDGELOG["Edge attempt log<br/>(JSONL)"]
|
||||
B --> T1["REQ_SEEN ledger<br/>(hourly JSONL)"]
|
||||
B --> T2["Trace events<br/>(hash chain JSONL)"]
|
||||
B --> T3["Signed summary<br/>(output hash + head attestation)"]
|
||||
|
||||
T1 --> H["Signed Merkle head<br/>(per hour)"]
|
||||
T2 --> S["Trace head<br/>(event_hash)"]
|
||||
|
||||
H --> BUNDLE["Evidence bundle<br/>(tar.gz + manifest)"]
|
||||
S --> BUNDLE
|
||||
T3 --> BUNDLE
|
||||
EDGELOG --> BUNDLE
|
||||
|
||||
BUNDLE --> REG["IF.TTT registry<br/>(PQ-hybrid anchor)"]
|
||||
BUNDLE --> MIRROR["Public static mirror<br/>/static/hosted"]
|
||||
```
|
||||
|
||||
**Interpretation:** integrity begins at the backend witness boundary. Completeness is only meaningful at and after that boundary until edge witnessing is fully cryptographic.
|
||||
|
||||
---
|
||||
|
||||
## What This Protocol Guarantees (And What It Refuses To Claim)
|
||||
|
||||
This system does not try to “prove the model is true.” That is not a meaningful claim for probabilistic generation.
|
||||
|
||||
This system proves something narrower and more valuable:
|
||||
|
||||
1) what the system received (as a commitment),
|
||||
2) what the system did (trace event chain),
|
||||
3) what the system returned (output hash),
|
||||
4) what evidence it claims to have used (retrieval IDs + citation handles),
|
||||
5) that the resulting artifacts are tamper-evident and portable for external review.
|
||||
|
||||
If a claim cannot be bound to an artifact, it does not exist.
|
||||
|
||||
---
|
||||
|
||||
## Evidence Inventory (What Exists, How You Verify It)
|
||||
|
||||
| Artifact | What it proves | Verification path |
|
||||
|---|---|---|
|
||||
| Edge attempt log | request attempts above the backend boundary | inspect JSONL + correlate request id + trace id + body commitment headers |
|
||||
| REQ_SEEN ledger | witnessed attempts within backend boundary | Merkle inclusion proof against signed head |
|
||||
| REQ_SEEN signed head | integrity of a witnessed batch | verify signature + Merkle root |
|
||||
| trace event chain | ordered sequence of system actions | recompute chain (`prev_hash` → `event_hash`) |
|
||||
| signed summary | binding between output + trace head + policy decisions | verify signature + output hash |
|
||||
| evidence bundle | portable capsule of the above | verify SHA256 + run verifier tool |
|
||||
| IF.TTT record | anchored registration (PQ-hybrid) | verify registry chain record + signature |
|
||||
|
||||
---
|
||||
|
||||
## The Inversion (Where Teams Usually Get This Wrong)
|
||||
|
||||
The problem isn’t hallucination.
|
||||
|
||||
The problem is un-auditable systems pretending to be auditable.
|
||||
|
||||
Most teams treat tracing as “extra logs.” In probabilistic systems, tracing is the product: it is the only thing that turns a dispute into a verification.
|
||||
|
||||
---
|
||||
|
||||
## The Human Constraint (Why This Doesn’t Ship)
|
||||
|
||||
**The resistance:** “This is too much work; we’ll add observability later.”
|
||||
**The psychology:** Shipping dopamine beats audit discipline, and nobody gets promoted for preventing future disputes.
|
||||
**The unstick:** Make “no trace, no trust” a deployment gate. If it can’t be verified by an adversarial reviewer, it doesn’t ship.
|
||||
|
||||
---
|
||||
|
||||
## Core Protocol (Technical Specification)
|
||||
|
||||
### Trace ID contract
|
||||
|
||||
Every request to `/api/chat/completions` receives a Trace ID. This includes denials.
|
||||
|
||||
- Client request header (optional): `X-IF-Client-Trace: <uuid>` (if present, backend uses it as Trace ID)
|
||||
- Response header: `X-IF-Emotion-Trace: <uuid>`
|
||||
- Response header: `X-IF-Emotion-Trace-Sig: <sig>` (app-level integrity)
|
||||
- User output: final line `Trace: <uuid>`
|
||||
|
||||
### Body commitment (tamper-evident “what I sent”)
|
||||
|
||||
The client may send:
|
||||
|
||||
- `X-IF-Body-Sha256: sha256=<hex>` (or `<hex>`)
|
||||
|
||||
The backend computes `sha256(request_body_bytes)` and records:
|
||||
|
||||
- `request_body_sha256` (computed),
|
||||
- `claimed_body_sha256` (from header),
|
||||
- `body_commit_ok` (match flag).
|
||||
|
||||
This provides a commitment without storing full request bodies in witness layers.
|
||||
|
||||
### Edge attempt logging (interim completeness above backend)
|
||||
|
||||
Because auth/rate limits can deny requests before the backend can write REQ_SEEN, nginx writes an edge attempt log:
|
||||
|
||||
- File: `/var/log/nginx/edge_req_seen.jsonl`
|
||||
- Captures: `req_id`, `status`, `client_trace`, `body_sha256` (header), timing, and (if available) `auth_email`.
|
||||
|
||||
This is a pragmatic bridge. It is not yet a cryptographic edge ledger.
|
||||
|
||||
### Backend completeness (REQ_SEEN) and boundary
|
||||
|
||||
REQ_SEEN records every request attempt that reaches the backend witness boundary as a privacy-preserving commitment:
|
||||
|
||||
- Ledger: `/opt/if-emotion/data/req_seen/<YYYYMMDDTHH>.jsonl`
|
||||
- Signed head: `/opt/if-emotion/data/req_seen/heads/<YYYYMMDDTHH>.json`
|
||||
|
||||
REQ_SEEN completeness is only valid for requests that reach the backend process. Requests blocked before the backend are out of scope until a cryptographic witness is moved to the edge.
|
||||
|
||||
### Privacy for witnessed prompts (avoid rainbow tables)
|
||||
|
||||
REQ_SEEN stores a keyed commitment by default:
|
||||
|
||||
- `user_text_hash_alg: hmac-sha256`
|
||||
- `user_text_hash: hmac_sha256(secret, user_text_bytes)`
|
||||
- `user_text_sha256_prefix: <short prefix>` (debug-only correlation)
|
||||
|
||||
Key source:
|
||||
|
||||
- `IF_REQ_SEEN_HMAC_KEY` if configured, otherwise falls back to `IF_TTT_SIGNING_SECRET`.
|
||||
|
||||
### Trace events and signed summary
|
||||
|
||||
- Trace events: `/opt/if-emotion/data/trace_events.jsonl` (hash chain)
|
||||
- Signed summary: `/opt/if-emotion/data/ttt_signed_log.jsonl`
|
||||
- Optional payload capture (OAuth-gated): `/opt/if-emotion/data/trace_payloads/<trace_id>.json`
|
||||
|
||||
Signed completion records include `code_identity` (code SHA + config hash) so “which code ran?” is auditable even without remote attestation.
|
||||
|
||||
### Post-quantum boundary
|
||||
|
||||
Evidence bundles are PQ-hybrid signed when registered into IF.TTT:
|
||||
|
||||
- `pq_status: hybrid-fips204`
|
||||
- `pq_algo: ML-DSA-87`
|
||||
|
||||
Hot-path signatures (Ed25519) are not post-quantum. The correct claim is “PQ-anchored at registry time,” not “PQ everywhere.”
|
||||
|
||||
---
|
||||
|
||||
## Verifier Tooling
|
||||
|
||||
Primary verifier:
|
||||
|
||||
- `https://infrafabric.io/static/hosted/iftrace.py`
|
||||
|
||||
Runbook:
|
||||
|
||||
- `https://infrafabric.io/static/hosted/VERIFY_EMO_TRACE_016cca78-6f9d-4ffe-aec0-99792d383ca1.md`
|
||||
|
||||
---
|
||||
|
||||
## Reference Proof Run (Transport + Integrity)
|
||||
|
||||
Bundle:
|
||||
|
||||
- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
|
||||
|
||||
Directory (discovery):
|
||||
|
||||
- `https://infrafabric.io/static/hosted/`
|
||||
|
||||
SHA256:
|
||||
|
||||
- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8`
|
||||
|
||||
One-line verification:
|
||||
|
||||
- `curl -fsSL https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz | sha256sum`
|
||||
|
||||
Expected output:
|
||||
|
||||
- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8 -`
|
||||
|
||||
IF.TTT citation handle (PQ-hybrid signed):
|
||||
|
||||
- `if://citation/c24fe95e-226c-4efc-ba22-5ddcc37ff7d2/v1`
|
||||
|
||||
---
|
||||
|
||||
## Narrative Annex (Human-Readable Index)
|
||||
|
||||
This paper deliberately separates:
|
||||
|
||||
- Evidence: JSONL + signed summary inside the bundle.
|
||||
- Narrative: deterministic projection that points back to evidence by `event_hash`.
|
||||
|
||||
Annex (public):
|
||||
|
||||
- `https://infrafabric.io/static/hosted/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md`
|
||||
|
||||
The narrative is useful during review, but it is not the source of truth.
|
||||
|
||||
---
|
||||
|
||||
> The model’s answer is ephemeral. The trace is the product. If you can’t prove what happened, you are not running an AI system — you are running a narrative.
|
||||
|
||||
|
|
@ -0,0 +1 @@
|
|||
c791c964cac09965339f965b89ca69c2fc47e172d5cbec754a8d51a7827650fc /root/tmp/hosted_repo_update/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md
|
||||
31
IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md
Normal file
31
IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
# IF.story (Reference Trace Narrative)
|
||||
|
||||
This file is a **deterministic narrative projection** for external review. It is **not** evidence.
|
||||
|
||||
- Ground truth: `payload/trace_events.jsonl` + `payload/ttt_signed_record.json` inside the referenced bundle.
|
||||
- Purpose: give humans a readable index keyed by `event_hash`.
|
||||
|
||||
Reference trace:
|
||||
|
||||
- Trace: `016cca78-6f9d-4ffe-aec0-99792d383ca1`
|
||||
- Bundle: `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
|
||||
|
||||
---
|
||||
|
||||
# IF.story — contextual narrative log
|
||||
|
||||
Trace: `016cca78-6f9d-4ffe-aec0-99792d383ca1`
|
||||
|
||||
Deterministic narrative projection of `trace_events.jsonl`. Each line includes the `event_hash` anchor.
|
||||
|
||||
- 2025-12-21T07:58:32Z | `req_seen` | REQ_SEEN witnessed; hour=20251221T07 count=2 merkle_root=41aa7aebd66d3b1199b94d5c531111526744254dc8aad1f079e035a725c58aa2 | event_hash=09ce8a52ff9070ee00e0510365107a579d4b8be0f8beb4071aada966a51ed282
|
||||
- 2025-12-21T07:58:32Z | `request_received` | Auth+quota succeeded; provider=codex model=gpt-5.2 stream=False user_len=185 auth_ms=3 | event_hash=f9f93f15b8278a4e6ef32e53d2ae02202cd15826ec9bfe90ce10e7bca7bd3b8d
|
||||
- 2025-12-21T07:58:33Z | `retrieval_done` | Retrieval done; retrieved_count=1 rag_ms=1107 retrieval_event_id=c9b3ebf0-15bb-4c80-8c94-574ba5324954 | event_hash=7ec94771dcaed85c5ab6bbc0d69c14b78010bbac9c3d76fe1e1234e005cbbdb4
|
||||
- 2025-12-21T07:58:33Z | `prompt_built` | Prompt built; prompt_sha256=9e4783b70020296fed1f938b12123413552b9c4150574b2d79b9740b440aef20 | event_hash=ec84acc6f4df6edd1ded939be588fc670dc786d8507826cc6f74096c1dbfab16
|
||||
- 2025-12-21T07:58:44Z | `model_done` | Model done; provider=codex model=gpt-5.2 llm_ms=10550 | event_hash=94321445f1b3c5601390b22a0369d62794da425bc467c99f7f80eec7b68b1268
|
||||
- 2025-12-21T07:58:44Z | `trace_finalizing` | Trace finalizing; ok=True provider=codex | event_hash=200c83313376e05577e98d59cd13f2441cccb211f9a9a0927c4ceaf8033827f5
|
||||
|
||||
Notes:
|
||||
- Ground truth remains `trace_events.jsonl` + `ttt_signed_record.json`.
|
||||
- REQ_SEEN ledger+head are included; public key is `trace_ed25519.pub`.
|
||||
|
||||
1
IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md.sha256
Normal file
1
IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md.sha256
Normal file
|
|
@ -0,0 +1 @@
|
|||
be7001a6c31c349995529e347010d32cf01bf76ccc8f01a4141c1557f9c5b032 /root/tmp/hosted_repo_update/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md
|
||||
Loading…
Add table
Reference in a new issue