diff --git a/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md b/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md new file mode 100644 index 0000000..57ce7b0 --- /dev/null +++ b/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md @@ -0,0 +1,239 @@ +# IF.EMOTION TRACE PROTOCOL v3.0: AUDITABLE DEBUGGING (WITHOUT WISHFUL THINKING) + +*Alternate title: Debugging Emotion — The Immutable Flight Recorder* + +**Subject:** End-to-End Traceability, Completeness Witnessing, and PQ-Anchored Evidence Binding +**Protocol:** IF.TTT (Traceable, Transparent, Trustworthy) +**Version:** 3.0 (Narrative Annex + Edge Commitments) +**Date:** 2025-12-21 +**Status:** AUDIT REQUIRED +**Citation:** `if://whitepaper/emotion/trace-protocol/v3.0` + +--- + +> Logging is cheap. Evidence is expensive. If your system can’t be verified by someone who doesn’t trust you, it doesn’t have evidence. + +## Why This Matters (Decision Context) + +AI incidents are not “debugging problems.” They are dispute problems. + +If you cannot prove what the system received, what it did, what it returned, and what evidence it claims to have used, you don’t have an incident process. You have a debate club with screenshots. + +*The goal is not truth. The goal is provenance: what was recorded, when, by which code, under which keys, and whether something was omitted within an explicit boundary.* + +--- + +## Layered Evidence Stack (Where Guarantees Begin) + +```mermaid +flowchart TB + U[User Browser] -->|HTTPS| E[Edge: TLS + Routing] + E -->|/oauth2/*| O[oauth2-proxy] + E -->|/api/*| N[nginx] + N --> B[Backend Witness Boundary] + + B --> R[Retrieval / RAG] + B --> P[Prompt Build] + B --> M[Model Inference] + B --> X[Postprocess: citations + trace footer] + + N --> EDGELOG["Edge attempt log
(JSONL)"] + B --> T1["REQ_SEEN ledger
(hourly JSONL)"] + B --> T2["Trace events
(hash chain JSONL)"] + B --> T3["Signed summary
(output hash + head attestation)"] + + T1 --> H["Signed Merkle head
(per hour)"] + T2 --> S["Trace head
(event_hash)"] + + H --> BUNDLE["Evidence bundle
(tar.gz + manifest)"] + S --> BUNDLE + T3 --> BUNDLE + EDGELOG --> BUNDLE + + BUNDLE --> REG["IF.TTT registry
(PQ-hybrid anchor)"] + BUNDLE --> MIRROR["Public static mirror
/static/hosted"] +``` + +**Interpretation:** integrity begins at the backend witness boundary. Completeness is only meaningful at and after that boundary until edge witnessing is fully cryptographic. + +--- + +## What This Protocol Guarantees (And What It Refuses To Claim) + +This system does not try to “prove the model is true.” That is not a meaningful claim for probabilistic generation. + +This system proves something narrower and more valuable: + +1) what the system received (as a commitment), +2) what the system did (trace event chain), +3) what the system returned (output hash), +4) what evidence it claims to have used (retrieval IDs + citation handles), +5) that the resulting artifacts are tamper-evident and portable for external review. + +If a claim cannot be bound to an artifact, it does not exist. + +--- + +## Evidence Inventory (What Exists, How You Verify It) + +| Artifact | What it proves | Verification path | +|---|---|---| +| Edge attempt log | request attempts above the backend boundary | inspect JSONL + correlate request id + trace id + body commitment headers | +| REQ_SEEN ledger | witnessed attempts within backend boundary | Merkle inclusion proof against signed head | +| REQ_SEEN signed head | integrity of a witnessed batch | verify signature + Merkle root | +| trace event chain | ordered sequence of system actions | recompute chain (`prev_hash` → `event_hash`) | +| signed summary | binding between output + trace head + policy decisions | verify signature + output hash | +| evidence bundle | portable capsule of the above | verify SHA256 + run verifier tool | +| IF.TTT record | anchored registration (PQ-hybrid) | verify registry chain record + signature | + +--- + +## The Inversion (Where Teams Usually Get This Wrong) + +The problem isn’t hallucination. + +The problem is un-auditable systems pretending to be auditable. + +Most teams treat tracing as “extra logs.” In probabilistic systems, tracing is the product: it is the only thing that turns a dispute into a verification. + +--- + +## The Human Constraint (Why This Doesn’t Ship) + +**The resistance:** “This is too much work; we’ll add observability later.” +**The psychology:** Shipping dopamine beats audit discipline, and nobody gets promoted for preventing future disputes. +**The unstick:** Make “no trace, no trust” a deployment gate. If it can’t be verified by an adversarial reviewer, it doesn’t ship. + +--- + +## Core Protocol (Technical Specification) + +### Trace ID contract + +Every request to `/api/chat/completions` receives a Trace ID. This includes denials. + +- Client request header (optional): `X-IF-Client-Trace: ` (if present, backend uses it as Trace ID) +- Response header: `X-IF-Emotion-Trace: ` +- Response header: `X-IF-Emotion-Trace-Sig: ` (app-level integrity) +- User output: final line `Trace: ` + +### Body commitment (tamper-evident “what I sent”) + +The client may send: + +- `X-IF-Body-Sha256: sha256=` (or ``) + +The backend computes `sha256(request_body_bytes)` and records: + +- `request_body_sha256` (computed), +- `claimed_body_sha256` (from header), +- `body_commit_ok` (match flag). + +This provides a commitment without storing full request bodies in witness layers. + +### Edge attempt logging (interim completeness above backend) + +Because auth/rate limits can deny requests before the backend can write REQ_SEEN, nginx writes an edge attempt log: + +- File: `/var/log/nginx/edge_req_seen.jsonl` +- Captures: `req_id`, `status`, `client_trace`, `body_sha256` (header), timing, and (if available) `auth_email`. + +This is a pragmatic bridge. It is not yet a cryptographic edge ledger. + +### Backend completeness (REQ_SEEN) and boundary + +REQ_SEEN records every request attempt that reaches the backend witness boundary as a privacy-preserving commitment: + +- Ledger: `/opt/if-emotion/data/req_seen/.jsonl` +- Signed head: `/opt/if-emotion/data/req_seen/heads/.json` + +REQ_SEEN completeness is only valid for requests that reach the backend process. Requests blocked before the backend are out of scope until a cryptographic witness is moved to the edge. + +### Privacy for witnessed prompts (avoid rainbow tables) + +REQ_SEEN stores a keyed commitment by default: + +- `user_text_hash_alg: hmac-sha256` +- `user_text_hash: hmac_sha256(secret, user_text_bytes)` +- `user_text_sha256_prefix: ` (debug-only correlation) + +Key source: + +- `IF_REQ_SEEN_HMAC_KEY` if configured, otherwise falls back to `IF_TTT_SIGNING_SECRET`. + +### Trace events and signed summary + +- Trace events: `/opt/if-emotion/data/trace_events.jsonl` (hash chain) +- Signed summary: `/opt/if-emotion/data/ttt_signed_log.jsonl` +- Optional payload capture (OAuth-gated): `/opt/if-emotion/data/trace_payloads/.json` + +Signed completion records include `code_identity` (code SHA + config hash) so “which code ran?” is auditable even without remote attestation. + +### Post-quantum boundary + +Evidence bundles are PQ-hybrid signed when registered into IF.TTT: + +- `pq_status: hybrid-fips204` +- `pq_algo: ML-DSA-87` + +Hot-path signatures (Ed25519) are not post-quantum. The correct claim is “PQ-anchored at registry time,” not “PQ everywhere.” + +--- + +## Verifier Tooling + +Primary verifier: + +- `https://infrafabric.io/static/hosted/iftrace.py` + +Runbook: + +- `https://infrafabric.io/static/hosted/VERIFY_EMO_TRACE_016cca78-6f9d-4ffe-aec0-99792d383ca1.md` + +--- + +## Reference Proof Run (Transport + Integrity) + +Bundle: + +- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz` + +Directory (discovery): + +- `https://infrafabric.io/static/hosted/` + +SHA256: + +- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8` + +One-line verification: + +- `curl -fsSL https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz | sha256sum` + +Expected output: + +- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8 -` + +IF.TTT citation handle (PQ-hybrid signed): + +- `if://citation/c24fe95e-226c-4efc-ba22-5ddcc37ff7d2/v1` + +--- + +## Narrative Annex (Human-Readable Index) + +This paper deliberately separates: + +- Evidence: JSONL + signed summary inside the bundle. +- Narrative: deterministic projection that points back to evidence by `event_hash`. + +Annex (public): + +- `https://infrafabric.io/static/hosted/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md` + +The narrative is useful during review, but it is not the source of truth. + +--- + +> The model’s answer is ephemeral. The trace is the product. If you can’t prove what happened, you are not running an AI system — you are running a narrative. + diff --git a/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md.sha256 b/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md.sha256 new file mode 100644 index 0000000..c53ca1d --- /dev/null +++ b/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md.sha256 @@ -0,0 +1 @@ +c791c964cac09965339f965b89ca69c2fc47e172d5cbec754a8d51a7827650fc /root/tmp/hosted_repo_update/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v3.0_STYLED.md diff --git a/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md b/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md new file mode 100644 index 0000000..d0506c9 --- /dev/null +++ b/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md @@ -0,0 +1,31 @@ +# IF.story (Reference Trace Narrative) + +This file is a **deterministic narrative projection** for external review. It is **not** evidence. + +- Ground truth: `payload/trace_events.jsonl` + `payload/ttt_signed_record.json` inside the referenced bundle. +- Purpose: give humans a readable index keyed by `event_hash`. + +Reference trace: + +- Trace: `016cca78-6f9d-4ffe-aec0-99792d383ca1` +- Bundle: `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz` + +--- + +# IF.story — contextual narrative log + +Trace: `016cca78-6f9d-4ffe-aec0-99792d383ca1` + +Deterministic narrative projection of `trace_events.jsonl`. Each line includes the `event_hash` anchor. + +- 2025-12-21T07:58:32Z | `req_seen` | REQ_SEEN witnessed; hour=20251221T07 count=2 merkle_root=41aa7aebd66d3b1199b94d5c531111526744254dc8aad1f079e035a725c58aa2 | event_hash=09ce8a52ff9070ee00e0510365107a579d4b8be0f8beb4071aada966a51ed282 +- 2025-12-21T07:58:32Z | `request_received` | Auth+quota succeeded; provider=codex model=gpt-5.2 stream=False user_len=185 auth_ms=3 | event_hash=f9f93f15b8278a4e6ef32e53d2ae02202cd15826ec9bfe90ce10e7bca7bd3b8d +- 2025-12-21T07:58:33Z | `retrieval_done` | Retrieval done; retrieved_count=1 rag_ms=1107 retrieval_event_id=c9b3ebf0-15bb-4c80-8c94-574ba5324954 | event_hash=7ec94771dcaed85c5ab6bbc0d69c14b78010bbac9c3d76fe1e1234e005cbbdb4 +- 2025-12-21T07:58:33Z | `prompt_built` | Prompt built; prompt_sha256=9e4783b70020296fed1f938b12123413552b9c4150574b2d79b9740b440aef20 | event_hash=ec84acc6f4df6edd1ded939be588fc670dc786d8507826cc6f74096c1dbfab16 +- 2025-12-21T07:58:44Z | `model_done` | Model done; provider=codex model=gpt-5.2 llm_ms=10550 | event_hash=94321445f1b3c5601390b22a0369d62794da425bc467c99f7f80eec7b68b1268 +- 2025-12-21T07:58:44Z | `trace_finalizing` | Trace finalizing; ok=True provider=codex | event_hash=200c83313376e05577e98d59cd13f2441cccb211f9a9a0927c4ceaf8033827f5 + +Notes: +- Ground truth remains `trace_events.jsonl` + `ttt_signed_record.json`. +- REQ_SEEN ledger+head are included; public key is `trace_ed25519.pub`. + diff --git a/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md.sha256 b/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md.sha256 new file mode 100644 index 0000000..da12aeb --- /dev/null +++ b/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md.sha256 @@ -0,0 +1 @@ +be7001a6c31c349995529e347010d32cf01bf76ccc8f01a4141c1557f9c5b032 /root/tmp/hosted_repo_update/IF_EMOTION_TRACE_REFERENCE_016cca78_IF_STORY.md