Add v2.7 styled trace paper applying presentation stack

IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.7_STYLED.md

@@ -0,0 +1,274 @@
+# IF.EMOTION TRACE PROTOCOL v2.7: AUDITABLE DEBUGGING (WITHOUT WISHFUL THINKING)
+
+**Subject:** End-to-End Traceability, Completeness Witnessing, and PQ-Anchored Evidence Binding  
+**Protocol:** IF.TTT (Traceable, Transparent, Trustworthy)  
+**Version:** 2.7 (Presentation Stack Applied)  
+**Date:** 2025-12-21  
+**Status:** AUDIT REQUIRED  
+**Citation:** `if://whitepaper/emotion/trace-protocol/v2.7`  
+
+---
+
+> Logging is cheap. Evidence is expensive. This protocol exists to force you to pay the expensive part on purpose.
+
+## Why This Matters (Decision Context)
+
+If your AI system can’t prove what happened, you don’t have an incident process. You have a debate club.
+
+When an external reviewer sees “415”, “no bytes retrieved”, or “the tool couldn’t download it”, the risk is not the download failure. The risk is that your entire chain-of-custody claim becomes non-falsifiable because the evidence is inaccessible or non-deterministic. This paper pins the boundary, defines what can be proven, and provides a verification path that does not require trusting the operator.
+
+*This is the same class of problem solved by append-only transparency logs: not “truth,” but provable provenance of what was recorded, and (within an explicit boundary) whether something was omitted.*
+
+---
+
+## 0) Layered Stack (Where Guarantees Live)
+
+```mermaid
+flowchart TB
+  U[User Browser] -->|HTTPS| E[Edge: TLS + Routing]
+  E -->|/oauth2/*| O[oauth2-proxy]
+  E -->|/api/*| N[nginx]
+  N --> B[Backend Witness Boundary]
+
+  B --> R[Retrieval / RAG]
+  B --> P[Prompt Build]
+  B --> M[Model Inference]
+  B --> X[Postprocess: citations + trace footer]
+
+  B --> T1["REQ_SEEN ledger<br/>(hourly JSONL)"]
+  B --> T2["Trace events<br/>(hash chain JSONL)"]
+  B --> T3["Signed summary<br/>(output hash + head attestation)"]
+
+  T1 --> H["Signed Merkle head<br/>(per hour)"]
+  T2 --> S["Trace head<br/>(event_hash)"]
+
+  H --> BUNDLE["Evidence bundle<br/>(tar.gz + manifest)"]
+  S --> BUNDLE
+  T3 --> BUNDLE
+
+  BUNDLE --> REG["IF.TTT registry<br/>(PQ-hybrid anchor)"]
+  BUNDLE --> MIRROR["Public static mirror<br/>/static/hosted"]
+```
+
+**Interpretation:** integrity starts at the backend witness boundary. Completeness (REQ_SEEN) is only meaningful at and after that boundary until edge witnessing is implemented.
+
+---
+
+## 1) What This Protocol Actually Guarantees
+
+This system does not try to “prove the model is true.” That is not a meaningful claim for probabilistic generation.
+
+This system proves something narrower and more valuable:
+
+1) what the system received (as a commitment),  
+2) what the system did (trace event chain),  
+3) what the system returned (output hash),  
+4) what evidence it claims to have used (retrieval IDs + citation handles),  
+5) that the resulting artifacts are tamper-evident and portable for external review.
+
+If a claim cannot be bound to an artifact, it does not exist.
+
+---
+
+## 2) Evidence Inventory (What Exists, Where It Lives, How You Verify It)
+
+| Artifact | What it proves | How to verify (independent) |
+|---|---|---|
+| REQ_SEEN hourly ledger | request attempts within the backend boundary | inclusion proof against signed Merkle head |
+| REQ_SEEN signed head | integrity of a batch of witnessed requests | verify signature + Merkle root |
+| trace event chain | ordered sequence of system actions (post-witness) | recompute `event_hash` and `prev_hash` links |
+| signed summary | binding between output + trace head + policy decisions | verify signature + output hash |
+| evidence bundle (tar.gz) | portable capsule of the above | verify SHA256 + run verifier tool |
+| IF.TTT registry record | anchored registration (PQ-hybrid) | verify registry chain record + signature |
+
+---
+
+## 3) The Transport Problem (And Why We Hardened It)
+
+The most sophisticated cryptographic protocol is still useless if reviewers can’t retrieve the artifact. The correct response is not “tell them to trust us.” It’s to make the transport layer boring and compatible.
+
+Transport hardening applied:
+
+- The public mirror serves `*.tar.gz` with `Content-Disposition: attachment`.
+- The public mirror serves `*.tar.gz` with `Content-Type: application/octet-stream` for maximum client compatibility.
+- The public mirror directory is browsable for discovery and typo recovery: `https://infrafabric.io/static/hosted/`.
+
+---
+
+## 4) The Real Failure Mode (Contrarian Reframe)
+
+The problem isn’t hallucination.
+
+The problem is *un-auditable systems pretending to be auditable*. Most teams treat tracing as “extra logs.” In probabilistic systems, tracing is the product: it is the only thing that turns a dispute into a verification.
+
+---
+
+## 5) The Human Constraint (Why Teams Avoid This)
+
+**The resistance:** “This is too much work; we’ll add observability later.”  
+**The psychology:** Shipping dopamine beats audit discipline, and nobody gets promoted for preventing future disputes.  
+**The unstick:** Make “no trace, no trust” a deployment gate. If it can’t be verified by an adversarial reviewer, it doesn’t ship.
+
+---
+
+## 6) Core Protocol (Technical Specification)
+
+### 6.1 Trace ID Contract (Non-Negotiable)
+
+Every request to `/api/chat/completions` receives a Trace ID. This includes denials.
+
+**Surfaces:**
+
+- **Header:** `X-IF-Emotion-Trace: <uuid>`
+- **Header:** `X-IF-Emotion-Trace-Sig: <sig>` (app-level integrity)
+- **User output:** final line `Trace: <uuid>`
+
+The Trace ID is the support ticket, the incident handle, and the audit join key.
+
+### 6.2 Completeness: REQ_SEEN Witness Ledger (And Its Boundary)
+
+REQ_SEEN records every request attempt that reaches the backend witness boundary as a privacy-preserving commitment:
+
+- `user_text_sha256`, `user_len`, decision/reason, and `leaf_hash`
+
+It writes:
+
+- Hour ledger: `/opt/if-emotion/data/req_seen/<YYYYMMDDTHH>.jsonl`
+- Signed Merkle head: `/opt/if-emotion/data/req_seen/heads/<YYYYMMDDTHH>.json`
+
+REQ_SEEN completeness is only valid for requests that reach the backend process. Requests blocked before the backend are out of scope until the witness is moved to the edge proxy.
+
+### 6.3 Trace Events: Hash Chain + Immediate Head Attestation
+
+Trace events are stored as a hash chain in:
+
+- `/opt/if-emotion/data/trace_events.jsonl`
+
+Each event includes:
+
+- `prev_hash` pointer
+- `event_hash` computed as `sha256(prev_hash || canonical_json(event_without_event_hash))`
+
+The trace head is attested in the signed completion record with an app-level Ed25519 signature so integrity can be verified immediately.
+
+### 6.4 Canonicalization: Stable Bytes or No Protocol
+
+v2.x mandates canonical JSON bytes for hashing/signing:
+
+- Primary: `canonicaljson.encode_canonical_json(obj)`
+- Fallback: stable JSON serialization (`sort_keys`, fixed separators, UTF-8)
+
+### 6.5 Key Management (POC-Grade Today, Audit-Grade Tomorrow)
+
+- App Ed25519 signing key:
+  - Private: `/opt/if-emotion/data/trace_ed25519.key` (0600)
+  - Public: `/opt/if-emotion/data/trace_ed25519.pub` (shipped in bundles)
+  - Key ID: `ed25519-app-v1`
+
+### 6.6 Post-Quantum: What Is PQ Today (And What Isn’t)
+
+Evidence bundles are PQ-hybrid signed when registered into IF.TTT. The IF.TTT registry record includes:
+
+- `pq_status: hybrid-fips204`
+- `pq_algo: ML-DSA-87`
+
+Hot-path app signatures (Ed25519) are not post-quantum. The correct claim is “PQ-anchored at registry time,” not “PQ everywhere.”
+
+### 6.7 IF.story: Readability Without Evidence Drift
+
+IF.story is a deterministic narrative projection of `trace_events.jsonl`.
+
+It is not evidence. It is an index.
+
+Each IF.story line includes the `event_hash` anchor, and auditors should verify those anchors against the raw JSONL.
+
+---
+
+## 7) Verifier Tooling (Independent Checks)
+
+Verifier:
+
+- `iftrace.py verify <tar.gz> --expected-sha256 <sha>`
+
+Merkle inclusion proof (REQ_SEEN):
+
+- `iftrace.py prove-inclusion --ledger <req_seen_hour.jsonl> --head <req_seen_head.json> --trace-id <uuid>`
+- `iftrace.py verify-inclusion <proof.json>`
+
+Checksum rules (important):
+
+- `sha256s.txt` intentionally excludes itself and `manifest.json` to avoid self-referential checksum traps.
+
+---
+
+## 8) Threat Model and Limitations (Explicit)
+
+### A) Truncation and external anchoring
+
+Hash chains detect edits. They do not prevent truncation unless head hashes are anchored externally or independently cached.
+
+Current mitigation:
+
+- IF.TTT registration anchors the tarball hash into a separate chain.
+
+Remaining requirement for certification:
+
+- scheduled external anchoring of IF.TTT head hashes to a public append-only log.
+
+### B) Clock integrity
+
+Timestamps are derived from system clocks and are not trusted for cryptographic time.
+
+Ordering is guaranteed by hash chain indices and hash pointers, not by wall-clock truth.
+
+### C) Code integrity
+
+Hash chains detect post-hoc tampering. They do not prevent a modified binary from choosing not to record.
+
+Certification path:
+
+- signed deploy attestations (image digest + config hash) bound into IF.TTT
+- optional remote attestation
+
+---
+
+## 9) Reference Proof Run (v2.1) — Public Transport Verified
+
+Trace ID:
+
+- `016cca78-6f9d-4ffe-aec0-99792d383ca1`
+
+Preferred download URL (static mirror for external reviewers):
+
+- `https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
+
+Static mirror directory:
+
+- `https://infrafabric.io/static/hosted/`
+
+Alternate download URL (Forgejo raw):
+
+- `https://git.infrafabric.io/danny/hosted/raw/branch/main/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz`
+
+Tarball SHA256:
+
+- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8`
+
+One-line verifier (shell):
+
+- `curl -fsSL https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz | sha256sum`
+
+Expected output:
+
+- `7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8  -`
+
+If a review tool reports `415 Unsupported Media Type` when fetching the tarball, it is usually the tool refusing to fetch binary content. Validate from a shell (curl/wget) or a normal browser download, then run the verifier locally.
+
+IF.TTT citation handle for the tarball (PQ hybrid signed):
+
+- `if://citation/c24fe95e-226c-4efc-ba22-5ddcc37ff7d2/v1`
+
+---
+
+> The model’s answer is ephemeral. The trace is the product. If you can’t prove what happened, you are not running an AI system — you are running a narrative.
+

IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.7_STYLED.md.sha256

@@ -0,0 +1 @@
+9b55a687e2a0a2e03305c597ba399b4a19f345b8aac3edf2ed775c218d7fbbc5  /root/tmp/hosted_repo_update/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.7_STYLED.md