Add IF.emotion trace whitepaper (styled v2.0)

IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.0_STYLED.md

@@ -0,0 +1,217 @@
+# IF.EMOTION DEBUGGING TRACE PROTOCOL: THE IMMUTABLE FLIGHT RECORDER (v2.0)
+
+**Subject:** End-to-End Traceability, Completeness Witnessing, and Post-Quantum Evidence Binding  
+**Protocol:** IF.TTT (Traceable, Transparent, Trustworthy)  
+**Version:** 2.0 (Methodology Upgrade)  
+**Date:** 2025-12-21  
+**Status:** AUDIT REQUIRED  
+**Citation:** `if://whitepaper/emotion/trace-protocol/v2`  
+
+---
+
+## 1) The Architecture of Accountability
+
+Logs are not evidence. Logs are a story you tell yourself after the fact.
+
+In a probabilistic system, the only defensible position is: every request is an incident record, every incident record is a cryptographic artifact, and every artifact is verifiable by someone who does not trust you.
+
+This protocol defines the v2.0 trace methodology for IF.emotion. It upgrades the system from “integrity of what we recorded” to “measurable completeness within the witnessed boundary, plus nested priors for key artifacts.”
+
+### What this system proves
+
+It proves what the system did: what it received, what it retrieved, what it emitted, and which code/config produced those artifacts—without requiring trust in screenshots or operator narration.
+
+### What this system does not prove
+
+It does not prove the model’s answer is “true about the world.” It proves provenance and integrity of the pipeline and binds claims to cited evidence. Truth is a separate problem.
+
+---
+
+## 2) The Trace ID Contract
+
+If a user can’t point to a trace, the system is effectively un-auditable.
+
+Every call to `/api/chat/completions` produces a Trace ID, even if denied.
+
+**Surfaces:**
+
+- **HTTP header:** `X-IF-Emotion-Trace: <uuid>`
+- **Signature header:** `X-IF-Emotion-Trace-Sig: <sig>` (app-level integrity)
+- **User-visible output:** final line `Trace: <uuid>`
+
+This is not a convenience feature. It is a commitment.
+
+---
+
+## 3) Completeness: REQ_SEEN (The Witness Ledger)
+
+Integrity alone is cheap. Completeness is where systems lie.
+
+v1 could prove: “this trace wasn’t modified.”  
+v2 adds: “a request attempt happened, and the system recorded that it happened,” including denials.
+
+### What REQ_SEEN is
+
+REQ_SEEN is a ledger that records every request attempt that reaches the backend, including those denied by:
+
+- missing login
+- quota exceeded
+- invalid input
+
+### How completeness is witnessed
+
+REQ_SEEN writes an hourly ledger:
+
+- `/opt/if-emotion/data/req_seen/<YYYYMMDDTHH>.jsonl`
+
+Each entry includes privacy-preserving commitments (hashes, not raw sensitive content), and a `leaf_hash`.
+
+An hourly Merkle head is computed and signed:
+
+- `/opt/if-emotion/data/req_seen/heads/<YYYYMMDDTHH>.json`
+
+The head contains `count`, `merkle_root`, and `sig_ed25519`.
+
+This does not magically prove “the entire internet sent us nothing else.” It proves measurable completeness within the boundary that actually witnesses requests.
+
+---
+
+## 4) Trace Events: The Flight Recorder
+
+The trace chain is the timeline. The witness ledger is the completeness anchor. You need both.
+
+Trace events are written as a hash chain:
+
+- `/opt/if-emotion/data/trace_events.jsonl`
+
+Each event contains:
+
+- `prev_hash` → pointer to predecessor
+- `event_hash` → hash of the event payload (canonical JSON bytes)
+
+If an event is removed or altered, the chain breaks.
+
+Core events include:
+
+- `req_seen` (witness head snapshot)
+- `request_received` or `request_denied`
+- `retrieval_done`
+- `prompt_built`
+- `model_done`
+- `replacement_applied` (citation normalization)
+- `guard_applied` (output transformation)
+- `trace_finalizing`
+
+---
+
+## 5) Canonicalization: What We Actually Hash
+
+Hashing “some JSON” is not a protocol. It’s a gamble.
+
+v2 makes canonicalization explicit:
+
+- Primary: `canonicaljson.encode_canonical_json(obj)`
+- Fallback: stable JSON encoding (`sort_keys`, no whitespace variance)
+
+This prevents silent hash drift across environments.
+
+---
+
+## 6) Hot-Path Signing (Immediate Integrity)
+
+There is always a window between “event happened” and “registry anchored it.” That window is where systems get accused—and where systems get hacked.
+
+v2 signs immediately using an app-level Ed25519 key:
+
+- Key file: `/opt/if-emotion/data/trace_ed25519.key`
+- Key id: `ed25519-app-v1`
+
+REQ_SEEN heads and trace chain objects can be signed immediately so reviewers don’t have to wait for a registry job.
+
+---
+
+## 7) Nested Priors: Per-Artifact Chain-of-Custody
+
+A single tarball signature is a strong fast-path check. It is not enough for serious audits.
+
+v2 registers key inner artifacts in IF.TTT as independent handles (“priors”), then includes:
+
+- `payload/ttt_children.json`
+- `payload/ttt_children_chain_records.json`
+
+This supports two audit modes:
+
+1) **Fast audit:** verify the tarball IF.TTT handle.  
+2) **Deep audit:** verify any inner artifact independently, then confirm the tarball contains the same bytes via `manifest.json` + per-file hashes.
+
+---
+
+## 8) IF.story: Contextual Logging (Readable, Not Editable)
+
+JSONL is evidence. It is not communication.
+
+IF.story is a deterministic narrative projection of the trace chain where every line includes the `event_hash` anchor.
+
+It improves comprehension without creating a second truth source.
+
+---
+
+## 9) Evidence Bundle Format (What Reviewers Get)
+
+The portable bundle is a tarball containing a `payload/` directory with:
+
+- raw request/response artifacts
+- trace chain (`trace_events.jsonl`)
+- signed completion record (`ttt_signed_record.json`)
+- REQ_SEEN ledger + signed Merkle head
+- IF.story narrative
+- nested prior mappings (`ttt_children*.json`)
+- canonical manifest (`manifest.json`) + per-file hashes (`sha256s.txt`)
+
+---
+
+## 10) Reference Proof Run (v2.0)
+
+Trace ID:
+
+- `09aad3e1-f420-451e-a189-e86f68073dc0`
+
+Hosted tarball URL:
+
+- `https://git.infrafabric.io/danny/hosted/raw/branch/main/emo_trace_payload_09aad3e1-f420-451e-a189-e86f68073dc0.tar.gz`
+
+Tarball SHA256:
+
+- `1e729429e29e47b2efd1c78d07c16df653af1363786468d436f27e81cc870b3d`
+
+IF.TTT citation handle for the tarball (PQ hybrid signed):
+
+- `if://citation/1c1cca4e-b63e-4259-8d0d-de2a038585fe/v1`
+
+---
+
+## 11) Threat Model and Limitations (The Part People Hide)
+
+### A) Completeness boundary
+
+REQ_SEEN proves completeness only for requests that reach the witness boundary. If the witness is inside the app process, requests blocked before the app are outside scope.
+
+Next hardening step: move REQ_SEEN to the true front door (edge proxy) and anchor hourly roots in IF.TTT automatically.
+
+### B) Registry trust boundary
+
+Physical separation is not enough if the same admin controls both systems. External certification requires independent control of registry keys and/or external anchoring of registry head hashes.
+
+### C) Code integrity
+
+Hash chains detect post-hoc tampering. They do not prevent a modified binary from choosing not to emit events. Binding traces to signed deploy attestations (image digest + config hash) is the next step.
+
+---
+
+## 12) Next Moves (What Gets This to External Certification)
+
+1) Edge-level REQ_SEEN witness + Merkle inclusion proofs (not just roots).  
+2) Verifier CLI that replays: tar hash → manifest → trace chain → REQ_SEEN head → IF.TTT chain verification.  
+3) External anchoring of IF.TTT head hashes on a schedule.  
+4) Signed deploy attestations bound into IF.TTT.
+

IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.0_STYLED.md.sha256

@@ -0,0 +1 @@
+a848a2641adb5c94a7c490b69f78470a39962967353638d92e10d6498728bd65  /root/tmp/hosted_repo_update/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.0_STYLED.md

README.md

@@ -16,3 +16,8 @@ Static hosted artifacts used in InfraFabric reviews.
 - SHA256: `1e729429e29e47b2efd1c78d07c16df653af1363786468d436f27e81cc870b3d`
 - IF.TTT citation (PQ hybrid signed): `if://citation/1c1cca4e-b63e-4259-8d0d-de2a038585fe/v1`
 - Notes: includes `payload/if_story.md`, `payload/req_seen_*` (Merkle witness), and `payload/ttt_children.json` + `payload/ttt_children_chain_records.json` (per-artifact priors).
+
+## IF.emotion trace whitepaper (styled)
+
+- File: `IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.0_STYLED.md`
+- Notes: methodology v2.0, references trace tarball `emo_trace_payload_09aad3e1-f420-451e-a189-e86f68073dc0.tar.gz`.