danny/hosted
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
hosted/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.0_STYLED.md

7.1 KiB
Raw Export PDF Permalink Blame History

IF.EMOTION DEBUGGING TRACE PROTOCOL: THE IMMUTABLE FLIGHT RECORDER (v2.0)

Subject: End-to-End Traceability, Completeness Witnessing, and Post-Quantum Evidence Binding
Protocol: IF.TTT (Traceable, Transparent, Trustworthy)
Version: 2.0 (Methodology Upgrade)
Date: 2025-12-21
Status: AUDIT REQUIRED
Citation: if://whitepaper/emotion/trace-protocol/v2

1) The Architecture of Accountability

Logs are not evidence. Logs are a story you tell yourself after the fact.

In a probabilistic system, the only defensible position is: every request is an incident record, every incident record is a cryptographic artifact, and every artifact is verifiable by someone who does not trust you.

This protocol defines the v2.0 trace methodology for IF.emotion. It upgrades the system from “integrity of what we recorded” to “measurable completeness within the witnessed boundary, plus nested priors for key artifacts.”

What this system proves

It proves what the system did: what it received, what it retrieved, what it emitted, and which code/config produced those artifacts—without requiring trust in screenshots or operator narration.

What this system does not prove

It does not prove the model’s answer is “true about the world.” It proves provenance and integrity of the pipeline and binds claims to cited evidence. Truth is a separate problem.

2) The Trace ID Contract

If a user can’t point to a trace, the system is effectively un-auditable.

Every call to /api/chat/completions produces a Trace ID, even if denied.

Surfaces:

  • HTTP header: X-IF-Emotion-Trace: <uuid>
  • Signature header: X-IF-Emotion-Trace-Sig: <sig> (app-level integrity)
  • User-visible output: final line Trace: <uuid>

This is not a convenience feature. It is a commitment.

3) Completeness: REQ_SEEN (The Witness Ledger)

Integrity alone is cheap. Completeness is where systems lie.

v1 could prove: “this trace wasn’t modified.”
v2 adds: “a request attempt happened, and the system recorded that it happened,” including denials.

What REQ_SEEN is

REQ_SEEN is a ledger that records every request attempt that reaches the backend, including those denied by:

  • missing login
  • quota exceeded
  • invalid input

How completeness is witnessed

REQ_SEEN writes an hourly ledger:

  • /opt/if-emotion/data/req_seen/<YYYYMMDDTHH>.jsonl

Each entry includes privacy-preserving commitments (hashes, not raw sensitive content), and a leaf_hash.

An hourly Merkle head is computed and signed:

  • /opt/if-emotion/data/req_seen/heads/<YYYYMMDDTHH>.json

The head contains count, merkle_root, and sig_ed25519.

This does not magically prove “the entire internet sent us nothing else.” It proves measurable completeness within the boundary that actually witnesses requests.

4) Trace Events: The Flight Recorder

The trace chain is the timeline. The witness ledger is the completeness anchor. You need both.

Trace events are written as a hash chain:

  • /opt/if-emotion/data/trace_events.jsonl

Each event contains:

  • prev_hash → pointer to predecessor
  • event_hash → hash of the event payload (canonical JSON bytes)

If an event is removed or altered, the chain breaks.

Core events include:

  • req_seen (witness head snapshot)
  • request_received or request_denied
  • retrieval_done
  • prompt_built
  • model_done
  • replacement_applied (citation normalization)
  • guard_applied (output transformation)
  • trace_finalizing

5) Canonicalization: What We Actually Hash

Hashing “some JSON” is not a protocol. It’s a gamble.

v2 makes canonicalization explicit:

  • Primary: canonicaljson.encode_canonical_json(obj)
  • Fallback: stable JSON encoding (sort_keys, no whitespace variance)

This prevents silent hash drift across environments.

6) Hot-Path Signing (Immediate Integrity)

There is always a window between “event happened” and “registry anchored it.” That window is where systems get accused—and where systems get hacked.

v2 signs immediately using an app-level Ed25519 key:

  • Key file: /opt/if-emotion/data/trace_ed25519.key
  • Key id: ed25519-app-v1

REQ_SEEN heads and trace chain objects can be signed immediately so reviewers don’t have to wait for a registry job.

7) Nested Priors: Per-Artifact Chain-of-Custody

A single tarball signature is a strong fast-path check. It is not enough for serious audits.

v2 registers key inner artifacts in IF.TTT as independent handles (“priors”), then includes:

  • payload/ttt_children.json
  • payload/ttt_children_chain_records.json

This supports two audit modes:

  1. Fast audit: verify the tarball IF.TTT handle.
  2. Deep audit: verify any inner artifact independently, then confirm the tarball contains the same bytes via manifest.json + per-file hashes.

8) IF.story: Contextual Logging (Readable, Not Editable)

JSONL is evidence. It is not communication.

IF.story is a deterministic narrative projection of the trace chain where every line includes the event_hash anchor.

It improves comprehension without creating a second truth source.

9) Evidence Bundle Format (What Reviewers Get)

The portable bundle is a tarball containing a payload/ directory with:

  • raw request/response artifacts
  • trace chain (trace_events.jsonl)
  • signed completion record (ttt_signed_record.json)
  • REQ_SEEN ledger + signed Merkle head
  • IF.story narrative
  • nested prior mappings (ttt_children*.json)
  • canonical manifest (manifest.json) + per-file hashes (sha256s.txt)

10) Reference Proof Run (v2.0)

Trace ID:

  • 09aad3e1-f420-451e-a189-e86f68073dc0

Hosted tarball URL:

  • https://git.infrafabric.io/danny/hosted/raw/branch/main/emo_trace_payload_09aad3e1-f420-451e-a189-e86f68073dc0.tar.gz

Tarball SHA256:

  • 1e729429e29e47b2efd1c78d07c16df653af1363786468d436f27e81cc870b3d

IF.TTT citation handle for the tarball (PQ hybrid signed):

  • if://citation/1c1cca4e-b63e-4259-8d0d-de2a038585fe/v1

11) Threat Model and Limitations (The Part People Hide)

A) Completeness boundary

REQ_SEEN proves completeness only for requests that reach the witness boundary. If the witness is inside the app process, requests blocked before the app are outside scope.

Next hardening step: move REQ_SEEN to the true front door (edge proxy) and anchor hourly roots in IF.TTT automatically.

B) Registry trust boundary

Physical separation is not enough if the same admin controls both systems. External certification requires independent control of registry keys and/or external anchoring of registry head hashes.

C) Code integrity

Hash chains detect post-hoc tampering. They do not prevent a modified binary from choosing not to emit events. Binding traces to signed deploy attestations (image digest + config hash) is the next step.

12) Next Moves (What Gets This to External Certification)

  1. Edge-level REQ_SEEN witness + Merkle inclusion proofs (not just roots).
  2. Verifier CLI that replays: tar hash → manifest → trace chain → REQ_SEEN head → IF.TTT chain verification.
  3. External anchoring of IF.TTT head hashes on a schedule.
  4. Signed deploy attestations bound into IF.TTT.