danny/hosted
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
hosted/IF_EMOTION_DEBUGGING_TRACE_WHITEPAPER_v2.8_STYLED.md

10 KiB
Raw Export PDF Permalink Blame History

IF.EMOTION TRACE PROTOCOL v2.8: AUDITABLE DEBUGGING (WITHOUT WISHFUL THINKING)

Alternate title: Debugging Emotion — The Immutable Flight Recorder

Subject: End-to-End Traceability, Completeness Witnessing, and PQ-Anchored Evidence Binding
Protocol: IF.TTT (Traceable, Transparent, Trustworthy)
Version: 2.8 (Alt Title Added)
Date: 2025-12-21
Status: AUDIT REQUIRED
Citation: if://whitepaper/emotion/trace-protocol/v2.8

Logging is cheap. Evidence is expensive. This protocol exists to force you to pay the expensive part on purpose.

Why This Matters (Decision Context)

If your AI system can’t prove what happened, you don’t have an incident process. You have a debate club.

When an external reviewer sees “415”, “no bytes retrieved”, or “the tool couldn’t download it”, the risk is not the download failure. The risk is that your entire chain-of-custody claim becomes non-falsifiable because the evidence is inaccessible or non-deterministic. This paper pins the boundary, defines what can be proven, and provides a verification path that does not require trusting the operator.

This is the same class of problem solved by append-only transparency logs: not “truth,” but provable provenance of what was recorded, and (within an explicit boundary) whether something was omitted.

0) Layered Stack (Where Guarantees Live)

flowchart TB
  U[User Browser] -->|HTTPS| E[Edge: TLS + Routing]
  E -->|/oauth2/*| O[oauth2-proxy]
  E -->|/api/*| N[nginx]
  N --> B[Backend Witness Boundary]

  B --> R[Retrieval / RAG]
  B --> P[Prompt Build]
  B --> M[Model Inference]
  B --> X[Postprocess: citations + trace footer]

  B --> T1["REQ_SEEN ledger<br/>(hourly JSONL)"]
  B --> T2["Trace events<br/>(hash chain JSONL)"]
  B --> T3["Signed summary<br/>(output hash + head attestation)"]

  T1 --> H["Signed Merkle head<br/>(per hour)"]
  T2 --> S["Trace head<br/>(event_hash)"]

  H --> BUNDLE["Evidence bundle<br/>(tar.gz + manifest)"]
  S --> BUNDLE
  T3 --> BUNDLE

  BUNDLE --> REG["IF.TTT registry<br/>(PQ-hybrid anchor)"]
  BUNDLE --> MIRROR["Public static mirror<br/>/static/hosted"]

Interpretation: integrity starts at the backend witness boundary. Completeness (REQ_SEEN) is only meaningful at and after that boundary until edge witnessing is implemented.

1) What This Protocol Actually Guarantees

This system does not try to “prove the model is true.” That is not a meaningful claim for probabilistic generation.

This system proves something narrower and more valuable:

  1. what the system received (as a commitment),
  2. what the system did (trace event chain),
  3. what the system returned (output hash),
  4. what evidence it claims to have used (retrieval IDs + citation handles),
  5. that the resulting artifacts are tamper-evident and portable for external review.

If a claim cannot be bound to an artifact, it does not exist.

2) Evidence Inventory (What Exists, Where It Lives, How You Verify It)

Artifact What it proves How to verify (independent)
REQ_SEEN hourly ledger request attempts within the backend boundary inclusion proof against signed Merkle head
REQ_SEEN signed head integrity of a batch of witnessed requests verify signature + Merkle root
trace event chain ordered sequence of system actions (post-witness) recompute event_hash and prev_hash links
signed summary binding between output + trace head + policy decisions verify signature + output hash
evidence bundle (tar.gz) portable capsule of the above verify SHA256 + run verifier tool
IF.TTT registry record anchored registration (PQ-hybrid) verify registry chain record + signature

3) The Transport Problem (And Why We Hardened It)

The most sophisticated cryptographic protocol is still useless if reviewers can’t retrieve the artifact. The correct response is not “tell them to trust us.” It’s to make the transport layer boring and compatible.

Transport hardening applied:

  • The public mirror serves *.tar.gz with Content-Disposition: attachment.
  • The public mirror serves *.tar.gz with Content-Type: application/octet-stream for maximum client compatibility.
  • The public mirror directory is browsable for discovery and typo recovery: https://infrafabric.io/static/hosted/.

4) The Real Failure Mode (Contrarian Reframe)

The problem isn’t hallucination.

The problem is un-auditable systems pretending to be auditable. Most teams treat tracing as “extra logs.” In probabilistic systems, tracing is the product: it is the only thing that turns a dispute into a verification.

5) The Human Constraint (Why Teams Avoid This)

The resistance: “This is too much work; we’ll add observability later.”
The psychology: Shipping dopamine beats audit discipline, and nobody gets promoted for preventing future disputes.
The unstick: Make “no trace, no trust” a deployment gate. If it can’t be verified by an adversarial reviewer, it doesn’t ship.

6) Core Protocol (Technical Specification)

6.1 Trace ID Contract (Non-Negotiable)

Every request to /api/chat/completions receives a Trace ID. This includes denials.

Surfaces:

  • Header: X-IF-Emotion-Trace: <uuid>
  • Header: X-IF-Emotion-Trace-Sig: <sig> (app-level integrity)
  • User output: final line Trace: <uuid>

The Trace ID is the support ticket, the incident handle, and the audit join key.

6.2 Completeness: REQ_SEEN Witness Ledger (And Its Boundary)

REQ_SEEN records every request attempt that reaches the backend witness boundary as a privacy-preserving commitment:

  • user_text_sha256, user_len, decision/reason, and leaf_hash

It writes:

  • Hour ledger: /opt/if-emotion/data/req_seen/<YYYYMMDDTHH>.jsonl
  • Signed Merkle head: /opt/if-emotion/data/req_seen/heads/<YYYYMMDDTHH>.json

REQ_SEEN completeness is only valid for requests that reach the backend process. Requests blocked before the backend are out of scope until the witness is moved to the edge proxy.

6.3 Trace Events: Hash Chain + Immediate Head Attestation

Trace events are stored as a hash chain in:

  • /opt/if-emotion/data/trace_events.jsonl

Each event includes:

  • prev_hash pointer
  • event_hash computed as sha256(prev_hash || canonical_json(event_without_event_hash))

The trace head is attested in the signed completion record with an app-level Ed25519 signature so integrity can be verified immediately.

6.4 Canonicalization: Stable Bytes or No Protocol

v2.x mandates canonical JSON bytes for hashing/signing:

  • Primary: canonicaljson.encode_canonical_json(obj)
  • Fallback: stable JSON serialization (sort_keys, fixed separators, UTF-8)

6.5 Key Management (POC-Grade Today, Audit-Grade Tomorrow)

  • App Ed25519 signing key:
    • Private: /opt/if-emotion/data/trace_ed25519.key (0600)
    • Public: /opt/if-emotion/data/trace_ed25519.pub (shipped in bundles)
    • Key ID: ed25519-app-v1

6.6 Post-Quantum: What Is PQ Today (And What Isn’t)

Evidence bundles are PQ-hybrid signed when registered into IF.TTT. The IF.TTT registry record includes:

  • pq_status: hybrid-fips204
  • pq_algo: ML-DSA-87

Hot-path app signatures (Ed25519) are not post-quantum. The correct claim is “PQ-anchored at registry time,” not “PQ everywhere.”

6.7 IF.story: Readability Without Evidence Drift

IF.story is a deterministic narrative projection of trace_events.jsonl.

It is not evidence. It is an index.

Each IF.story line includes the event_hash anchor, and auditors should verify those anchors against the raw JSONL.

7) Verifier Tooling (Independent Checks)

Verifier:

  • iftrace.py verify <tar.gz> --expected-sha256 <sha>

Merkle inclusion proof (REQ_SEEN):

  • iftrace.py prove-inclusion --ledger <req_seen_hour.jsonl> --head <req_seen_head.json> --trace-id <uuid>
  • iftrace.py verify-inclusion <proof.json>

Checksum rules (important):

  • sha256s.txt intentionally excludes itself and manifest.json to avoid self-referential checksum traps.

8) Threat Model and Limitations (Explicit)

A) Truncation and external anchoring

Hash chains detect edits. They do not prevent truncation unless head hashes are anchored externally or independently cached.

Current mitigation:

  • IF.TTT registration anchors the tarball hash into a separate chain.

Remaining requirement for certification:

  • scheduled external anchoring of IF.TTT head hashes to a public append-only log.

B) Clock integrity

Timestamps are derived from system clocks and are not trusted for cryptographic time.

Ordering is guaranteed by hash chain indices and hash pointers, not by wall-clock truth.

C) Code integrity

Hash chains detect post-hoc tampering. They do not prevent a modified binary from choosing not to record.

Certification path:

  • signed deploy attestations (image digest + config hash) bound into IF.TTT
  • optional remote attestation

9) Reference Proof Run (v2.1) — Public Transport Verified

Trace ID:

  • 016cca78-6f9d-4ffe-aec0-99792d383ca1

Preferred download URL (static mirror for external reviewers):

  • https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz

Static mirror directory:

  • https://infrafabric.io/static/hosted/

Alternate download URL (Forgejo raw):

  • https://git.infrafabric.io/danny/hosted/raw/branch/main/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz

Tarball SHA256:

  • 7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8

One-line verifier (shell):

  • curl -fsSL https://infrafabric.io/static/hosted/emo_trace_payload_016cca78-6f9d-4ffe-aec0-99792d383ca1.tar.gz | sha256sum

Expected output:

  • 7101ff9c38fc759a66157f6a6ab9c0936af547d0ec77a51b5d05db07069966c8 -

If a review tool reports 415 Unsupported Media Type when fetching the tarball, it is usually the tool refusing to fetch binary content. Validate from a shell (curl/wget) or a normal browser download, then run the verifier locally.

IF.TTT citation handle for the tarball (PQ hybrid signed):

  • if://citation/c24fe95e-226c-4efc-ba22-5ddcc37ff7d2/v1

The model’s answer is ephemeral. The trace is the product. If you can’t prove what happened, you are not running an AI system — you are running a narrative.