---
title: "Concepts"
description: "NovaFabric concepts: Run Capsules, zero-code capture, the five replay modes, structural diff, lineage, the Asset Registry and how signed evidence is made."
canonical: https://novafabric.ai/docs/concepts/
last_updated: 2026-10-25
source: https://github.com/MSKazemi/novafabric/blob/main/docs/concepts.md
---

# Concepts

This page is the conceptual reference for NovaFabric. It explains the nouns you
will meet everywhere else in the docs and on the `nova` command line — what a
Run Capsule is, how capture works without touching your code, the five replay
modes, structural diff, lineage, the Asset Registry and its lifecycle, and how
signed evidence is produced.

**What you will learn**

- The **five primitives** NovaFabric is built on — Asset Registry, Run Capsule,
  Replay, Lineage, and Evidence Bundle — and the two journeys that branch from
  the Run Capsule: **Capture → Replay → Diff** (developer) and
  **Capture → Seal → Verify → Audit** (trust; sealing is opt-in).
- The exact on-disk anatomy of a **Run Capsule** and why it is the source of
  truth (everything else is a rebuildable index).
- How **zero-code-change capture** works via a `sitecustomize.py` loader and
  per-SDK plus wire-level hooks.
- The **five replay modes** — `forensic`, `mocked`, `semantic`, `exact`, and experimental
  `intervention` — and what each one honestly promises.
- How **structural diff** becomes a CI regression gate, and how **lineage**
  answers provenance and blast-radius questions.
- How the **Asset Registry lifecycle** works and why promotion is governance
  metadata only — it never restarts or redeploys anything.
- What is **shipped today** versus **planned** (clearly labeled throughout).

> **Maturity note.** NovaFabric is local-first and **beta / pre-v1**. Nearly all
> shipped surfaces carry `experimental` maturity: they work today and are tested,
> but on-disk formats are **not frozen** until the v1.0 schema freeze.
> Anything labeled **PLANNED** or **FUTURE DESIGN** below is documented design
> intent (in the ADRs) and is **not implemented** — never treat it as shipped.

---

## The five primitives at a glance

NovaFabric is framed around exactly five primitives, each with a public spec and
JSON Schema. Cryptographic sealing is part of the Evidence Bundle / trust layer,
not a sixth primitive.

| # | Primitive | What it is | Since |
|---|---|---|---|
| 1 | **Asset Registry** | Local SQLite registry of versioned AI assets (`name@version`), pinned to a git SHA, with a six-state lifecycle | v0.1 |
| 2 | **Run Capsule** | The fundamental unit of capture: a ULID-named directory holding every observable fact of one execution | v0.2 |
| 3 | **Replay** | Re-execute or inspect a capsule with external calls controlled, in five explicit modes | v0.3 |
| 4 | **Lineage** | A directed provenance graph (SQLite cache) with mechanical edge types and OpenLineage emission | v0.4 |
| 5 | **Evidence Bundle** | A signed, self-contained ZIP an auditor can verify offline with only `sha256sum` + an ed25519 verifier | v0.4 |

The design invariant behind all five: **the capsule is the source of truth; the
registry, metadata DB, and lineage graph are derived, rebuildable indexes.**
Only two top-level formats exist — Run Capsule and Evidence Bundle — and
introducing a third requires an accepted ADR.

---

## Run Capsule

A **Run Capsule** is the fundamental unit of capture in NovaFabric. It is a
directory recording a single command execution: the command, timing,
environment, the LLM calls and tool invocations capture can see, stdout/stderr,
and a secret-scan record (rule-based scanning for known key and token patterns).

Capsules are identified by a [ULID](https://github.com/ulid/spec) — a
lexicographically sortable, time-prefixed unique identifier.

Capture writes each capsule to `$NOVAFABRIC_CAPSULE_DIR/<ulid>/` when that
variable is set, otherwise to `$NOVAFABRIC_HOME/capsules/<ulid>/`
(`~/.novafabric/capsules/<ulid>/` by default; `_paths.default_capsule_dir`).

```
~/.novafabric/capsules/01HXAY7M5JZ8R7K4P9DPBYK2WX/
  capsule.yaml          ← run manifest
  trace.jsonl           ← execution spans (OTel)
  model-calls.jsonl     ← LLM API calls (OTel GenAI semconv)
  tool-calls.jsonl      ← tool invocations
  assets.jsonl          ← asset references
  env.lock              ← environment snapshot
  redaction-proof.json  ← secret scan proof
  replay.yaml           ← replay constraints
  lineage.jsonl         ← lineage edges
  inputs/
  outputs/
    stdout.txt
    stderr.txt
```

Capsules are written on success **and** failure. A failed run produces a
complete capsule with `status: failure` and an `error` block — the evidence of a
failure is just as durable as the evidence of a success.

Capsules support additive, optional `extensions:` blocks (for example `slurm`,
`kubernetes`, `ray`, `openlineage`) so the schema can grow without introducing a
new top-level format. Readers tolerate unknown fields; old capsules stay readable
forever.

### Links inside a capsule are never followed (works today)

A capsule is a plain directory, so anyone who can write into it can plant a
symbolic link such as `inputs/data.csv -> ~/.ssh/id_rsa`. NovaFabric never
writes a link into a capsule, and neither a ZIP entry nor the capsule Merkle root
can represent one. So nothing that copies, archives or serves capsule files
follows one. They all share one module (`novafabric._capsule_fs`):

- **Refused with a named error, nothing written:** `nova export-evidence`,
  `nova export-blob` (exit 2, "Export refused"), `nova migrate` and
  `/api/capsule-migrate`, `nova export-rocrate` and
  `/api/compliance/export/rocrate` (HTTP 422), `nova backup create` (also for
  links under `spool/` and `seal/ratchet/`), `nova audit bundle`,
  `nova export-examiner bagit`, and the SDK's `upload_capsule`. Any link counts:
  one to a file or a directory outside the capsule, a dangling one, and one that
  stays inside the capsule. The error names each link and where it points.
- **Never followed, even when planted mid-export:** files are opened one path
  component at a time with `O_NOFOLLOW`, so a link swapped in after the check
  is refused at open time instead of copied.
- **The capsule Merkle root:** the root (`capsule_merkle_root`, both the v1 and
  v2 leaves) is the signed subject of Evidence Bundles, the `nova serve`
  evidence cart, consent attestations, AI-BOM citations, OpenLineage capsule
  facets, chain-of-custody blocks, session bundles, baseline pins, experiment and
  offline-eval scores, and comment, annotation and score-submission subjects. It
  walks the capsule without following a link and refuses a capsule holding one,
  so it can never commit to a file outside the capsule. No root is computed
  over the remaining files. The CLI prints one `error:` line and exits 2 (`nova
  score submit` reports `capsule_unusable`, exit 2; `nova consent attest` exits
  1, and `nova consent verify-attestation` reports the root check as failed,
  exit 1, not stale). `nova serve` answers 422, also for the evidence cart even
  with `accept_unresolved`, so a linked capsule never becomes a recorded
  omission. Server-mode score submission answers 409. In `nova experiment run`
  the item is recorded as an error, with no score. Each file is opened with
  `O_NOFOLLOW` and read in chunks. A FIFO, socket or device in the capsule is
  left out without being opened, as it always was, and a file swapped for one
  mid-hash is refused. So the root never blocks. For a capsule without links,
  the root is byte-identical to the one earlier releases computed.
- **`nova serve`:** `GET /api/runs/{id}/file/{path}` answers 404 for a link (or
  a FIFO) and never returns its target. A run whose capsule holds a link or a
  special file (FIFO, socket, device) at its top level (`capsule.yaml`, a JSONL
  stream, `redaction-proof.json`, `inputs`, `outputs`) is refused with 403 by
  every `/api/runs/{id}/...` route. The run listing never parses a linked
  `capsule.yaml`, and the `inputs`/`outputs` listings leave links out.
- **Scans of many capsules** (the `nova serve` run listing, search, cost report,
  `suggest-register`, run lineage, children, topology, AI-BOM status, the run
  index, KG bulk ingest, `nova query`, the examiner and audit exporters, server
  mode's capsule listing) never list an entry of the capsule directory that is
  itself a link, whether it points inside the directory (`latest -> 01J...`, an
  alias of a capsule the scan already lists) or outside it. Every file they read
  in each capsule goes through the same no-follow open, so a link or a FIFO
  planted in a *sibling* capsule is never followed either.
- **A capsule root that is a link, in `nova serve`:** a run named directly
  (`/api/runs/latest`) whose directory entry is a link is served only when the
  link resolves inside the server's read roots (the capsule directory,
  `$NOVAFABRIC_HOME` minus its secrets, any `NOVAFABRIC_SERVE_PATH_ROOTS` entry);
  otherwise every route answers 403. The server reads through the resolved path.
- **Readers:** every command that parses one well-known capsule file (`nova
  export --html`, `nova diagnose`, `nova diff`, `nova replay`, `nova assure`,
  `nova validate`, `nova verify`, `nova eval contamination-check`, the lineage
  and OpenLineage importers, the knowledge-graph consumer, the content and query
  indexes, and the compliance exporters) opens it the same way. A link, a FIFO
  or a device is never followed and never blocks.
- **A required file that is present but unusable is an error, never empty
  output.** An *absent* file still means "no data". But when `capsule.yaml`,
  `capsule.json` or a stream a command's output depends on (`model-calls.jsonl`,
  `tool-calls.jsonl`, `trace.jsonl`, `lineage.jsonl`, `replay.yaml`, `env.lock`,
  `redaction-proof.json`, ...) is there but is a link, a FIFO, a directory, an
  unreadable or an over-size file, every command whose output claims to be
  complete or correct stops. That covers every export and compliance artifact,
  report, verdict and attestation, replay, migration, lineage write, PROV-O
  mapping and annotation-queue fill. The CLI prints one `error:` line naming the
  file and exits 2, unless the command has its own refusal code (`nova validate`
  reports it as a validation error, exit 1). It writes no output file. `nova
  serve` answers 422 and server mode answers 409. Such a command never reports
  0 model calls, an empty Art. 12 log, a "no secrets detected" assurance pass or
  a best-effort replay pin for a file nobody could read. Only best-effort
  listings (the dashboard run list, search and content indexes, session view,
  lineage child listings) still skip such a file. They log
  `capsule_file_not_followed`, and the knowledge-graph bulk ingest counts it as
  `skipped`. `nova export --html` skips such a section with a warning on stderr
  and in the page; a linked `capsule.yaml` refuses the export (exit 2).
- **`nova restore`** refuses a set whose listed member is a link entry, and
  never writes through a link at a destination path. `nova migrate-schema`
  skips (and reports) a capsule holding a link, and AI-BOM generation never
  writes `aibom.json` through one.
- **Writers into a finished capsule** never write through a link either:
  `nova energy attest` and the SLURM runner's sacct receipt
  (`energy-receipts.jsonl`), `nova validate --write` (`tool-calls.jsonl`), the
  registry's asset-consumption append (`assets.jsonl`), every score append to
  `scores.jsonl` (`nova eval offline`, the annotation queue, score submission,
  `nova experiment run`), and the `nova capture` workload's capture-health
  hand-off all open the file with `O_NOFOLLOW`
  (`_capsule_fs.open_capsule_file_for_write`). A link, a dangling link or a FIFO
  at that path is refused with a named error (`nova energy attest` exits 2; the
  SLURM receipt is skipped with a warning) and its target is left untouched.
  Whole-file writers replace the entry by an atomic rename instead, so a link
  or hard link planted at the name is replaced, never written through: the
  dataset-provenance facets (`extensions/dev.novafabric.dataset-provenance/`;
  a linked `extensions/` directory is refused), `nova export-c2pa`'s default
  `c2pa-manifest.json`, and `nova capture --emit-otel-genai`'s
  `otel-genai-spans.json`.
- **`nova capture` itself, once the workload has run.** The workload writes into
  its own capsule, and with `--runner docker` the capsule directory is mounted
  read-write into the container (with `--runner kubernetes` it is copied back by
  `kubectl cp`). So capture treats the capsule as untrusted input from the moment
  the runner returns. Following a link the container planted would be a
  container-to-host write: before this was fixed, a `model-calls.jsonl` link made
  the secret scanner rewrite the file it pointed at, and a FIFO at
  `tool-calls.jsonl` hung capture forever. Now, before any host-side read or
  write, capture checks every capsule-managed path (the JSONL streams,
  `capsule.yaml`, `env.lock`, `replay.yaml`, `redaction-proof.json`,
  `capture-health.json`, `lineage.jsonl`, `c2pa-manifest.json`,
  `redaction_manifest.json`, `outputs/stdout.txt`, `outputs/stderr.txt`, the
  `inputs/` and `outputs/` directories, and `.seal/`, which cannot exist yet)
  with `lstat` and never opens it. A link, FIFO, socket or device there, a
  directory where a file belongs, or any `.seal/` is removed without being
  followed (removing a link never touches its target), and the empty stream or
  directory is recreated. Capture then walks the **whole** capsule tree, one
  `O_NOFOLLOW` directory at a time, and removes every symlink, FIFO, socket and
  device at any depth: `inputs/data.csv -> ~/.ssh/id_rsa`,
  `outputs/media/blob.bin`, `extensions/...`, a linked `outputs/dirlink`, a
  FIFO at `inputs/pipe`, a link at `scores.jsonl`. NovaFabric never writes any
  of these, so each one was planted, including one that points inside the
  capsule (for example `outputs/latest -> model.bin`). A directory nested more
  than 64 levels deep is not descended and is recorded as `not-swept`. Each
  entry found marks the capsule **tampered**: `metadata.capture_tampered` in the
  signed manifest, `tampered_entries` in `capture-health.json`, and a skipped
  `<path>#tampered` target (`skip_reason: "stream_tampered: ..."`) in
  `redaction-proof.json` (the first 1,000 entries are listed one by one; the
  summary always gives the total). Every later write is an atomic rename or an
  `O_NOFOLLOW` append, and every later read is a no-follow, non-blocking open,
  so an entry planted *after* the sweep (by a process the workload left
  running, which only the local runner allows) is refused or replaced, never
  followed. That covers the `--pii-subject` stage too: it reads and rewrites
  every target, and writes `redaction_manifest.json`, the same way. When the
  evidence digests are computed, any link or special file found anyway is added
  to `metadata.capture_tampered` ("planted after the sweep: ...") before the
  manifest is sealed. After the sweep the capsule directory is pinned by its
  device and inode: if it is then swapped for a link or another directory, every
  read and write refuses (`CapsuleRootChangedError`) and `nova capture` prints
  `✗ capture integrity: ...` and exits 1. Otherwise capture stays fail-open
  (ADR-0021): it exits with the workload's own exit code and prints `✗ capture
  integrity: ...` on stderr. `nova verify` and `nova validate` fail a tampered
  capsule (exit 1). If the capsule directory itself was moved and replaced
  while the workload ran, capture writes nothing more and says so.
- **`nova verify` and `nova validate` fail any link or special file**, wherever
  it is in the capsule and whenever it was planted, including after sealing:
  each is named in the output (`nova verify --json` lists them under
  `special_entries` and in `errors`) and the command exits 1. The walk never
  follows or opens an entry, so a FIFO cannot hang either command.

The capsule *path you pass* to a CLI command may itself be a link
(`runs/latest`); only links inside it are refused. That one link is followed
because you named it. A scan never follows one it finds, and `nova serve`
follows one only into its read roots.

**Hard links are read, never written through.** A hard link is an ordinary
directory entry for a regular file, so it is read and exported like any other
file, and capture does not record it as tampering. Under Linux's default
`fs.protected_hardlinks = 1`, a user can only hard-link a file they own or can
already read and write, so a planted hard link exposes nothing the planter
could not read already. NovaFabric never creates hard links. Refusing every
file with more than one link would break capsule trees kept in
`rsync --link-dest` or `cp -al` snapshots, and workloads that build a
virtualenv in `outputs/` with a hard-linking installer. What a hard link can no
longer do is carry a *write* out of the capsule: a whole-file rewrite (the
secret scanner, the maskers, the `--pii-subject` stage, the facet and C2PA
writers) replaces the entry by a rename, and an append or truncate first gives
the file a private copy, so the file's other names are never modified. On a
system with `fs.protected_hardlinks = 0`, do not export capsules from a
directory other users can write to.

### Sessions vs parent/child (two different groupings)

Two distinct, composable ways of relating capsules exist — do not conflate them:

- **Session (experimental, ADR-0122).** A *logical, multi-turn* grouping: N
  independent runs (e.g. the turns of a conversation or the stages of a
  workflow) referenced in order by a content-addressed `session.json` manifest
  (`nova session new/add/list/show`). Members are ordinary capsules, each with
  its own writer; the session copies no capsule data. `nova session export`
  packs a session with its members into one verifiable ZIP for another
  machine (experimental). `nova session replay` (experimental, ADR-0123)
  replays the members in sequence order, whole or as a `--from/--to` slice.
- **Parent/child (prototype, ADR-0039).** A *physical, distributed-run*
  hierarchy: one logical execution fanned out across nodes (e.g. a Slurm DDP
  job) recorded as a PARENT capsule plus N WORKER capsules under a single
  `global_run_id`.

A session member may itself be a distributed-run PARENT. A capsule that belongs
to a session carries two additive optional back-reference fields (`session_id`,
`sequence`); absence means a standalone run, byte-identical to before.

> **Related:** [Replay Modes](#replay-modes) (a replay writes a
> `replay_result.yaml` report; only `intervention` also writes a counterfactual
> capsule you can diff), [Lineage Graph](#lineage-graph), and
> [Evidence Bundle](#evidence-bundle).

---

## Evidence facets (experimental)

A capsule can carry optional, additive **evidence facets** — a `facets` map
(ADR-0196) whose keys are per-domain evidence objects, each owned by its own ADR.
A capsule without the `facets` key is valid and byte-identical to before, so
facets never change the base schema. `facets` holds *first-party,
NovaFabric-recorded evidence*; it is deliberately distinct from `extensions`,
which carries *third-party vendor data* — an auditor must be able to tell
NovaFabric-recorded evidence from a vendor annotation. The registry is **closed
on purpose** (ADR-0196 D2): only the registered names below are accepted, so a
typo cannot silently produce a capsule missing the evidence its author believed
it carried.

> **Not the same as OpenLineage facets.** The word "facet" is also used,
> unrelatedly, for the [OpenLineage run/dataset facets](#openlineage-integration)
> that `nova lineage emit-openlineage --with-facets` attaches to emitted lineage
> events. Those live in OpenLineage payloads, not on the capsule. This section is
> about the capsule `facets` container.

Each facet is populated by a **record-only** evidence module: it records what
another system did and never orchestrates, enforces, adjudicates, moves funds,
controls a device, or sits in a hot path. All are **experimental**.

Most are **Python APIs only** — NovaFabric ships the schema, the validation and
the binding rules, and *your* integration code produces the facet. Nothing in
NovaFabric writes them for you, and they register no `nova` command.

The exception is `novafabric.a2a`, which additionally ships
[`nova a2a-card`](https://novafabric.ai/docs/cli-reference/) and [`nova a2a-objects`](https://novafabric.ai/docs/cli-reference/) for
the portable Agent Card and the Task/Message/Artifact mapping (ADR-0149). The
`a2a_messages` facet in the table below remains API-only.

| Module | Capsule facet | Records | ADR |
|---|---|---|---|
| `novafabric.a2a` | `a2a_messages` | Multi-agent A2A messages & handoffs — the wire between agents. Distinct from the `novafabric.adapters.a2a` capture-time SDK adapter, which is a separate subsystem. | ADR-0142 |
| `novafabric.settlement` | `settlement` | Agentic-commerce settlement provenance — mandate reconciliation, finality, and non-repudiation binding (P1/P2), plus the agent-to-agent payment-provenance chain walked offline by `nova settlement chain` (each `parent_hop` must resolve to an earlier hop; no forks, no FX between hops), negotiated-agreement evidence (`agreement_ref`, `terms_digest`, `parties`) and invoice/receipt binding (`matches_settlement` against the finality record; `unbound: true` when the document does not resolve) — P3, experimental. Amounts are integer minor units, never floats. Never processes payments, holds/moves funds, or decides a dispute. | ADR-0163 |
| `novafabric.embodied` | `embodied` | Embodied / cyber-physical agent evidence — declared sensor streams and actuation records (P1), plus the declared ODD with observed excursions (`verdict` always null) and the perception→actuation trajectory chain, walked offline by `nova embodied trajectory verify` (P2, experimental), plus sim-to-real lineage (`unbound: true` when the sim policy does not resolve), pseudonymous teleop handoffs and per-clock-domain timing evidence (`nova embodied sim2real show|verify`, `teleop list`, `timing show` — P3, experimental). Stores references, digests, and counts only (never frames, point clouds, audio, or control credentials); never in a control path. | ADR-0162 |
| `novafabric.science` | `science_provenance` | Scientific-reproducibility / research-integrity provenance as a verifiable DAG of research steps. Never runs experiments or adjudicates validity. | ADR-0164 |
| `novafabric.memstore` | `memstore_mutation` | Persistent-knowledge / organisational-memory governance — an append-only ledger of who changed which shared-KB entry and when (store-external); P2 adds an `access` block (which agent read/wrote which entry, `contained:false` as out-of-scope *evidence*, never enforcement) and cross-run derivation (a read traced to the write that seeded it) — `nova memstore`, experimental. Distinct from the [`nova memory`](https://novafabric.ai/docs/cli-reference/) capture feature. | ADR-0171 |
| `novafabric.retrieval` | `fetch_provenance` (+ source pinning) | Retrieval-source authority & knowledge provenance — what an external retriever or the agent fetched, and whether a pinned source still matches. Never fetches, crawls, or ranks. | ADR-0153 |
| `novafabric.context` | *(standalone artifacts)* | Context provenance — an ordered manifest of what entered the model's context window (`ContextManifest`) and the span→chunk support map a producer claimed (`GroundingMap`). Records `sha256:` digests only; never scores groundedness. Ships as standalone artifacts, not a registered capsule facet (ADR-0196 D2). | ADR-0143 (NF-112/113) |

---

## Capture Hook Mechanism

`nova capture <cmd>` works without modifying application code. It injects a
`sitecustomize.py` loader into the subprocess via `PYTHONPATH`. When Python
starts the subprocess, it imports this loader automatically, which installs
monkey-patches for the libraries below. Python imports only one `sitecustomize`, so the
loader then runs the one it shadows — your own on `PYTHONPATH`, the virtualenv's or the
distribution's — with `site.py`'s contract (an error is printed, the workload still starts);
releases up to v0.107.0 skipped it, so certificates, proxies or warning filters set up there
were missing under capture (**works today**). It patches:

- **OpenAI** — `Completions.create`; since v0.105.0 also `AsyncCompletions.create`, `Responses.create` and `AsyncResponses.create`, in `openai.resources` (`capture/hooks/_openai.py`). A `stream=True` call is folded into one record when the stream ends (ADR-0304) — and that record goes to the capture that *opened* the stream, whichever task, thread or later capture ends it; a stream that ends after its own capture has finished is dropped and counted (`nova_event=capture.stream_record_dropped`), never filed into another capsule (**works today**) — and a call that raised is recorded with the SDK error detail (`io.novafabric.sdk_error`, `capture/hooks/_sdk_errors.py`) that mocked replay needs to raise it again.
- **Anthropic** — `anthropic.resources.messages.Messages.create`; since v0.105.0 also `AsyncMessages.create`, streamed or not (`capture/hooks/_anthropic.py`).
- **httpx** — `httpx.Client.send`, recording requests classified by the URL registry (`src/novafabric/capture/hooks/url_registry.yaml` + `~/.novafabric/url_registry.yaml` override). Default coverage: OpenAI, Anthropic, Cohere, Together, Mistral, Replicate, AWS Bedrock, Ollama (default port 11434). Non-default Ollama ports are detected automatically from `OLLAMA_BASE_URL` / `OLLAMA_HOST` at call time.
- **httpx2** — `httpx2.Client.send` / `httpx2.AsyncClient.send`, same classification and records as `httpx` (network events carry `library: "httpx2"`). `openai` 3.x builds its default client on `httpx2`, so without this hook an OpenAI call left no transport record and no network event. Installed only when `httpx2` is importable — never a NovaFabric dependency. Under an SDK call each attempt is still a `transport` record (ADR-0305), so one call counts once. **Works today.**
- **requests** — `requests.Session.send`, same URL-registry classification (v0.5; RFC-0001 Option C wire-level layer). Covers LangChain HTTP adapters, LlamaIndex REST clients, and any SDK that ships over `requests`.
- **aiohttp** — `aiohttp.ClientSession._request`, async wire-level capture (v0.6 / C-3.1). Catches LangChain async paths, FastAPI agents, streaming-first SDKs.
- **urllib3** — `urllib3.connectionpool.HTTPConnectionPool.urlopen`, lowest-tier wire-level capture (v0.6 / C-3.2). Sits below `requests` and `boto3`; transitively covers AWS Bedrock and any SDK that uses `urllib3` directly. Layered with the higher hooks via a context-local guard so a single `requests` call records exactly once.
- **MCP (Model Context Protocol)** — `mcp.client.session.ClientSession.call_tool`, recording every tool invocation as a `tool-calls.jsonl` entry with `transport=mcp` and a synthesized JSON-RPC envelope (v0.5; ADR-0015 §Primary). A result the server marks `isError: true` — how MCP reports a failing tool — is recorded as `status: error` with `error.type: McpToolError` and the result kept; mocked replay returns that result, as the original call did (releases up to v0.107.0 recorded it as a success). The same holds for `nova mcp-proxy`. **Works today — fail-open:** the hooked call returns, or raises, exactly what the un-hooked call would have. If the record cannot be built or written (a full disk, a removed capsule directory, a response object whose attributes raise), the record is dropped — never reported as a tool error — and the workload continues. The loss is counted (`MCPHook.dropped_records` and the capture's in-process drop counter) and logged once as a structured warning (`nova_event=capture.record_dropped`, exception type only). Under `nova capture` the hook runs in the workload's own process; that process hands its drop counts to `nova capture` when it exits, and they land in the capsule's `capture-health.json` under `sources.workload` (a process that ends with `os._exit` or is killed hands off nothing).
- **Third-party plugins** — any class registered under the `novafabric.hooks` entry-point group is auto-discovered alongside the built-ins (v0.5.x experimental; RFC-0001 §Detailed design — Option C). Plugin authors: see [`docs/integrations/writing-a-hook-plugin.md`](https://novafabric.ai/docs/integrations/writing-a-hook-plugin/).

The layering is deliberate: **per-SDK hooks** capture rich, structured request
and response objects; **wire-level hooks** form a safety net beneath them so a
call that bypasses a known SDK (or uses a version whose surface changed) is still
recorded once at the transport layer. Because the hooks reach down to `urllib3`,
capture does not depend on your choice of HTTP client library.

A call made through the OpenAI or Anthropic SDK passes through both layers, so it
leaves two kinds of record: the SDK hook's **logical** record (with the response) and
one wire-level **transport** record per HTTP attempt beneath it, retries included.
Both are kept as evidence; the transport records are marked
`extensions["io.novafabric.record_role"]: "transport"` and link to the SDK record
through `io.novafabric.logical_call_id`, and every count — `model_call_count`,
`nova cost`, `nova diff`, replay counters, `nova query`, the dashboard — counts
logical calls only (ADR-0305, works today). A capsule captured before this marker
existed is read with the same rule: readers recognise the old wire-plus-SDK duplicate
shape. See [Model-call record roles](https://novafabric.ai/docs/architecture/run-capsule/#model-call-record-roles).

**A hook never changes what the call does (works today; ADR-0021).** In particular:

- **Streamed responses are never read by capture.** A wire hook records the call when
  the response object comes back — for `requests.post(..., stream=True)`,
  `httpx` `client.stream(...)` / `send(..., stream=True)`, an `aiohttp` response or a
  `urllib3` `preload_content=False` response, that is when the headers arrive, before
  the body. The hook does not consume, buffer or wait for the body: the caller gets
  the response at the same moment as without capture, and iterates the same bytes —
  an endless server-sent-events stream included. The network event's
  `response_size_bytes` is the exact length of a body the client library had already
  read, else the declared `Content-Length`, else `null` (unknown — a chunked or
  endless stream). Before this fix the `requests` hook read `response.content` to size
  the event, which drained a streamed body inside `Session.send`, so `send` returned
  only when the stream ended — never, for an endless one.
- **Work before the call is fail-open too.** Deciding whether a request is a model
  call (building its URL, the URL-registry lookup) cannot stop the request: if it
  fails, the request is sent unrecorded and the loss is counted
  (`nova_event=capture.record_dropped`).
- **A positional call reaches the SDK as written.** The OpenAI and Anthropic `create`
  methods are keyword-only; a call with positional arguments is passed through
  unrecorded, so the SDK rejects it exactly as it would without capture (the hook
  used to drop the positional arguments and record the call).

Patches are removed after the run. If an SDK is not installed, its hook is
silently skipped. Capture works even if none of the AI SDKs are present — the
capsule is still written with environment, stdout/stderr, and timing.

**For uninstrumented agents** (Claude Desktop, Cursor, third-party SDKs that do not import the Python `mcp` package), `nova mcp-proxy` is a transparent stdio proxy that records the same `tool-calls.jsonl` schema by sitting between the client and an upstream MCP server. See [`docs/cli-reference.md`](https://novafabric.ai/docs/cli-reference/) and ADR-0015 §Secondary; experimental in v0.5.x.

**For non-Python clients over HTTP**, `nova api-proxy` is a transparent HTTP
proxy that classifies traffic against the same vendored provider registry — so a
client in any language can be captured without SDK hooks (v0.6).

### Runners — where the captured workload executes

The orchestrator delegates subprocess execution to a **runner** (ADR-0025).
Choose one with `nova capture --runner <name>`:

| Runner | Since | What it does | Key required options |
|---|---|---|---|
| **`local`** | v0.5.x | Runs the workload as a local subprocess. Default; no remote infrastructure needed. | — |
| **`docker`** | v0.6 | Runs the workload inside a Docker container. The image must already have NovaFabric installed. Capsule directory is mounted as a volume so artifacts land on the host. | `image` |
| **`kubernetes`** | v0.6.1 | Runs the workload as a Kubernetes `Job` via `kubectl` shell-out. Artifacts are pulled back via `kubectl cp`. | `image`, `namespace` |
| **`slurm`** | v0.6.1 | Runs the workload as a SLURM batch job: the generated script goes to `sbatch` on stdin (never `--wrap`), polled with `sacct`. Trusts a shared filesystem for artifacts (no rsync). | `partition` |
| **`lsf`** | — | Runs the workload as an IBM LSF batch job: the script goes to `bsub` on stdin, polled with `bjobs`. Shared filesystem, no rsync. | — (`queue` defaults to `normal`) |
| **`pbs`** | — | Runs the workload as a PBS batch job: `qsub <capsule>/nova-<run>.pbs`, polled with `qstat`. Shared filesystem, no rsync. | — (`queue` defaults to `default`) |

Runner details:

- **`docker`** — pass `--runner-option image=<ref>` (e.g. `pip install novafabric`
  in your Dockerfile). Other options: `network`, `workdir`, `user`,
  `extra_volumes`, `extra_env`. No privileged containers, no docker-socket
  mounts, no host namespaces by default — see ADR-0025 §Anti-patterns. The
  capsule directory is mounted read-write, so the container can write anything
  into it, including links. Host-side finalisation never follows one: see
  [Links inside a capsule are never followed](#links-inside-a-capsule-are-never-followed-works-today).
- **`kubernetes`** — optional: `service_account`, `node_selector`, `resources`.
  Anti-patterns are enforced in the Job manifest itself:
  `securityContext.privileged: false`, `hostNetwork: false`, `hostPID: false`,
  `hostIPC: false`, `backoffLimit: 0` (one shot, no retries). RBAC required:
  `create`/`get`/`watch`/`delete` on `jobs`+`pods`, `get` on
  `pods/log`+`pods/exec`. **No cluster-admin.**
- **`slurm`** — optional: `account`, `qos`, `time`, `nodes`, `gres`, `mem`,
  `constraint`. The capsule directory **must** be on a filesystem shared between
  submit and compute nodes (typically `/scratch`, `/lustre/...`, or the user's
  home). The runner does not rsync artifacts; it trusts the shared FS.

> **Note.** The single-node runner is the smallest case of a distributed run.
> The cluster-scale tiers shipped in v0.10+ with honest maturity labels:
> parent/child capsules (`nova run …`) are a **prototype** — implemented and
> tested, not yet validated at target scale — and the collector tier, object
> capsule store, and Postgres metadata DB are **experimental**. See
> [ROADMAP.md](https://github.com/MSKazemi/novafabric/blob/main/ROADMAP.md) for per-component labels; the federation and
> at-scale graph tiers remain **future design**.

---

## `model-calls.jsonl`

Every LLM API call intercepted by the capture hooks is recorded as one JSONL
line following the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/):

```json
{
  "schema_version": "0.1.0",
  "semconv_version": "1.30.0",
  "model_call_id": "01HXAY7M6FN9TQGE0V0M7PAY1Q",
  "parent_span_id": "657bff2c61ddad1c",
  "started_at": "2026-05-09T12:34:56.123456Z",
  "finished_at": "2026-05-09T12:34:57.353456Z",
  "duration_ms": 1230,
  "status": "success",
  "gen_ai.system": "openai",
  "gen_ai.operation.name": "chat",
  "gen_ai.request.model": "gpt-4o",
  "gen_ai.response.model": "gpt-4o-2024-08-06",
  "gen_ai.response.id": "chatcmpl-abc123",
  "gen_ai.request.temperature": 0.7,
  "gen_ai.request.max_tokens": 1024,
  "gen_ai.request.top_p": 0.95,
  "gen_ai.request.seed": 42,
  "gen_ai.request.stop_sequences": ["END"],
  "gen_ai.request.messages": [...],
  "gen_ai.response.choices": [...],
  "gen_ai.response.finish_reasons": ["stop"],
  "gen_ai.usage.input_tokens": 42,
  "gen_ai.usage.output_tokens": 18,
  "endpoint": "https://api.openai.com/v1/chat/completions"
}
```

The full set of OTel "Required when applicable" fields is extracted by both
the per-SDK hooks (`_openai`, `_anthropic`) and the wire-level hooks
(`_httpx`, `_requests`, `_aiohttp`, `_urllib3`) when present in the request
body — so `nova replay --mode exact` has the determinism inputs it needs
(temperature, top_p, seed) regardless of which transport the captured call
took. See `design/spec/model-call-v0.md` (private) for the full
field reference.

This format is stable across providers and is the basis for two things:

- **`nova diff` call alignment** — aligned model calls are compared field by
  field (see [Structural Diff](#structural-diff)).
- **`nova replay` mock serving** — recorded calls are replayed from the capsule
  cache in `mocked` mode (see [Replay Modes](#replay-modes)).

---

## Environment Lock (`env.lock`)

The environment lock records the full execution environment at capture time:

- Python version and interpreter path
- All installed packages, sorted by name (releases up to v0.107.0 kept an arbitrary 200)
- Safe environment variables (secrets are excluded)
- Host OS, CPU architecture, CPU count, total memory (read from `/proc/meminfo`;
  `0` where that file does not exist)
- A GPU field (`hardware.gpus`), which is not populated today: no GPU inventory
  is collected
- Detected package manager (`uv.lock`, `poetry.lock`, `pdm.lock`, `requirements.txt`) and the
  SHA-256 of the **whole** lock file (`python.lock_file_hash`; releases up to v0.107.0 hashed
  only its first 64 KiB, so two lock files differing past that hashed the same). Only a
  hash-pinned `uv.lock` makes the lock `deterministic`; the others are `best-effort` with a
  reason, as ADR-0007 specifies, so `--mode exact` refuses them

`nova replay` uses `env.lock` to warn about environment mismatches before
re-executing the command. This is what makes the `exact` replay mode
*falsifiable*: eligibility for byte-exact replay depends on a matching,
deterministic environment plus a per-call seed, and `env.lock` is where that
determination starts.

---

## Secret Scanning and Redaction

Before a capsule is finalized, `SecretScannerV0` scans every capsule file against the
`gitleaks-core-v0` rule pack — LLM provider keys (Anthropic, OpenAI, HuggingFace,
Replicate, Langfuse, and others) and, since v0.105.0, AWS and GitHub credentials
(18 rules in pack 0.7.0). Detected values are redacted in-place as
`[REDACTED:rule-id]`. The scan is pattern-based: formats it does not detect (PEM keys,
JWTs, passwords, connection-string credentials, …) are listed in
[the Run Capsule reference](https://novafabric.ai/docs/architecture/run-capsule/).

After scanning, `redaction-proof.json` is written. It records:

- How many findings were detected
- Which files were scanned
- A cryptographic chain hash proving the scan ran

Capsules without `redaction-proof.json` are considered invalid by
`nova validate` and **cannot be exported** into an Evidence Bundle. Verifiable
redaction is therefore not optional — it is a precondition of every downstream
audit artifact.

---

## Replay Modes

A replay re-executes or inspects a capsule. When it re-executes, NovaFabric
controls only the calls it intercepts (see [`mocked` mode](#mocked-mode)); the
rest run live. There are **five explicit, falsifiable modes**; `intervention` is the
**experimental** counterfactual mode. Every mode writes a `replay_result.yaml`
report; only `intervention` also writes a new (counterfactual) capsule, which you
can `nova diff` against the original run.

| Mode | Spawns subprocess? | Network? | Best for |
|---|---|---|---|
| **`forensic`** | No | No | Audit / post-incident inspection |
| **`mocked`** | Yes | OpenAI/Anthropic model replies (sync or async, streamed or not), MCP `call_tool` results and (experimental) `record.tool` function results served from the capsule; **other tools run live**; fails closed on divergence | CI / regression |
| **`semantic`** | No | No | Consistency score over the capsule's *recorded* model responses — does **not** re-execute |
| **`exact`** | No | No | Eligibility check for a byte-exact re-run — does **not** re-execute |
| **`intervention`** (experimental, ADR-0086) | Yes, under mocked semantics | Model replies served from the capsule (with the one substitution); **tools run live**, so a substituted tool result is not delivered to the workload | Counterfactual root-cause: substitute one captured event per an `InterventionSpec`, re-execute downstream, and record whether the outcome flips |

> **Honesty note.** NovaFabric explicitly does **not** claim byte-exact replay of
> remote LLM calls. `exact` mode requires a deterministic environment and a
> per-call seed, which is realistic for local/on-prem models but not for a
> remote endpoint that can change under you. Neither `exact` nor `semantic`
> re-executes anything today: `exact` reports *whether* a byte-exact re-run is
> possible, and `semantic` scores the recorded responses — a live re-run judged
> on meaning is **future design** (no implementation, not yet roadmapped).

### `forensic` mode

Read-only inspection. NovaFabric does not spawn a subprocess. It reads and
returns the manifest, traces, and model calls directly from the capsule.
No network access, no mutation.

Use forensic mode to inspect what happened without any risk of side effects.

### `mocked` mode

The original command is re-spawned as a subprocess (**works today** for Python
workloads; what v0.104.0 served versus v0.105.0 — MCP results, async,
streamed and Responses API calls, the fail-closed contract — is listed under
[Release scope](https://novafabric.ai/docs/architecture/replay-modes/)). A capsule that records **no command to re-run** — written
by a framework adapter or the `@agent` decorator (`novafabric.sdk.agent`; `capture_mode: sdk-decorator`, a
`@framework:name` label) or imported from OpenTelemetry spans — is refused
before anything is spawned (`CapsuleNotReplayable`, exit 1; since v0.105.0).
Inside the re-spawned process:

- `MockModelDispatcher` serves the recorded responses, in order, for OpenAI
  `chat.completions.create`, OpenAI `responses.create` and Anthropic
  `messages.create` calls — sync or async, and with `stream=True` as the chunk or
  event stream the SDK would have produced (ADR-0304) — including the
  assistant's recorded tool-call requests. `parse`, legacy completions,
  Anthropic `messages.stream()` and `with_raw_response` calls are **refused**,
  not sent to the network. Since v0.105.0 (experimental), a recorded call that **failed** (a rate limit, a 4xx,
  a 5xx after the SDK's retries, a timeout) is replayed by raising the same SDK
  exception class at its position, from an allow-list
  (`replay/_model_errors.py`); the wire hook's per-attempt transport records are
  never served (ADR-0305).
- `MockToolDispatcher` serves recorded **MCP** results through
  `mcp.ClientSession.call_tool`, one recorded result per call (matched by tool
  name and arguments, repeated identical calls in recorded order). A call with
  no recorded result is **refused** — the live tool does not run.
- **Experimental (ADR-0306 slice 1, since v0.105.0):** it also serves functions the
  workload declared with `novafabric.capture.record.tool`, before the function
  body runs, matched by name and signature-bound arguments. Records are servable
  only when captured at the `forensic`/`air_gapped` level with JSON-native
  values; anything else is refused with the reason. *Since v0.106.0
  (slice 4):* a declared function that itself called a model or tool is served too,
  and the records it wrote are consumed as covered; MCP arguments a capsule
  redacted at seal now match the live call (redact-then-hash).
- **Experimental (ADR-0306 slice 3, since v0.106.0):** it also
  serves **Google ADK** tool calls made through the NovaFabric tool plugin
  (`novafabric.adapters.google_adk.make_tool_plugin()`, first in the `Runner`'s
  plugins), through ADK's documented `before_tool_callback` short-circuit — for
  workloads captured with `nova capture python …` only.
- **Experimental (ADR-0306 slice 3, since v0.106.0):** it also
  serves **OpenAI Agents SDK** function tools passed through
  `novafabric.adapters.openai_agents.wrap_tools()`, by replacing the tool's
  `on_invoke_tool` — an **undocumented SDK seam**, checked against
  `openai-agents` 0.17.x — again for `nova capture python …` workloads only.
- **Every other tool runs live**: HTTP requests, shell commands, file writes,
  other framework-native tools, undeclared functions and other providers' SDKs are not
  intercepted. The
  result reports them as `tool_calls_not_interceptable`, and the outbound
  connections the replayed process opened as `network_connections_live`
  (observed, not blocked). *Since v0.106.0 (slice 4, experimental):*
  the result an undeclared function sends back to the model is compared with the
  recorded one and a difference is reported in `replay_contract.tool_result_echo`
  — report-only, never a failure.

The replay **fails closed**: an extra model call, a call on an unsupported
surface, an unmatched MCP, `record.tool`, ADK or Agents SDK tool call, or a recorded response that is never requested
marks the replay `failure` with a `divergence_reason`, even if the workload
caught the exception. `--permissive` (Python: `ReplayFlags(permissive=True)`)
keeps the older behaviour — an empty reply on an exhausted queue, unsupported
model surfaces run live — and still reports every divergence; an unmatched MCP or
`record.tool` call runs live only if a safety-ladder flag permits its class (an MCP
call always counts as `unknown`: `--allow-unknown-mutation`). *(Changed in v0.105.0,
ADR-0306: `--permissive` alone used to run every unmatched MCP call live.)* The result counts what was
actually served: `model_calls_mocked` of `model_calls_available`,
`tool_calls_mocked`/`tool_calls_live`/`tool_calls_unmatched`, and
`queues_fully_consumed`. See the
[support matrix](https://novafabric.ai/docs/architecture/replay-modes/#support-matrix).

### Safety ladder (mocked replay)

The safety ladder classifies the capsule's recorded tool calls by side-effect
level. It drives the `--dry-run` report, and every write-permitting rung
(`--allow-mutating`, `--allow-external-side-effects`, `--allow-unknown-mutation`)
triggers a policy-engine gate (an audited allow/deny) before the replay starts. In
`mocked` mode the run-time rule is fixed by ADR-0300: an intercepted call (MCP
`call_tool`, a `record.tool` function, an ADK tool through the tool plugin, or an Agents SDK tool through `wrap_tools`) is served from the capsule or refused,
and every other tool runs live (the `--dry-run` report marks those `[LIVE]`). The
rungs gate the intercepted surfaces in two places only (ADR-0306, experimental):
under `--permissive` an unmatched call runs live only if a rung permits its class,
and a `replay.yaml` `allow: true` override re-executes a tool only if a rung
permits its class. The class is the one a `record.tool` decorator declares, or
`unknown` for every MCP call — never a class the capsule recorded. Ladder-based
enforcement for tools replay does not intercept is **future design**. The rungs:

```
(none)               — deny all tool calls
  ↓  --allow-readonly
read-only            — permit read-only tool calls
  ↓  --allow-mutating
idempotent-write     — permit idempotent writes
non-idempotent-write
  ↓  --allow-external-side-effects
external-side-effect — permit calls with external effects
  ↓  --allow-unknown-mutation
unknown              — permit unclassified tool calls
```

Per-tool overrides can be specified in `replay.yaml`, one entry per tool name
(`schemas/replay-policy.schema.json`, `ToolOverride`):

```yaml
tool_overrides:
  - tool_name: safe_lookup
    allow: true      # re-execute it -- only if the operator also passes the ladder flag
  - tool_name: send_email
    allow: false     # never run it live
```

In `mocked` mode the overrides are **enforced inside the replayed process**
(experimental, ADR-0306, since v0.105.0) on the two surfaces replay intercepts — MCP
`ClientSession.call_tool` and functions declared with `record.tool` — and `--dry-run`
prints exactly the same decisions. Because `replay.yaml` travels inside the capsule, a
restriction from it is trusted and a permission is not:

- `allow: false` — a recorded result is still served (serving is not re-execution); a
  call with no recorded result is refused, **even under `--permissive`**. If the tool
  was recorded on a transport replay cannot intercept (HTTP, shell, an undeclared
  function), the override cannot be enforced, so a strict replay **refuses to start**
  (`ToolOverrideUnenforceable`, exit 3); `--permissive` starts and reports
  `override_unenforceable`.
- `allow: true` — the tool is re-executed live **only** if the operator also passes the
  ladder flag for its class (an MCP tool always counts as `unknown`, so
  `--allow-unknown-mutation`). Without it the recorded result is served and the result
  reports `override_not_honoured`.

The result lists every override under `replay_contract.tool_overrides`. Outside `mocked`
mode no tool dispatcher exists and the overrides only colour the `--dry-run` report. The
older `action: replay | refuse` form is still read as `allow: true | false`.

### `semantic` mode

**Does not re-execute and calls no model.** It reads the capsule's recorded
model responses and returns `similarity_score` — the mean pairwise text
similarity (`difflib`, 0.0–1.0) between those responses, i.e. how consistent the
recorded run was with itself (1.0 when there are fewer than two responses). The
intended mode — re-run against live models and judge the new outputs against
the recording on *meaning* — is **future design**, not implemented.

### `exact` mode

**Does not re-execute.** Reports `exact_eligible` and the reasons it is not:
the `env.lock` must be `mode: deterministic`, every model call must carry a seed,
and no stored tool result may have drifted from its declared schema (ADR-0128).
It answers *whether* a byte-exact re-run of a local / on-prem model is possible;
performing and hash-verifying that re-run is **future design**.

### `intervention` mode (experimental)

Answers a counterfactual question: *if this one recorded event had gone
differently, would the run's outcome have changed?* An `InterventionSpec`
names one captured model or tool call and a substitute outcome for it; the
engine re-executes everything downstream of that point under mocked
semantics (zero live tokens) and writes a diffable capsule hard-marked
`replay_mode: intervention`, never mistakable for a real run. Only a substituted
**model** response reaches the re-executed workload: intervention installs no
tool dispatcher, so tools run live and a substituted **tool** result changes the
output capsule and the checks only. Since v0.105.0 the result says which
with `intervention.substitution_delivered_to_workload`. A capsule with no command to
re-run is not re-executed at all (`downstream_reexecuted: false`). This is the
building block behind the no-LLM causal-graph diagnostic suite below — see
[Diagnose: causal-graph attribution and counterfactual root-cause
search](#diagnose-causal-graph-attribution-and-counterfactual-root-cause-search-experimental).

---

## Diagnose: causal-graph attribution and counterfactual root-cause search (experimental)

`nova diagnose <run-id>` (ADR-0084) attributes a failed run to its most likely
responsible step: it walks the captured trace plus the lineage graph and
produces a ranked attribution — which agent/step is most likely responsible,
plus an `AgentErrorTaxonomy` label (`MEMORY` / `REFLECTION` / `PLANNING` /
`ACTION` / `SYSTEM` / `UNKNOWN`). Scores are relative ranking weights, not
probabilities, and every candidate is honestly marked `verification:
unverified` unless tested.

Two further, **experimental** layers (ADR-0101) turn a ranking into evidence:

- **No-LLM causal-graph back-trace (§NF-019/022).** `causal_root_candidates`
  back-traces the span graph — no LLM involved, purely structural — to find
  **root failure nodes**: a failing node with no failing ancestor. This is
  what candidates the search below.
- **Replay-proven root cause (`--intervene` / `--search-root-cause`, §NF-017/018).**
  `--intervene` auto-synthesizes an `InterventionSpec` for the top hypothesis
  and replays it under `intervention` mode (see [Replay Modes](#replay-modes))
  to check whether the outcome actually flips. `--search-root-cause` widens
  this into a bounded sweep over the NF-019 candidates, earliest/shallowest
  first, until one confirms a flip — the decisive root cause, replay-proven
  rather than merely ranked.

None of this uses an LLM judge: attribution is structural (trace + lineage
walk) and verification is a real, zero-token replay — a hallucination-risk or
root-cause finding you can point at evidence for, not a model's opinion.

---

## Accountability Spine (experimental)

Three complementary, append-only evidence surfaces (ADR-0093/0094/0095),
collectively referred to as the Accountability Spine, sit alongside NovaSeal
and the Evidence Bundle:

| Surface | CLI | Records | ADR |
|---|---|---|---|
| Energy-Anchored Action Receipts | `nova energy probe/attest/verify/report` | Per-action energy receipts + a conservation check, so an energy claim is falsifiable rather than asserted | ADR-0093 |
| Adversary-anchored accountability ledger | `nova ledger anchor/verify/status` | Sidecar hash chains + signed checkpoints for every capsule `.jsonl` stream, so tampering with a stream after the fact is detectable | ADR-0094 |
| Structured safety case | `nova safety-case build/verify/export` | A compiled, evidence-grounded Claims-Arguments-Evidence (CAE) safety case over a capsule, exportable as JSON, Markdown, an EU AI Act Annex IV section, or a NIST AI RMF report | ADR-0095 |

The dashboard surfaces all three read-only under the **Spine** tab. Like the
[evidence facets](#evidence-facets-experimental) above, these are record-only:
they never enforce, adjudicate, or gate anything themselves — they make an
accountability claim checkable.

---

## Structural Diff

`nova diff` compares two capsules field by field:

- **Model call alignment** — a `parent_span_id` unique on both sides pairs
  exactly; the remaining calls pair by sequence position, anchored on
  identical requests so an inserted or deleted call does not shift the rest.
  Aligned pairs are compared for the request (provider, model and messages)
  and the response choices.
- **Tool call alignment** — calls are matched by `tool_name` + argument hash,
  each call used once, then by position among calls with the same tool name.
  A pair is changed when its arguments, result, `status` or `error.type` differ,
  including the `io.novafabric.arguments_digest` / `result_digest` extensions
  that are all a record holds at the default capture level (a digest on one
  side only counts too). Releases up to v0.107.0 compared only `arguments` and
  `result`, so a tool that started failing diffed as unchanged.
- **Environment diff** — Python version, OS, architecture, interpreter path.
- **Output diff** — every regular file under `outputs/`, recursively, by
  capsule-relative path and content hash. Symlinks are skipped, never followed.

`DiffReport` has three counters: `changed`, `added`, `removed`, and one
property, `has_changes`, that is true when any of them is non-zero. The
`--assert-no-regressions` flag exits 1 on `has_changes`, and the text and
GitHub-annotation output read the same property, making it suitable as a CI
gate. Exit 1 means only that; a comparison that could not be made (for
example, a capsule path that does not exist, or a capsule with unreadable
record lines, which are counted and reported rather than skipped silently)
exits 2:

```bash
# Fail the pipeline if today's run diverges from a known-good baseline
nova diff baseline-capsule/ candidate-capsule/ --assert-no-regressions
```

This is the mechanism behind the "worked yesterday, fails today" flaky-agent
problem: wire `nova diff --assert-no-regressions` into CI and a structural
regression stops the merge.

### Offline analytics over capsules (experimental)

Because every metric a run produced — cost, tokens, latency, scores — is already
recorded inside its capsule, analytics never needs a server. The **experimental**
v0.59 surfaces read those recorded facts, offline and read-only: `nova query` is
a bounded filter → group-by → aggregate DSL over the local capsule directory,
`nova view` persists a named query as a small versionable file, and `nova trend`
buckets one metric over time or by asset into a JSON/HTML report. Nothing is
recomputed or fetched; the capsule directory is never written. See the
[CLI reference](https://novafabric.ai/docs/cli-reference/query-search-drift/#offline-metrics-query-experimental-adr-0129).

---

## Lineage Graph

The lineage graph is a directed graph stored in SQLite — a rebuildable cache
derived from each capsule's `lineage.jsonl`. It has two table types:

- **`lineage_nodes`** — runs, assets, and artifacts. Each node has a
  deterministic ID: the first 26 hex characters of SHA-256(`kind:ref`)
  (`lineage/_types.py::node_id_for`).
- **`lineage_edges`** — typed, directional edges between nodes.

Because the graph is derived from the capsules, it can always be rebuilt from
them — the capsules remain the source of truth. The SQLite backend is the
local-mode default and is crash-safe (WAL); it is well suited below roughly one
million edges. Four **at-scale backends** exist for larger graphs, all
**experimental** and testcontainers-verified, selected via `nova lineage-store
migrate` / `profile`: **Kuzu** (embedded, benchmark-cleared at 10M edges, p99
`blast_radius` 45.5ms), **Postgres** (recursive-CTE, no extension needed),
**Apache AGE** (openCypher on Postgres), and **JanusGraph** (Gremlin, needs the
GraphSON serializer). There are currently zero `NotImplementedError` stubs
among them. Billion-edge cross-cluster **federation** remains **FUTURE
DESIGN**, not implemented.

### Edge types

The capsule's `lineage.jsonl` vocabulary (`src/novafabric/schemas/lineage-edge.schema.json`),
labelled by who writes each type today:

| Edge type | Meaning | Written by |
|---|---|---|
| `consumed` | The run read/used this asset | `nova capture`, from `assets.jsonl` |
| `produced_by` | This artifact was produced by the run | `nova capture`, from the capsule's outputs (or `produced.jsonl`) |
| `replayed_from` | This run is a replay of the referenced run | `nova capture`, from `replay_of_run_id` in `capsule.yaml` |
| `depends_on` | This asset depends on another asset | the registry, from an asset's declared dependencies |
| `migration_source` | This v1.0.0 capsule was upgraded from a v0.1.x capsule (same run) | `nova migrate` (ADR-0034 §6) |
| `derived_from`, `evaluated_by`, `attested_by` | Reserved by the schema | no NovaFabric writer emits them today |

Two further vocabularies share the lineage store
(`lineage/_types.py::LINEAGE_EDGE_TYPES`): the typed parent/child and session edges
(`contains`, `spawned`, `delegated_to`, `member_of_session`; ADR-0044, ADR-0122) and
the memory-provenance pair `wrote_memory` / `read_memory` (ADR-0143). The memory
edges are **derived on demand** from a capsule's memory-operation events
(`lineage/memory.py`), keyed by memory key only; capture does not write them to
`lineage.jsonl`.

### Confidence levels

| Confidence | Meaning |
|---|---|
| `observed` | Directly recorded at runtime (e.g., an explicit asset reference in `assets.jsonl`) |
| `inferred` | Derived from capsule structure (e.g., output file presence) |
| `declared` | Stated by a record the run or capsule carries (e.g., `produced.jsonl`, `replay_of_run_id`) |

### Graph queries

| Query | Direction | Use case |
|---|---|---|
| `provenance` | Backward (ancestors) | "What did this run depend on?" |
| `blast-radius` | Forward (descendants) | "What runs consume this asset?" |
| `replay-chain` | Backward via `replayed_from` | "What is the original run for this replay?" |
| `time-travel` | Backward, filtered by timestamp | "What was the lineage state as of time T?" |

---

### Graph analytics (experimental)

Traversal answers "what connects to this node?". A read-only **analytics layer**
(ADRs 0212–0215) answers "what does the whole graph mean?": centrality
(`nova lineage metrics` — hubs and single points of failure), node-level
root-cause ranking (`nova lineage root-cause`), interop export to
GraphML/GEXF/Cypher (`nova lineage export-graph`), and a synthesized
intelligence report (`nova insights`). All are descriptive rankings for
attention, not calibrated importance, and degrade honestly when a data source
(such as cost) is absent.

## OpenLineage Integration

NovaFabric emits [OpenLineage](https://openlineage.io) 2.0.2 events from
capsules. Events are typed as `START` and `COMPLETE` (or `FAIL`).

Events can be emitted:

- Automatically at capture time if `OPENLINEAGE_URL` is set
- On demand via `nova lineage emit-openlineage`

This enables integration with data-catalog tools such as Marquez, Atlan, and
OpenMetadata — so NovaFabric's run-level provenance can join the broader
data-pipeline lineage you already track.

**Custom run facets (experimental).** `emit-openlineage --with-facets` attaches
NovaFabric-specific facets to the COMPLETE event — the capsule id/hash, the eval verdict,
the promotion-policy decision, and reproducibility run params — and
`--otel-correlation` adds `trace_id`/`span_id` so a lineage node links to its OTel GenAI
spans. Facets are additive and schema-validated before emission, so a catalog that
ignores them still receives valid core OpenLineage events. Run facets, dataset provenance
cards, and benchmark-contamination checks are shown hands-on in the
[feature tour §17](https://novafabric.ai/docs/tutorials/feature-tour/#17-prove-supply-chain-provenance--eval-integrity).

> **Note:** these OpenLineage facets are unrelated to the capsule
> [`facets` container](#evidence-facets-experimental) (ADR-0196). Same word, two
> different places: OpenLineage facets live in emitted lineage events; evidence
> facets live on the capsule.

---

## AI Asset

An **AI asset** is any versioned artifact in an ML or LLM system. NovaFabric
tracks seven types:

| Type | What it represents |
|---|---|
| `model` | A trained ML/LLM model — framework, artifact path |
| `agent` | An AI agent — model ref, tools, prompts, policies, eval suites |
| `prompt` | A prompt template used by one or more agents |
| `tool` | A callable function exposed to an agent |
| `dataset` | A dataset used for training, fine-tuning, or evaluation |
| `evaluation` | An evaluation suite definition (test cases, scoring logic) |
| `deployment` | A running endpoint — base URL, environment |

Assets are described by YAML spec files and stored in the local Asset Registry.
Every registered asset is addressed as `name@version`, pinned to a git commit
SHA, and carries a lifecycle status that tracks where it is in your
development-to-production pipeline.

**Prompts as versioned assets (experimental, ADR-0112).** Beyond the YAML spec
route, `nova prompt register` turns a prompt into an **immutable,
content-addressed** registry version: every edit registers a new version (never
a mutation), and a run references it as `prompt:<id>@<version>+sha256:<hex>` —
so a capsule can be tied to exactly the prompt bytes that ran. Mutable
deployment labels (`nova label`, ADR-0113) point a name like `production` at one
immutable version, and prompt composition (`nova prompt compose`, ADR-0115)
snapshots pinned includes so an assembled prompt rebuilds byte-identically.
NovaFabric never renders or serves prompts — it versions and proves them.

---

## Asset Lifecycle

### The state machine

```
development ──► validated ──► pending_approval ──► staging ──► production ──► archived
      │                                                  ▲
      └──────────────────────────────────────────────────┘
           (direct skip: development → staging or production)
```

All six statuses in plain language:

| Status | Meaning |
|---|---|
| `development` | Work in progress. Default when registered. No constraints. |
| `validated` | Passed automated checks (`nova validate`). Spec is well-formed, secrets are clean. |
| `pending_approval` | Waiting for a human sign-off (`nova approve`). Useful in regulated environments. |
| `staging` | Cleared for pre-production use. **Agents require a passing eval to reach here:** the agent's most recent eval must have passed (an earlier pass does not clear a later failure). |
| `production` | Live. **Agents require a passing eval to reach here** (same rule). |
| `archived` | Retired. No further promotion possible. Capsules and lineage are still readable. |

Direct skips (e.g. `development → production`) are allowed. Once archived, an
asset cannot be re-promoted. The `validated` and `pending_approval` steps are
optional — small teams often go straight from `development` to `staging`.

### What actually changes when you promote an asset

**Promoting an asset updates one database record.** That is all NovaFabric does.

```
nova promote direct my-agent@v1.2.0 --to staging
```

Internally this:

1. Checks the policy engine — allow or deny, written to the audit log. A target of
   `production` is a *protected stage* (ADR-0312): a single actor needs an explicit
   `promote_direct` policy allow, and none exists by default, so `production` is
   normally reached through `nova promote propose` + `nova promote approve`.
2. For `agent` type going to `staging`/`production`: verifies a passing eval
   result exists. Blocks if not (unless `--force`).
3. Runs `UPDATE assets SET status = 'staging', promoted_at = ..., promoted_by = 'cli-user'`
   (the CLI records the fixed actor `cli-user`; a named human sign-off is the
   approval record written by `nova approve <name@version> --approver alice`).
4. Returns the updated record.

**Nothing else happens.** Your agent does not restart. Your prompt does not
reload. No traffic is rerouted. No container is restarted. The capture hooks
and runners are unchanged.

The status is **governance metadata** — a reliable signal your tooling can
read and act on. NovaFabric gives you the signal; your platform decides what
to do with it. (Policy checks and maker-checker approval are backed by OPA/Rego
and signed decision logs — see [Eval-Gated Promotion](#eval-gated-promotion).)

### How external tooling consumes the status

The pattern is: your CI/CD pipeline (or any script) reads the status from
NovaFabric and makes a deployment decision based on it.

```bash
# In a GitHub Actions deploy step or Argo CD sync hook:
# (`nova inspect` prints a human-readable panel, not JSON — read the record
# through the Python API instead.)
STATUS=$(python -c 'from novafabric.registry.service import get_asset; print(get_asset("my-agent", "v1.2.0")["status"])')

if [ "$STATUS" = "production" ]; then
  kubectl set image deployment/my-agent container=my-agent:v1.2.0
else
  echo "Asset not in production — deploy blocked (status=$STATUS)"
  exit 1
fi
```

NovaFabric doesn't own your deployment. It owns the evidence that the asset
is ready for deployment.

### Use cases

**Use case 1 — CI/CD gate for an LLM-powered feature**

You have a customer-facing summarisation agent. Your CI pipeline runs evals
on every PR. Only if evals pass and a team lead approves does the agent reach
`production`, which triggers your Helm chart to update the image tag.

```bash
# .github/workflows/deploy-agent.yml (simplified)
- run: nova eval agent summariser@${{ github.sha }}
- run: nova promote propose summariser@${{ github.sha }} --to production --identity ci-bot
# ... the team lead countersigns from their own machine (a separate, trust-listed key):
#     nova promote approve summariser@<sha> --identity team-lead
# ... a later job, once the registry reports `production`:
- run: helm upgrade summariser ./chart --set image.tag=${{ github.sha }}
```

`production` is a protected stage, so a single-actor `nova promote direct --to
production` in CI is refused by default (ADR-0312).

**Use case 2 — Safe prompt rollout without a full redeploy**

You update a system prompt template. You don't want to redeploy the entire
agent service — just swap the prompt version your agent reads at startup. The
registry tells your agent loader which version is `production`.

```python
# agent startup code
from novafabric.registry.service import get_asset

asset = get_asset("system-prompt-v2", version="latest")
if asset["status"] != "production":
    raise RuntimeError(f"Prompt not promoted to production (status={asset['status']})")

prompt_text = json.loads(asset["spec_json"])["spec"]["template"]
```

If the prompt has a bug and you archive it, the next agent restart fails fast
rather than silently loading a bad prompt.

**Use case 3 — Audit trail for a regulated environment**

Your company needs to demonstrate that no AI model reached production without
a human sign-off. Every promotion through `pending_approval` creates an
approval record (approver, timestamp, note) and an audit log entry. At audit
time:

```bash
python -c 'import json; from novafabric.registry.service import get_asset
a = get_asset("risk-scorer", "v3.1.0")
print(json.dumps({k: a[k] for k in ("status", "promoted_by", "promoted_at", "forced_promotion")}))'
# → { "status": "production", "promoted_by": "cli-user", "promoted_at": "2026-03-15T10:22:01Z", "forced_promotion": false }
```

No manual tracking spreadsheet. The registry is the record.

**Use case 4 — Blast-radius control when an agent misbehaves**

An agent in production starts producing bad outputs after an upstream model
update. You archive it:

```bash
nova promote direct my-agent@v2.0.0 --to archived
```

Your deployment hook detects the status change, rolls back to the previous
`production` version (`my-agent@v1.9.0`), and pages the team. The capsules
from the bad version are still fully intact for forensic replay — nothing
is deleted.

**Use case 5 — Multi-team handoff on an HPC cluster**

A research team develops and validates an agent on their workstation. When
they're ready for the ML engineering team to integrate it into the batch
pipeline, they promote it to `staging`. The ML team's Slurm job template only
launches agents whose status is `staging` or `production`:

```bash
# check_asset_status.sh — called from sbatch prolog
STATUS=$(NOVAFABRIC_DB_PATH=/shared/nova/registry.db python -c \
  'import sys; from novafabric.registry.service import get_asset; print(get_asset(sys.argv[1], sys.argv[2])["status"])' \
  "$AGENT_NAME" "$AGENT_VERSION")
if [[ "$STATUS" != "staging" && "$STATUS" != "production" ]]; then
  echo "PROLOG FAIL: $AGENT_NAME@$AGENT_VERSION is not ready (status=$STATUS)" >&2
  exit 1
fi
```

The registry on the shared filesystem (`/shared/nova/registry.db`) is the
handoff protocol between the two teams — no Slack messages, no wiki pages.

---

## Asset Registry

The registry is a local SQLite database at `~/.novafabric/registry.db`
(override: `NOVAFABRIC_DB_PATH`). It stores:

| Table | Contents |
|---|---|
| `assets` | Every registered asset — name, version, type, status, full spec JSON, git SHA, promotion history |
| `eval_results` | Pass/fail eval results per asset, per suite |
| `approvals` | Human sign-off records: approver, timestamp, note |

The registry is entirely local. There is no server, no credentials, and no
network dependency for local mode.

In server mode (`nova server start`, experimental — not the single-user
`nova serve` dashboard), the same data lives in the server's database (SQLite by
default, Postgres via `backend: postgres`) and is accessible via the REST API
under `/v0/assets`, guarded by bearer-token auth (OIDC, offline tokens, API keys
or the local token) and an RBAC model (`reader < writer < admin`, plus an
orthogonal `auditor`). Server mode is strictly additive: **local mode never requires it.**

---

## Eval-Gated Promotion

`nova eval agent <agent@version>` discovers evaluation suites declared in the
agent spec (`spec.evals`) and resolves them via Python entry points in the
`novafabric.evals` group. Results are stored in the `eval_results` table.

Promotion to `staging` or `production` queries `eval_results` for a passing
result. If none exists, promotion is blocked unless `--force` is used. A
forced promotion is recorded as such (`forced_promotion = true`) and appears
in `nova inspect` output and the audit log — the escape hatch is visible,
not silent.

Standard eval suites ship OCI-pinned for reproducibility: **GAIA, SWE-bench,
AgentBench, MMLU, TruthfulQA, and a Smoke suite** (v0.9; the `novafabric.evals`
entry points in `pyproject.toml`). Promotion can be Rego-gated to
block on a regression against a prior result.

---

## Evidence Bundle

An **Evidence Bundle** is the compliance primitive: a signed, self-contained ZIP
built by `nova export-evidence` that embeds

- a copy of the Run Capsule,
- its lineage edges,
- the JSON schemas,
- Ed25519-signed in-toto DSSE attestations, and
- a `manifest.json` with a SHA-256 for every file, plus the verification recipe

([full list](https://novafabric.ai/docs/architecture/sealing-and-verification/#what-is-in-the-bundle)).

Its defining property is **offline verifiability**: a recipient can verify it
with only `sha256sum` plus an ed25519 verifier — **no NovaFabric runtime
required**. That is what makes the capsule a durable, portable artifact you own
rather than a row in someone else's database. A capsule missing its
`redaction-proof.json` cannot be exported (see
[Secret Scanning and Redaction](#secret-scanning-and-redaction)).

**Shipped trust-layer surfaces (v0.4+):** ed25519-signed Evidence Bundles,
in-toto DSSE attestations, secret scanning with verifiable redaction proofs,
OPA/Rego policy gates with maker-checker promotion, and WORM storage adapters
(S3 Object Lock / Azure immutable blob / GCS Bucket Lock, with legal holds).

**Compliance-export cohort (experimental, ADR-0107 + related).** Beyond the
general-purpose Evidence Bundle, `nova export-compliance` and a long tail of
dedicated `nova export-*` commands render capsule/lineage evidence into
regulator-shaped artifacts: EU AI Act (Annex IV, Art.12 record-keeping, Art.50
marking + C2PA/SynthID assertion, Art.53 GPAI hash-chained form, Art.73
incidents with a deadline clock chained to Art.72 post-market monitoring),
ISO 42001/42005, NIST AI RMF, the NIST GenAI/CSA profile, GDPR (Art.30 RoPA,
Art.17 erasure), CycloneDX AI-SBOM, and more. Every exporter is a **pure
projection over already-captured evidence** — it renders facts, never
adjudicates compliance — and each is marked with an `evidence_source`
provenance tag (`operator_asserted` / `capsule_verified` / `unverifiable`,
specified in the maintainers' private `design/` tree)
so a reader can tell what was actually observed from what an operator merely
declared.

> **Experimental — NovaSeal signing core.** The in-process NovaSeal core
> shipped in v0.10+ (`experimental`): DSSE signing (ECDSA P-256), opt-in
> **RFC 3161 trusted timestamps** (no default TSA and no network call unless
> `tsa_url` is set — ADR-0292), and an append-only SQLite Merkle log,
> verified offline by `nova verify` (`signature_ok` / `timestamp_ok` /
> `log_integrity_ok`, plus `timestamp_trust`), plus the maker-checker `nova seal propose/approve/verify`
> chain (ADR-0059). Still **PLANNED / FUTURE DESIGN** per
> [ADR-0041](https://novafabric.ai/docs/decisions/): the
> dedicated network signing *service*, qualified timestamps, Sigstore-keyless
> signing as the default path, WORM-backed retention at the seal layer, and
> PQC/ML-DSA. The most stable portable proof today remains the ed25519 /
> in-toto DSSE Evidence Bundle described above.

---

## SDK Decorator

The `@agent` decorator (`from novafabric.sdk.agent import agent`) is an in-process alternative to `nova capture`. It wraps
an agent function with the same capture hooks used by the CLI, recording all
LLM calls into a capsule. Without `capsule_dir`, it emits OTel spans only —
this is the v0.1 observability mode.

---

## ULID

Run IDs and replay IDs are [Universally Unique Lexicographically Sortable
Identifiers](https://github.com/ulid/spec). They embed a millisecond timestamp
as the high bits, so capsule directories naturally sort chronologically without
a separate index.

---

## Summary and next steps

You now have the vocabulary NovaFabric is built on:

- **Run Capsule** — the ULID-named directory that is the source of truth for one
  execution, written on success and failure.
- **Capture** — zero-code-change instrumentation via a `sitecustomize.py`
  loader, per-SDK hooks, and a wire-level safety net down to `urllib3`.
- **Replay** — five explicit modes (`forensic`, `mocked`, `semantic`, `exact`,
  `intervention`); only `mocked` and `intervention` re-execute, and `intervention`
  remains experimental, used for counterfactual replay.
- **Diff** — field-by-field structural comparison, wireable as a CI regression
  gate with `--assert-no-regressions`.
- **Diagnose** — structural (no-LLM) causal-graph attribution over a failed
  run, with an experimental counterfactual root-cause search that
  replay-proves rather than merely ranks a hypothesis.
- **Lineage** — a rebuildable provenance graph answering `provenance`,
  `blast-radius`, `replay-chain`, and `time-travel`, with OpenLineage emission;
  SQLite is the local default, with four experimental at-scale backends
  (Kuzu, Postgres, AGE, JanusGraph) for larger graphs.
- **Registry + lifecycle** — versioned assets with a six-state lifecycle where
  promotion is governance metadata only.
- **Evidence Bundle** — offline-verifiable signed export; the NovaSeal signing
  core (DSSE + RFC 3161 timestamps + Merkle log, `nova verify`) is
  **experimental**, as is the Accountability Spine (energy receipts, ledger,
  safety case) and the EU AI Act / ISO / NIST compliance-export cohort, while
  the dedicated signing service and seal-layer WORM retention remain
  **PLANNED**.

Where to go next:

- **CLI reference** — every command and flag: [`docs/cli-reference.md`](https://novafabric.ai/docs/cli-reference/)
- **Writing a hook plugin** — [`docs/integrations/writing-a-hook-plugin.md`](https://novafabric.ai/docs/integrations/writing-a-hook-plugin/)
- **Model-call field reference** — `design/spec/model-call-v0.md` (private)
- **Runner design and anti-patterns** — ADR-0025
- **Trust-layer and sealing design intent** — [ADR-0041](https://novafabric.ai/docs/decisions/)