Concepts

This page is the conceptual reference for NovaFabric. It explains the nouns you will meet everywhere else in the docs and on the nova command line — what a Run Capsule is, how capture works without touching your code, the five replay modes, structural diff, lineage, the Asset Registry and its lifecycle, and how signed evidence is produced.

What you will learn

The five primitives NovaFabric is built on — Asset Registry, Run Capsule, Replay, Lineage, and Evidence Bundle — and the two journeys that branch from the Run Capsule: Capture → Replay → Diff (developer) and Capture → Seal → Verify → Audit (trust; sealing is opt-in).
The exact on-disk anatomy of a Run Capsule and why it is the source of truth (everything else is a rebuildable index).
How zero-code-change capture works via a sitecustomize.py loader and per-SDK plus wire-level hooks.
The five replay modes — forensic, mocked, semantic, exact, and experimental intervention — and what each one honestly promises.
How structural diff becomes a CI regression gate, and how lineage answers provenance and blast-radius questions.
How the Asset Registry lifecycle works and why promotion is governance metadata only — it never restarts or redeploys anything.
What is shipped today versus planned (clearly labeled throughout).

Maturity note. NovaFabric is local-first and beta / pre-v1. Nearly all shipped surfaces carry experimental maturity: they work today and are tested, but on-disk formats are not frozen until the v1.0 schema freeze. Anything labeled PLANNED or FUTURE DESIGN below is documented design intent (in the ADRs) and is not implemented — never treat it as shipped.

The five primitives at a glance

NovaFabric is framed around exactly five primitives, each with a public spec and JSON Schema. Cryptographic sealing is part of the Evidence Bundle / trust layer, not a sixth primitive.

#	Primitive	What it is	Since
1	Asset Registry	Local SQLite registry of versioned AI assets (`name@version`), pinned to a git SHA, with a six-state lifecycle	v0.1
2	Run Capsule	The fundamental unit of capture: a ULID-named directory holding every observable fact of one execution	v0.2
3	Replay	Re-execute or inspect a capsule with external calls controlled, in five explicit modes	v0.3
4	Lineage	A directed provenance graph (SQLite cache) with mechanical edge types and OpenLineage emission	v0.4
5	Evidence Bundle	A signed, self-contained ZIP an auditor can verify offline with only `sha256sum` + an ed25519 verifier	v0.4

The design invariant behind all five: the capsule is the source of truth; the registry, metadata DB, and lineage graph are derived, rebuildable indexes. Only two top-level formats exist — Run Capsule and Evidence Bundle — and introducing a third requires an accepted ADR.

Run Capsule

A Run Capsule is the fundamental unit of capture in NovaFabric. It is a directory recording a single command execution: the command, timing, environment, the LLM calls and tool invocations capture can see, stdout/stderr, and a secret-scan record (rule-based scanning for known key and token patterns).

Capsules are identified by a ULID — a lexicographically sortable, time-prefixed unique identifier.

Capture writes each capsule to $NOVAFABRIC_CAPSULE_DIR/<ulid>/ when that variable is set, otherwise to $NOVAFABRIC_HOME/capsules/<ulid>/ (~/.novafabric/capsules/<ulid>/ by default; _paths.default_capsule_dir).

~/.novafabric/capsules/01HXAY7M5JZ8R7K4P9DPBYK2WX/
  capsule.yaml          ← run manifest
  trace.jsonl           ← execution spans (OTel)
  model-calls.jsonl     ← LLM API calls (OTel GenAI semconv)
  tool-calls.jsonl      ← tool invocations
  assets.jsonl          ← asset references
  env.lock              ← environment snapshot
  redaction-proof.json  ← secret scan proof
  replay.yaml           ← replay constraints
  lineage.jsonl         ← lineage edges
  inputs/
  outputs/
    stdout.txt
    stderr.txt

Capsules are written on success and failure. A failed run produces a complete capsule with status: failure and an error block — the evidence of a failure is just as durable as the evidence of a success.

Capsules support additive, optional extensions: blocks (for example slurm, kubernetes, ray, openlineage) so the schema can grow without introducing a new top-level format. Readers tolerate unknown fields; old capsules stay readable forever.

Links inside a capsule are never followed (works today)

A capsule is a plain directory, so anyone who can write into it can plant a symbolic link such as inputs/data.csv -> ~/.ssh/id_rsa. NovaFabric never writes a link into a capsule, and neither a ZIP entry nor the capsule Merkle root can represent one. So nothing that copies, archives or serves capsule files follows one. They all share one module (novafabric._capsule_fs):

Refused with a named error, nothing written: nova export-evidence, nova export-blob (exit 2, "Export refused"), nova migrate and /api/capsule-migrate, nova export-rocrate and /api/compliance/export/rocrate (HTTP 422), nova backup create (also for links under spool/ and seal/ratchet/), nova audit bundle, nova export-examiner bagit, and the SDK's upload_capsule. Any link counts: one to a file or a directory outside the capsule, a dangling one, and one that stays inside the capsule. The error names each link and where it points.
Never followed, even when planted mid-export: files are opened one path component at a time with O_NOFOLLOW, so a link swapped in after the check is refused at open time instead of copied.
The capsule Merkle root: the root (capsule_merkle_root, both the v1 and v2 leaves) is the signed subject of Evidence Bundles, the nova serve evidence cart, consent attestations, AI-BOM citations, OpenLineage capsule facets, chain-of-custody blocks, session bundles, baseline pins, experiment and offline-eval scores, and comment, annotation and score-submission subjects. It walks the capsule without following a link and refuses a capsule holding one, so it can never commit to a file outside the capsule. No root is computed over the remaining files. The CLI prints one error: line and exits 2 (nova score submit reports capsule_unusable, exit 2; nova consent attest exits 1, and nova consent verify-attestation reports the root check as failed, exit 1, not stale). nova serve answers 422, also for the evidence cart even with accept_unresolved, so a linked capsule never becomes a recorded omission. Server-mode score submission answers 409. In nova experiment run the item is recorded as an error, with no score. Each file is opened with O_NOFOLLOW and read in chunks. A FIFO, socket or device in the capsule is left out without being opened, as it always was, and a file swapped for one mid-hash is refused. So the root never blocks. For a capsule without links, the root is byte-identical to the one earlier releases computed.
nova serve: GET /api/runs/{id}/file/{path} answers 404 for a link (or a FIFO) and never returns its target. A run whose capsule holds a link or a special file (FIFO, socket, device) at its top level (capsule.yaml, a JSONL stream, redaction-proof.json, inputs, outputs) is refused with 403 by every /api/runs/{id}/... route. The run listing never parses a linked capsule.yaml, and the inputs/outputs listings leave links out.
Scans of many capsules (the nova serve run listing, search, cost report, suggest-register, run lineage, children, topology, AI-BOM status, the run index, KG bulk ingest, nova query, the examiner and audit exporters, server mode's capsule listing) never list an entry of the capsule directory that is itself a link, whether it points inside the directory (latest -> 01J..., an alias of a capsule the scan already lists) or outside it. Every file they read in each capsule goes through the same no-follow open, so a link or a FIFO planted in a sibling capsule is never followed either.
A capsule root that is a link, in nova serve: a run named directly (/api/runs/latest) whose directory entry is a link is served only when the link resolves inside the server's read roots (the capsule directory, $NOVAFABRIC_HOME minus its secrets, any NOVAFABRIC_SERVE_PATH_ROOTS entry); otherwise every route answers 403. The server reads through the resolved path.
Readers: every command that parses one well-known capsule file (nova export --html, nova diagnose, nova diff, nova replay, nova assure, nova validate, nova verify, nova eval contamination-check, the lineage and OpenLineage importers, the knowledge-graph consumer, the content and query indexes, and the compliance exporters) opens it the same way. A link, a FIFO or a device is never followed and never blocks.
A required file that is present but unusable is an error, never empty output. An absent file still means "no data". But when capsule.yaml, capsule.json or a stream a command's output depends on (model-calls.jsonl, tool-calls.jsonl, trace.jsonl, lineage.jsonl, replay.yaml, env.lock, redaction-proof.json, ...) is there but is a link, a FIFO, a directory, an unreadable or an over-size file, every command whose output claims to be complete or correct stops. That covers every export and compliance artifact, report, verdict and attestation, replay, migration, lineage write, PROV-O mapping and annotation-queue fill. The CLI prints one error: line naming the file and exits 2, unless the command has its own refusal code (nova validate reports it as a validation error, exit 1). It writes no output file. nova serve answers 422 and server mode answers 409. Such a command never reports 0 model calls, an empty Art. 12 log, a "no secrets detected" assurance pass or a best-effort replay pin for a file nobody could read. Only best-effort listings (the dashboard run list, search and content indexes, session view, lineage child listings) still skip such a file. They log capsule_file_not_followed, and the knowledge-graph bulk ingest counts it as skipped. nova export --html skips such a section with a warning on stderr and in the page; a linked capsule.yaml refuses the export (exit 2).
nova restore refuses a set whose listed member is a link entry, and never writes through a link at a destination path. nova migrate-schema skips (and reports) a capsule holding a link, and AI-BOM generation never writes aibom.json through one.
Writers into a finished capsule never write through a link either: nova energy attest and the SLURM runner's sacct receipt (energy-receipts.jsonl), nova validate --write (tool-calls.jsonl), the registry's asset-consumption append (assets.jsonl), every score append to scores.jsonl (nova eval offline, the annotation queue, score submission, nova experiment run), and the nova capture workload's capture-health hand-off all open the file with O_NOFOLLOW (_capsule_fs.open_capsule_file_for_write). A link, a dangling link or a FIFO at that path is refused with a named error (nova energy attest exits 2; the SLURM receipt is skipped with a warning) and its target is left untouched. Whole-file writers replace the entry by an atomic rename instead, so a link or hard link planted at the name is replaced, never written through: the dataset-provenance facets (extensions/dev.novafabric.dataset-provenance/; a linked extensions/ directory is refused), nova export-c2pa's default c2pa-manifest.json, and nova capture --emit-otel-genai's otel-genai-spans.json.
nova capture itself, once the workload has run. The workload writes into its own capsule, and with --runner docker the capsule directory is mounted read-write into the container (with --runner kubernetes it is copied back by kubectl cp). So capture treats the capsule as untrusted input from the moment the runner returns. Following a link the container planted would be a container-to-host write: before this was fixed, a model-calls.jsonl link made the secret scanner rewrite the file it pointed at, and a FIFO at tool-calls.jsonl hung capture forever. Now, before any host-side read or write, capture checks every capsule-managed path (the JSONL streams, capsule.yaml, env.lock, replay.yaml, redaction-proof.json, capture-health.json, lineage.jsonl, c2pa-manifest.json, redaction_manifest.json, outputs/stdout.txt, outputs/stderr.txt, the inputs/ and outputs/ directories, and .seal/, which cannot exist yet) with lstat and never opens it. A link, FIFO, socket or device there, a directory where a file belongs, or any .seal/ is removed without being followed (removing a link never touches its target), and the empty stream or directory is recreated. Capture then walks the whole capsule tree, one O_NOFOLLOW directory at a time, and removes every symlink, FIFO, socket and device at any depth: inputs/data.csv -> ~/.ssh/id_rsa, outputs/media/blob.bin, extensions/..., a linked outputs/dirlink, a FIFO at inputs/pipe, a link at scores.jsonl. NovaFabric never writes any of these, so each one was planted, including one that points inside the capsule (for example outputs/latest -> model.bin). A directory nested more than 64 levels deep is not descended and is recorded as not-swept. Each entry found marks the capsule tampered: metadata.capture_tampered in the signed manifest, tampered_entries in capture-health.json, and a skipped <path>#tampered target (skip_reason: "stream_tampered: ...") in redaction-proof.json (the first 1,000 entries are listed one by one; the summary always gives the total). Every later write is an atomic rename or an O_NOFOLLOW append, and every later read is a no-follow, non-blocking open, so an entry planted after the sweep (by a process the workload left running, which only the local runner allows) is refused or replaced, never followed. That covers the --pii-subject stage too: it reads and rewrites every target, and writes redaction_manifest.json, the same way. When the evidence digests are computed, any link or special file found anyway is added to metadata.capture_tampered ("planted after the sweep: ...") before the manifest is sealed. After the sweep the capsule directory is pinned by its device and inode: if it is then swapped for a link or another directory, every read and write refuses (CapsuleRootChangedError) and nova capture prints ✗ capture integrity: ... and exits 1. Otherwise capture stays fail-open (ADR-0021): it exits with the workload's own exit code and prints ✗ capture integrity: ... on stderr. nova verify and nova validate fail a tampered capsule (exit 1). If the capsule directory itself was moved and replaced while the workload ran, capture writes nothing more and says so.
nova verify and nova validate fail any link or special file, wherever it is in the capsule and whenever it was planted, including after sealing: each is named in the output (nova verify --json lists them under special_entries and in errors) and the command exits 1. The walk never follows or opens an entry, so a FIFO cannot hang either command.

The capsule path you pass to a CLI command may itself be a link (runs/latest); only links inside it are refused. That one link is followed because you named it. A scan never follows one it finds, and nova serve follows one only into its read roots.

Hard links are read, never written through. A hard link is an ordinary directory entry for a regular file, so it is read and exported like any other file, and capture does not record it as tampering. Under Linux's default fs.protected_hardlinks = 1, a user can only hard-link a file they own or can already read and write, so a planted hard link exposes nothing the planter could not read already. NovaFabric never creates hard links. Refusing every file with more than one link would break capsule trees kept in rsync --link-dest or cp -al snapshots, and workloads that build a virtualenv in outputs/ with a hard-linking installer. What a hard link can no longer do is carry a write out of the capsule: a whole-file rewrite (the secret scanner, the maskers, the --pii-subject stage, the facet and C2PA writers) replaces the entry by a rename, and an append or truncate first gives the file a private copy, so the file's other names are never modified. On a system with fs.protected_hardlinks = 0, do not export capsules from a directory other users can write to.

Sessions vs parent/child (two different groupings)

Two distinct, composable ways of relating capsules exist — do not conflate them:

Session (experimental, ADR-0122). A logical, multi-turn grouping: N independent runs (e.g. the turns of a conversation or the stages of a workflow) referenced in order by a content-addressed session.json manifest (nova session new/add/list/show). Members are ordinary capsules, each with its own writer; the session copies no capsule data. nova session export packs a session with its members into one verifiable ZIP for another machine (experimental). nova session replay (experimental, ADR-0123) replays the members in sequence order, whole or as a --from/--to slice.
Parent/child (prototype, ADR-0039). A physical, distributed-run hierarchy: one logical execution fanned out across nodes (e.g. a Slurm DDP job) recorded as a PARENT capsule plus N WORKER capsules under a single global_run_id.

A session member may itself be a distributed-run PARENT. A capsule that belongs to a session carries two additive optional back-reference fields (session_id, sequence); absence means a standalone run, byte-identical to before.

Related: Replay Modes (a replay writes a replay_result.yaml report; only intervention also writes a counterfactual capsule you can diff), Lineage Graph, and Evidence Bundle.

A capsule can carry optional, additive evidence facets — a facets map (ADR-0196) whose keys are per-domain evidence objects, each owned by its own ADR. A capsule without the facets key is valid and byte-identical to before, so facets never change the base schema. facets holds first-party, NovaFabric-recorded evidence; it is deliberately distinct from extensions, which carries third-party vendor data — an auditor must be able to tell NovaFabric-recorded evidence from a vendor annotation. The registry is closed on purpose (ADR-0196 D2): only the registered names below are accepted, so a typo cannot silently produce a capsule missing the evidence its author believed it carried.

Not the same as OpenLineage facets. The word "facet" is also used, unrelatedly, for the OpenLineage run/dataset facets that nova lineage emit-openlineage --with-facets attaches to emitted lineage events. Those live in OpenLineage payloads, not on the capsule. This section is about the capsule facets container.

Each facet is populated by a record-only evidence module: it records what another system did and never orchestrates, enforces, adjudicates, moves funds, controls a device, or sits in a hot path. All are experimental.

Most are Python APIs only — NovaFabric ships the schema, the validation and the binding rules, and your integration code produces the facet. Nothing in NovaFabric writes them for you, and they register no nova command.

The exception is novafabric.a2a, which additionally ships nova a2a-card and nova a2a-objects for the portable Agent Card and the Task/Message/Artifact mapping (ADR-0149). The a2a_messages facet in the table below remains API-only.

Module	Capsule facet	Records	ADR
`novafabric.a2a`	`a2a_messages`	Multi-agent A2A messages & handoffs — the wire between agents. Distinct from the `novafabric.adapters.a2a` capture-time SDK adapter, which is a separate subsystem.	ADR-0142
`novafabric.settlement`	`settlement`	Agentic-commerce settlement provenance — mandate reconciliation, finality, and non-repudiation binding (P1/P2), plus the agent-to-agent payment-provenance chain walked offline by `nova settlement chain` (each `parent_hop` must resolve to an earlier hop; no forks, no FX between hops), negotiated-agreement evidence (`agreement_ref`, `terms_digest`, `parties`) and invoice/receipt binding (`matches_settlement` against the finality record; `unbound: true` when the document does not resolve) — P3, experimental. Amounts are integer minor units, never floats. Never processes payments, holds/moves funds, or decides a dispute.	ADR-0163
`novafabric.embodied`	`embodied`	Embodied / cyber-physical agent evidence — declared sensor streams and actuation records (P1), plus the declared ODD with observed excursions (`verdict` always null) and the perception→actuation trajectory chain, walked offline by `nova embodied trajectory verify` (P2, experimental), plus sim-to-real lineage (`unbound: true` when the sim policy does not resolve), pseudonymous teleop handoffs and per-clock-domain timing evidence (`nova embodied sim2real show	verify`,` teleop list`,` timing show` — P3, experimental). Stores references, digests, and counts only (never frames, point clouds, audio, or control credentials); never in a control path.
`novafabric.science`	`science_provenance`	Scientific-reproducibility / research-integrity provenance as a verifiable DAG of research steps. Never runs experiments or adjudicates validity.	ADR-0164
`novafabric.memstore`	`memstore_mutation`	Persistent-knowledge / organisational-memory governance — an append-only ledger of who changed which shared-KB entry and when (store-external); P2 adds an `access` block (which agent read/wrote which entry, `contained:false` as out-of-scope evidence, never enforcement) and cross-run derivation (a read traced to the write that seeded it) — `nova memstore`, experimental. Distinct from the `nova memory` capture feature.	ADR-0171
`novafabric.retrieval`	`fetch_provenance` (+ source pinning)	Retrieval-source authority & knowledge provenance — what an external retriever or the agent fetched, and whether a pinned source still matches. Never fetches, crawls, or ranks.	ADR-0153
`novafabric.context`	(standalone artifacts)	Context provenance — an ordered manifest of what entered the model's context window (`ContextManifest`) and the span→chunk support map a producer claimed (`GroundingMap`). Records `sha256:` digests only; never scores groundedness. Ships as standalone artifacts, not a registered capsule facet (ADR-0196 D2).	ADR-0143 (NF-112/113)

Capture Hook Mechanism

nova capture <cmd> works without modifying application code. It injects a sitecustomize.py loader into the subprocess via PYTHONPATH. When Python starts the subprocess, it imports this loader automatically, which installs monkey-patches for the libraries below. Python imports only one sitecustomize, so the loader then runs the one it shadows — your own on PYTHONPATH, the virtualenv's or the distribution's — with site.py's contract (an error is printed, the workload still starts); releases up to v0.107.0 skipped it, so certificates, proxies or warning filters set up there were missing under capture (works today). It patches:

OpenAI — Completions.create; since v0.105.0 also AsyncCompletions.create, Responses.create and AsyncResponses.create, in openai.resources (capture/hooks/_openai.py). A stream=True call is folded into one record when the stream ends (ADR-0304) — and that record goes to the capture that opened the stream, whichever task, thread or later capture ends it; a stream that ends after its own capture has finished is dropped and counted (nova_event=capture.stream_record_dropped), never filed into another capsule (works today) — and a call that raised is recorded with the SDK error detail (io.novafabric.sdk_error, capture/hooks/_sdk_errors.py) that mocked replay needs to raise it again.
Anthropic — anthropic.resources.messages.Messages.create; since v0.105.0 also AsyncMessages.create, streamed or not (capture/hooks/_anthropic.py).
httpx — httpx.Client.send, recording requests classified by the URL registry (src/novafabric/capture/hooks/url_registry.yaml + ~/.novafabric/url_registry.yaml override). Default coverage: OpenAI, Anthropic, Cohere, Together, Mistral, Replicate, AWS Bedrock, Ollama (default port 11434). Non-default Ollama ports are detected automatically from OLLAMA_BASE_URL / OLLAMA_HOST at call time.
httpx2 — httpx2.Client.send / httpx2.AsyncClient.send, same classification and records as httpx (network events carry library: "httpx2"). openai 3.x builds its default client on httpx2, so without this hook an OpenAI call left no transport record and no network event. Installed only when httpx2 is importable — never a NovaFabric dependency. Under an SDK call each attempt is still a transport record (ADR-0305), so one call counts once. Works today.
requests — requests.Session.send, same URL-registry classification (v0.5; RFC-0001 Option C wire-level layer). Covers LangChain HTTP adapters, LlamaIndex REST clients, and any SDK that ships over requests.
aiohttp — aiohttp.ClientSession._request, async wire-level capture (v0.6 / C-3.1). Catches LangChain async paths, FastAPI agents, streaming-first SDKs.
urllib3 — urllib3.connectionpool.HTTPConnectionPool.urlopen, lowest-tier wire-level capture (v0.6 / C-3.2). Sits below requests and boto3; transitively covers AWS Bedrock and any SDK that uses urllib3 directly. Layered with the higher hooks via a context-local guard so a single requests call records exactly once.
MCP (Model Context Protocol) — mcp.client.session.ClientSession.call_tool, recording every tool invocation as a tool-calls.jsonl entry with transport=mcp and a synthesized JSON-RPC envelope (v0.5; ADR-0015 §Primary). A result the server marks isError: true — how MCP reports a failing tool — is recorded as status: error with error.type: McpToolError and the result kept; mocked replay returns that result, as the original call did (releases up to v0.107.0 recorded it as a success). The same holds for nova mcp-proxy. Works today — fail-open: the hooked call returns, or raises, exactly what the un-hooked call would have. If the record cannot be built or written (a full disk, a removed capsule directory, a response object whose attributes raise), the record is dropped — never reported as a tool error — and the workload continues. The loss is counted (MCPHook.dropped_records and the capture's in-process drop counter) and logged once as a structured warning (nova_event=capture.record_dropped, exception type only). Under nova capture the hook runs in the workload's own process; that process hands its drop counts to nova capture when it exits, and they land in the capsule's capture-health.json under sources.workload (a process that ends with os._exit or is killed hands off nothing).
Third-party plugins — any class registered under the novafabric.hooks entry-point group is auto-discovered alongside the built-ins (v0.5.x experimental; RFC-0001 §Detailed design — Option C). Plugin authors: see docs/integrations/writing-a-hook-plugin.md.

The layering is deliberate: per-SDK hooks capture rich, structured request and response objects; wire-level hooks form a safety net beneath them so a call that bypasses a known SDK (or uses a version whose surface changed) is still recorded once at the transport layer. Because the hooks reach down to urllib3, capture does not depend on your choice of HTTP client library.

A call made through the OpenAI or Anthropic SDK passes through both layers, so it leaves two kinds of record: the SDK hook's logical record (with the response) and one wire-level transport record per HTTP attempt beneath it, retries included. Both are kept as evidence; the transport records are marked extensions["io.novafabric.record_role"]: "transport" and link to the SDK record through io.novafabric.logical_call_id, and every count — model_call_count, nova cost, nova diff, replay counters, nova query, the dashboard — counts logical calls only (ADR-0305, works today). A capsule captured before this marker existed is read with the same rule: readers recognise the old wire-plus-SDK duplicate shape. See Model-call record roles.

A hook never changes what the call does (works today; ADR-0021). In particular:

Streamed responses are never read by capture. A wire hook records the call when the response object comes back — for requests.post(..., stream=True), httpx client.stream(...) / send(..., stream=True), an aiohttp response or a urllib3 preload_content=False response, that is when the headers arrive, before the body. The hook does not consume, buffer or wait for the body: the caller gets the response at the same moment as without capture, and iterates the same bytes — an endless server-sent-events stream included. The network event's response_size_bytes is the exact length of a body the client library had already read, else the declared Content-Length, else null (unknown — a chunked or endless stream). Before this fix the requests hook read response.content to size the event, which drained a streamed body inside Session.send, so send returned only when the stream ended — never, for an endless one.
Work before the call is fail-open too. Deciding whether a request is a model call (building its URL, the URL-registry lookup) cannot stop the request: if it fails, the request is sent unrecorded and the loss is counted (nova_event=capture.record_dropped).
A positional call reaches the SDK as written. The OpenAI and Anthropic create methods are keyword-only; a call with positional arguments is passed through unrecorded, so the SDK rejects it exactly as it would without capture (the hook used to drop the positional arguments and record the call).

Patches are removed after the run. If an SDK is not installed, its hook is silently skipped. Capture works even if none of the AI SDKs are present — the capsule is still written with environment, stdout/stderr, and timing.

For uninstrumented agents (Claude Desktop, Cursor, third-party SDKs that do not import the Python mcp package), nova mcp-proxy is a transparent stdio proxy that records the same tool-calls.jsonl schema by sitting between the client and an upstream MCP server. See docs/cli-reference.md and ADR-0015 §Secondary; experimental in v0.5.x.

For non-Python clients over HTTP, nova api-proxy is a transparent HTTP proxy that classifies traffic against the same vendored provider registry — so a client in any language can be captured without SDK hooks (v0.6).

Runners — where the captured workload executes

The orchestrator delegates subprocess execution to a runner (ADR-0025). Choose one with nova capture --runner <name>:

Runner	Since	What it does	Key required options
`local`	v0.5.x	Runs the workload as a local subprocess. Default; no remote infrastructure needed.	—
`docker`	v0.6	Runs the workload inside a Docker container. The image must already have NovaFabric installed. Capsule directory is mounted as a volume so artifacts land on the host.	`image`
`kubernetes`	v0.6.1	Runs the workload as a Kubernetes `Job` via `kubectl` shell-out. Artifacts are pulled back via `kubectl cp`.	`image`, `namespace`
`slurm`	v0.6.1	Runs the workload as a SLURM batch job: the generated script goes to `sbatch` on stdin (never `--wrap`), polled with `sacct`. Trusts a shared filesystem for artifacts (no rsync).	`partition`
`lsf`	—	Runs the workload as an IBM LSF batch job: the script goes to `bsub` on stdin, polled with `bjobs`. Shared filesystem, no rsync.	— (`queue` defaults to `normal`)
`pbs`	—	Runs the workload as a PBS batch job: `qsub <capsule>/nova-<run>.pbs`, polled with `qstat`. Shared filesystem, no rsync.	— (`queue` defaults to `default`)

Runner details:

docker — pass --runner-option image=<ref> (e.g. pip install novafabric in your Dockerfile). Other options: network, workdir, user, extra_volumes, extra_env. No privileged containers, no docker-socket mounts, no host namespaces by default — see ADR-0025 §Anti-patterns. The capsule directory is mounted read-write, so the container can write anything into it, including links. Host-side finalisation never follows one: see Links inside a capsule are never followed.
kubernetes — optional: service_account, node_selector, resources. Anti-patterns are enforced in the Job manifest itself: securityContext.privileged: false, hostNetwork: false, hostPID: false, hostIPC: false, backoffLimit: 0 (one shot, no retries). RBAC required: create/get/watch/delete on jobs+pods, get on pods/log+pods/exec. No cluster-admin.
slurm — optional: account, qos, time, nodes, gres, mem, constraint. The capsule directory must be on a filesystem shared between submit and compute nodes (typically /scratch, /lustre/..., or the user's home). The runner does not rsync artifacts; it trusts the shared FS.

Note. The single-node runner is the smallest case of a distributed run. The cluster-scale tiers shipped in v0.10+ with honest maturity labels: parent/child capsules (nova run …) are a prototype — implemented and tested, not yet validated at target scale — and the collector tier, object capsule store, and Postgres metadata DB are experimental. See ROADMAP.md for per-component labels; the federation and at-scale graph tiers remain future design.

`model-calls.jsonl`

Every LLM API call intercepted by the capture hooks is recorded as one JSONL line following the OpenTelemetry GenAI semantic conventions:

{
  "schema_version": "0.1.0",
  "semconv_version": "1.30.0",
  "model_call_id": "01HXAY7M6FN9TQGE0V0M7PAY1Q",
  "parent_span_id": "657bff2c61ddad1c",
  "started_at": "2026-05-09T12:34:56.123456Z",
  "finished_at": "2026-05-09T12:34:57.353456Z",
  "duration_ms": 1230,
  "status": "success",
  "gen_ai.system": "openai",
  "gen_ai.operation.name": "chat",
  "gen_ai.request.model": "gpt-4o",
  "gen_ai.response.model": "gpt-4o-2024-08-06",
  "gen_ai.response.id": "chatcmpl-abc123",
  "gen_ai.request.temperature": 0.7,
  "gen_ai.request.max_tokens": 1024,
  "gen_ai.request.top_p": 0.95,
  "gen_ai.request.seed": 42,
  "gen_ai.request.stop_sequences": ["END"],
  "gen_ai.request.messages": [...],
  "gen_ai.response.choices": [...],
  "gen_ai.response.finish_reasons": ["stop"],
  "gen_ai.usage.input_tokens": 42,
  "gen_ai.usage.output_tokens": 18,
  "endpoint": "https://api.openai.com/v1/chat/completions"
}

The full set of OTel "Required when applicable" fields is extracted by both the per-SDK hooks (_openai, _anthropic) and the wire-level hooks (_httpx, _requests, _aiohttp, _urllib3) when present in the request body — so nova replay --mode exact has the determinism inputs it needs (temperature, top_p, seed) regardless of which transport the captured call took. See design/spec/model-call-v0.md (private) for the full field reference.

This format is stable across providers and is the basis for two things:

nova diff call alignment — aligned model calls are compared field by field (see Structural Diff).
nova replay mock serving — recorded calls are replayed from the capsule cache in mocked mode (see Replay Modes).

Environment Lock (`env.lock`)

The environment lock records the full execution environment at capture time:

Python version and interpreter path
All installed packages, sorted by name (releases up to v0.107.0 kept an arbitrary 200)
Safe environment variables (secrets are excluded)
Host OS, CPU architecture, CPU count, total memory (read from /proc/meminfo; 0 where that file does not exist)
A GPU field (hardware.gpus), which is not populated today: no GPU inventory is collected
Detected package manager (uv.lock, poetry.lock, pdm.lock, requirements.txt) and the SHA-256 of the whole lock file (python.lock_file_hash; releases up to v0.107.0 hashed only its first 64 KiB, so two lock files differing past that hashed the same). Only a hash-pinned uv.lock makes the lock deterministic; the others are best-effort with a reason, as ADR-0007 specifies, so --mode exact refuses them

nova replay uses env.lock to warn about environment mismatches before re-executing the command. This is what makes the exact replay mode falsifiable: eligibility for byte-exact replay depends on a matching, deterministic environment plus a per-call seed, and env.lock is where that determination starts.

Secret Scanning and Redaction

Before a capsule is finalized, SecretScannerV0 scans every capsule file against the gitleaks-core-v0 rule pack — LLM provider keys (Anthropic, OpenAI, HuggingFace, Replicate, Langfuse, and others) and, since v0.105.0, AWS and GitHub credentials (18 rules in pack 0.7.0). Detected values are redacted in-place as [REDACTED:rule-id]. The scan is pattern-based: formats it does not detect (PEM keys, JWTs, passwords, connection-string credentials, …) are listed in the Run Capsule reference.

After scanning, redaction-proof.json is written. It records:

How many findings were detected
Which files were scanned
A cryptographic chain hash proving the scan ran

Capsules without redaction-proof.json are considered invalid by nova validate and cannot be exported into an Evidence Bundle. Verifiable redaction is therefore not optional — it is a precondition of every downstream audit artifact.

Replay Modes

A replay re-executes or inspects a capsule. When it re-executes, NovaFabric controls only the calls it intercepts (see mocked mode); the rest run live. There are five explicit, falsifiable modes; intervention is the experimental counterfactual mode. Every mode writes a replay_result.yaml report; only intervention also writes a new (counterfactual) capsule, which you can nova diff against the original run.

Mode	Spawns subprocess?	Network?	Best for
`forensic`	No	No	Audit / post-incident inspection
`mocked`	Yes	OpenAI/Anthropic model replies (sync or async, streamed or not), MCP `call_tool` results and (experimental) `record.tool` function results served from the capsule; other tools run live; fails closed on divergence	CI / regression
`semantic`	No	No	Consistency score over the capsule's recorded model responses — does not re-execute
`exact`	No	No	Eligibility check for a byte-exact re-run — does not re-execute
`intervention` (experimental, ADR-0086)	Yes, under mocked semantics	Model replies served from the capsule (with the one substitution); tools run live, so a substituted tool result is not delivered to the workload	Counterfactual root-cause: substitute one captured event per an `InterventionSpec`, re-execute downstream, and record whether the outcome flips

Honesty note. NovaFabric explicitly does not claim byte-exact replay of remote LLM calls. exact mode requires a deterministic environment and a per-call seed, which is realistic for local/on-prem models but not for a remote endpoint that can change under you. Neither exact nor semantic re-executes anything today: exact reports whether a byte-exact re-run is possible, and semantic scores the recorded responses — a live re-run judged on meaning is future design (no implementation, not yet roadmapped).

`forensic` mode

Read-only inspection. NovaFabric does not spawn a subprocess. It reads and returns the manifest, traces, and model calls directly from the capsule. No network access, no mutation.

Use forensic mode to inspect what happened without any risk of side effects.

`mocked` mode

The original command is re-spawned as a subprocess (works today for Python workloads; what v0.104.0 served versus v0.105.0 — MCP results, async, streamed and Responses API calls, the fail-closed contract — is listed under Release scope). A capsule that records no command to re-run — written by a framework adapter or the @agent decorator (novafabric.sdk.agent; capture_mode: sdk-decorator, a @framework:name label) or imported from OpenTelemetry spans — is refused before anything is spawned (CapsuleNotReplayable, exit 1; since v0.105.0). Inside the re-spawned process:

MockModelDispatcher serves the recorded responses, in order, for OpenAI chat.completions.create, OpenAI responses.create and Anthropic messages.create calls — sync or async, and with stream=True as the chunk or event stream the SDK would have produced (ADR-0304) — including the assistant's recorded tool-call requests. parse, legacy completions, Anthropic messages.stream() and with_raw_response calls are refused, not sent to the network. Since v0.105.0 (experimental), a recorded call that failed (a rate limit, a 4xx, a 5xx after the SDK's retries, a timeout) is replayed by raising the same SDK exception class at its position, from an allow-list (replay/_model_errors.py); the wire hook's per-attempt transport records are never served (ADR-0305).
MockToolDispatcher serves recorded MCP results through mcp.ClientSession.call_tool, one recorded result per call (matched by tool name and arguments, repeated identical calls in recorded order). A call with no recorded result is refused — the live tool does not run.
Experimental (ADR-0306 slice 1, since v0.105.0): it also serves functions the workload declared with novafabric.capture.record.tool, before the function body runs, matched by name and signature-bound arguments. Records are servable only when captured at the forensic/air_gapped level with JSON-native values; anything else is refused with the reason. Since v0.106.0 (slice 4): a declared function that itself called a model or tool is served too, and the records it wrote are consumed as covered; MCP arguments a capsule redacted at seal now match the live call (redact-then-hash).
Experimental (ADR-0306 slice 3, since v0.106.0): it also serves Google ADK tool calls made through the NovaFabric tool plugin (novafabric.adapters.google_adk.make_tool_plugin(), first in the Runner's plugins), through ADK's documented before_tool_callback short-circuit — for workloads captured with nova capture python … only.
Experimental (ADR-0306 slice 3, since v0.106.0): it also serves OpenAI Agents SDK function tools passed through novafabric.adapters.openai_agents.wrap_tools(), by replacing the tool's on_invoke_tool — an undocumented SDK seam, checked against openai-agents 0.17.x — again for nova capture python … workloads only.
Every other tool runs live: HTTP requests, shell commands, file writes, other framework-native tools, undeclared functions and other providers' SDKs are not intercepted. The result reports them as tool_calls_not_interceptable, and the outbound connections the replayed process opened as network_connections_live (observed, not blocked). Since v0.106.0 (slice 4, experimental): the result an undeclared function sends back to the model is compared with the recorded one and a difference is reported in replay_contract.tool_result_echo — report-only, never a failure.

The replay fails closed: an extra model call, a call on an unsupported surface, an unmatched MCP, record.tool, ADK or Agents SDK tool call, or a recorded response that is never requested marks the replay failure with a divergence_reason, even if the workload caught the exception. --permissive (Python: ReplayFlags(permissive=True)) keeps the older behaviour — an empty reply on an exhausted queue, unsupported model surfaces run live — and still reports every divergence; an unmatched MCP or record.tool call runs live only if a safety-ladder flag permits its class (an MCP call always counts as unknown: --allow-unknown-mutation). (Changed in v0.105.0, ADR-0306: --permissive alone used to run every unmatched MCP call live.) The result counts what was actually served: model_calls_mocked of model_calls_available, tool_calls_mocked/tool_calls_live/tool_calls_unmatched, and queues_fully_consumed. See the support matrix.

Safety ladder (mocked replay)

The safety ladder classifies the capsule's recorded tool calls by side-effect level. It drives the --dry-run report, and every write-permitting rung (--allow-mutating, --allow-external-side-effects, --allow-unknown-mutation) triggers a policy-engine gate (an audited allow/deny) before the replay starts. In mocked mode the run-time rule is fixed by ADR-0300: an intercepted call (MCP call_tool, a record.tool function, an ADK tool through the tool plugin, or an Agents SDK tool through wrap_tools) is served from the capsule or refused, and every other tool runs live (the --dry-run report marks those [LIVE]). The rungs gate the intercepted surfaces in two places only (ADR-0306, experimental): under --permissive an unmatched call runs live only if a rung permits its class, and a replay.yaml allow: true override re-executes a tool only if a rung permits its class. The class is the one a record.tool decorator declares, or unknown for every MCP call — never a class the capsule recorded. Ladder-based enforcement for tools replay does not intercept is future design. The rungs:

(none)               — deny all tool calls
  ↓  --allow-readonly
read-only            — permit read-only tool calls
  ↓  --allow-mutating
idempotent-write     — permit idempotent writes
non-idempotent-write
  ↓  --allow-external-side-effects
external-side-effect — permit calls with external effects
  ↓  --allow-unknown-mutation
unknown              — permit unclassified tool calls

Per-tool overrides can be specified in replay.yaml, one entry per tool name (schemas/replay-policy.schema.json, ToolOverride):

tool_overrides:
  - tool_name: safe_lookup
    allow: true      # re-execute it -- only if the operator also passes the ladder flag
  - tool_name: send_email
    allow: false     # never run it live

In mocked mode the overrides are enforced inside the replayed process (experimental, ADR-0306, since v0.105.0) on the two surfaces replay intercepts — MCP ClientSession.call_tool and functions declared with record.tool — and --dry-run prints exactly the same decisions. Because replay.yaml travels inside the capsule, a restriction from it is trusted and a permission is not:

allow: false — a recorded result is still served (serving is not re-execution); a call with no recorded result is refused, even under --permissive. If the tool was recorded on a transport replay cannot intercept (HTTP, shell, an undeclared function), the override cannot be enforced, so a strict replay refuses to start (ToolOverrideUnenforceable, exit 3); --permissive starts and reports override_unenforceable.
allow: true — the tool is re-executed live only if the operator also passes the ladder flag for its class (an MCP tool always counts as unknown, so --allow-unknown-mutation). Without it the recorded result is served and the result reports override_not_honoured.

The result lists every override under replay_contract.tool_overrides. Outside mocked mode no tool dispatcher exists and the overrides only colour the --dry-run report. The older action: replay | refuse form is still read as allow: true | false.

`semantic` mode

Does not re-execute and calls no model. It reads the capsule's recorded model responses and returns similarity_score — the mean pairwise text similarity (difflib, 0.0–1.0) between those responses, i.e. how consistent the recorded run was with itself (1.0 when there are fewer than two responses). The intended mode — re-run against live models and judge the new outputs against the recording on meaning — is future design, not implemented.

`exact` mode

Does not re-execute. Reports exact_eligible and the reasons it is not: the env.lock must be mode: deterministic, every model call must carry a seed, and no stored tool result may have drifted from its declared schema (ADR-0128). It answers whether a byte-exact re-run of a local / on-prem model is possible; performing and hash-verifying that re-run is future design.

`intervention` mode (experimental)

Answers a counterfactual question: if this one recorded event had gone differently, would the run's outcome have changed? An InterventionSpec names one captured model or tool call and a substitute outcome for it; the engine re-executes everything downstream of that point under mocked semantics (zero live tokens) and writes a diffable capsule hard-marked replay_mode: intervention, never mistakable for a real run. Only a substituted model response reaches the re-executed workload: intervention installs no tool dispatcher, so tools run live and a substituted tool result changes the output capsule and the checks only. Since v0.105.0 the result says which with intervention.substitution_delivered_to_workload. A capsule with no command to re-run is not re-executed at all (downstream_reexecuted: false). This is the building block behind the no-LLM causal-graph diagnostic suite below — see Diagnose: causal-graph attribution and counterfactual root-cause search.

Diagnose: causal-graph attribution and counterfactual root-cause search (experimental)

nova diagnose <run-id> (ADR-0084) attributes a failed run to its most likely responsible step: it walks the captured trace plus the lineage graph and produces a ranked attribution — which agent/step is most likely responsible, plus an AgentErrorTaxonomy label (MEMORY / REFLECTION / PLANNING / ACTION / SYSTEM / UNKNOWN). Scores are relative ranking weights, not probabilities, and every candidate is honestly marked verification: unverified unless tested.

Two further, experimental layers (ADR-0101) turn a ranking into evidence:

No-LLM causal-graph back-trace (§NF-019/022). causal_root_candidates back-traces the span graph — no LLM involved, purely structural — to find root failure nodes: a failing node with no failing ancestor. This is what candidates the search below.
Replay-proven root cause (--intervene / --search-root-cause, §NF-017/018). --intervene auto-synthesizes an InterventionSpec for the top hypothesis and replays it under intervention mode (see Replay Modes) to check whether the outcome actually flips. --search-root-cause widens this into a bounded sweep over the NF-019 candidates, earliest/shallowest first, until one confirms a flip — the decisive root cause, replay-proven rather than merely ranked.

None of this uses an LLM judge: attribution is structural (trace + lineage walk) and verification is a real, zero-token replay — a hallucination-risk or root-cause finding you can point at evidence for, not a model's opinion.

Accountability Spine (experimental)

Three complementary, append-only evidence surfaces (ADR-0093/0094/0095), collectively referred to as the Accountability Spine, sit alongside NovaSeal and the Evidence Bundle:

Surface	CLI	Records	ADR
Energy-Anchored Action Receipts	`nova energy probe/attest/verify/report`	Per-action energy receipts + a conservation check, so an energy claim is falsifiable rather than asserted	ADR-0093
Adversary-anchored accountability ledger	`nova ledger anchor/verify/status`	Sidecar hash chains + signed checkpoints for every capsule `.jsonl` stream, so tampering with a stream after the fact is detectable	ADR-0094
Structured safety case	`nova safety-case build/verify/export`	A compiled, evidence-grounded Claims-Arguments-Evidence (CAE) safety case over a capsule, exportable as JSON, Markdown, an EU AI Act Annex IV section, or a NIST AI RMF report	ADR-0095

The dashboard surfaces all three read-only under the Spine tab. Like the evidence facets above, these are record-only: they never enforce, adjudicate, or gate anything themselves — they make an accountability claim checkable.

Structural Diff

nova diff compares two capsules field by field:

Model call alignment — a parent_span_id unique on both sides pairs exactly; the remaining calls pair by sequence position, anchored on identical requests so an inserted or deleted call does not shift the rest. Aligned pairs are compared for the request (provider, model and messages) and the response choices.
Tool call alignment — calls are matched by tool_name + argument hash, each call used once, then by position among calls with the same tool name. A pair is changed when its arguments, result, status or error.type differ, including the io.novafabric.arguments_digest / result_digest extensions that are all a record holds at the default capture level (a digest on one side only counts too). Releases up to v0.107.0 compared only arguments and result, so a tool that started failing diffed as unchanged.
Environment diff — Python version, OS, architecture, interpreter path.
Output diff — every regular file under outputs/, recursively, by capsule-relative path and content hash. Symlinks are skipped, never followed.

DiffReport has three counters: changed, added, removed, and one property, has_changes, that is true when any of them is non-zero. The --assert-no-regressions flag exits 1 on has_changes, and the text and GitHub-annotation output read the same property, making it suitable as a CI gate. Exit 1 means only that; a comparison that could not be made (for example, a capsule path that does not exist, or a capsule with unreadable record lines, which are counted and reported rather than skipped silently) exits 2:

# Fail the pipeline if today's run diverges from a known-good baseline
nova diff baseline-capsule/ candidate-capsule/ --assert-no-regressions

This is the mechanism behind the "worked yesterday, fails today" flaky-agent problem: wire nova diff --assert-no-regressions into CI and a structural regression stops the merge.

Offline analytics over capsules (experimental)

Because every metric a run produced — cost, tokens, latency, scores — is already recorded inside its capsule, analytics never needs a server. The experimental v0.59 surfaces read those recorded facts, offline and read-only: nova query is a bounded filter → group-by → aggregate DSL over the local capsule directory, nova view persists a named query as a small versionable file, and nova trend buckets one metric over time or by asset into a JSON/HTML report. Nothing is recomputed or fetched; the capsule directory is never written. See the CLI reference.

Lineage Graph

The lineage graph is a directed graph stored in SQLite — a rebuildable cache derived from each capsule's lineage.jsonl. It has two table types:

lineage_nodes — runs, assets, and artifacts. Each node has a deterministic ID: the first 26 hex characters of SHA-256(kind:ref) (lineage/_types.py::node_id_for).
lineage_edges — typed, directional edges between nodes.

Because the graph is derived from the capsules, it can always be rebuilt from them — the capsules remain the source of truth. The SQLite backend is the local-mode default and is crash-safe (WAL); it is well suited below roughly one million edges. Four at-scale backends exist for larger graphs, all experimental and testcontainers-verified, selected via nova lineage-store migrate / profile: Kuzu (embedded, benchmark-cleared at 10M edges, p99 blast_radius 45.5ms), Postgres (recursive-CTE, no extension needed), Apache AGE (openCypher on Postgres), and JanusGraph (Gremlin, needs the GraphSON serializer). There are currently zero NotImplementedError stubs among them. Billion-edge cross-cluster federation remains FUTURE DESIGN, not implemented.

Edge types

The capsule's lineage.jsonl vocabulary (src/novafabric/schemas/lineage-edge.schema.json), labelled by who writes each type today:

Edge type	Meaning	Written by
`consumed`	The run read/used this asset	`nova capture`, from `assets.jsonl`
`produced_by`	This artifact was produced by the run	`nova capture`, from the capsule's outputs (or `produced.jsonl`)
`replayed_from`	This run is a replay of the referenced run	`nova capture`, from `replay_of_run_id` in `capsule.yaml`
`depends_on`	This asset depends on another asset	the registry, from an asset's declared dependencies
`migration_source`	This v1.0.0 capsule was upgraded from a v0.1.x capsule (same run)	`nova migrate` (ADR-0034 §6)
`derived_from`, `evaluated_by`, `attested_by`	Reserved by the schema	no NovaFabric writer emits them today

Two further vocabularies share the lineage store (lineage/_types.py::LINEAGE_EDGE_TYPES): the typed parent/child and session edges (contains, spawned, delegated_to, member_of_session; ADR-0044, ADR-0122) and the memory-provenance pair wrote_memory / read_memory (ADR-0143). The memory edges are derived on demand from a capsule's memory-operation events (lineage/memory.py), keyed by memory key only; capture does not write them to lineage.jsonl.

Confidence levels

Confidence	Meaning
`observed`	Directly recorded at runtime (e.g., an explicit asset reference in `assets.jsonl`)
`inferred`	Derived from capsule structure (e.g., output file presence)
`declared`	Stated by a record the run or capsule carries (e.g., `produced.jsonl`, `replay_of_run_id`)

Graph queries

Query	Direction	Use case
`provenance`	Backward (ancestors)	"What did this run depend on?"
`blast-radius`	Forward (descendants)	"What runs consume this asset?"
`replay-chain`	Backward via `replayed_from`	"What is the original run for this replay?"
`time-travel`	Backward, filtered by timestamp	"What was the lineage state as of time T?"

Graph analytics (experimental)

Traversal answers "what connects to this node?". A read-only analytics layer (ADRs 0212–0215) answers "what does the whole graph mean?": centrality (nova lineage metrics — hubs and single points of failure), node-level root-cause ranking (nova lineage root-cause), interop export to GraphML/GEXF/Cypher (nova lineage export-graph), and a synthesized intelligence report (nova insights). All are descriptive rankings for attention, not calibrated importance, and degrade honestly when a data source (such as cost) is absent.

OpenLineage Integration

NovaFabric emits OpenLineage 2.0.2 events from capsules. Events are typed as START and COMPLETE (or FAIL).

Events can be emitted:

Automatically at capture time if OPENLINEAGE_URL is set
On demand via nova lineage emit-openlineage

This enables integration with data-catalog tools such as Marquez, Atlan, and OpenMetadata — so NovaFabric's run-level provenance can join the broader data-pipeline lineage you already track.

Custom run facets (experimental). emit-openlineage --with-facets attaches NovaFabric-specific facets to the COMPLETE event — the capsule id/hash, the eval verdict, the promotion-policy decision, and reproducibility run params — and --otel-correlation adds trace_id/span_id so a lineage node links to its OTel GenAI spans. Facets are additive and schema-validated before emission, so a catalog that ignores them still receives valid core OpenLineage events. Run facets, dataset provenance cards, and benchmark-contamination checks are shown hands-on in the feature tour §17.

Note: these OpenLineage facets are unrelated to the capsule facets container (ADR-0196). Same word, two different places: OpenLineage facets live in emitted lineage events; evidence facets live on the capsule.

AI Asset

An AI asset is any versioned artifact in an ML or LLM system. NovaFabric tracks seven types:

Type	What it represents
`model`	A trained ML/LLM model — framework, artifact path
`agent`	An AI agent — model ref, tools, prompts, policies, eval suites
`prompt`	A prompt template used by one or more agents
`tool`	A callable function exposed to an agent
`dataset`	A dataset used for training, fine-tuning, or evaluation
`evaluation`	An evaluation suite definition (test cases, scoring logic)
`deployment`	A running endpoint — base URL, environment

Assets are described by YAML spec files and stored in the local Asset Registry. Every registered asset is addressed as name@version, pinned to a git commit SHA, and carries a lifecycle status that tracks where it is in your development-to-production pipeline.

Prompts as versioned assets (experimental, ADR-0112). Beyond the YAML spec route, nova prompt register turns a prompt into an immutable, content-addressed registry version: every edit registers a new version (never a mutation), and a run references it as prompt:<id>@<version>+sha256:<hex> — so a capsule can be tied to exactly the prompt bytes that ran. Mutable deployment labels (nova label, ADR-0113) point a name like production at one immutable version, and prompt composition (nova prompt compose, ADR-0115) snapshots pinned includes so an assembled prompt rebuilds byte-identically. NovaFabric never renders or serves prompts — it versions and proves them.

Asset Lifecycle

The state machine

development ──► validated ──► pending_approval ──► staging ──► production ──► archived
      │                                                  ▲
      └──────────────────────────────────────────────────┘
           (direct skip: development → staging or production)

All six statuses in plain language:

Status	Meaning
`development`	Work in progress. Default when registered. No constraints.
`validated`	Passed automated checks (`nova validate`). Spec is well-formed, secrets are clean.
`pending_approval`	Waiting for a human sign-off (`nova approve`). Useful in regulated environments.
`staging`	Cleared for pre-production use. Agents require a passing eval to reach here: the agent's most recent eval must have passed (an earlier pass does not clear a later failure).
`production`	Live. Agents require a passing eval to reach here (same rule).
`archived`	Retired. No further promotion possible. Capsules and lineage are still readable.

Direct skips (e.g. development → production) are allowed. Once archived, an asset cannot be re-promoted. The validated and pending_approval steps are optional — small teams often go straight from development to staging.

What actually changes when you promote an asset

Promoting an asset updates one database record. That is all NovaFabric does.

nova promote direct my-agent@v1.2.0 --to staging

Internally this:

Checks the policy engine — allow or deny, written to the audit log. A target of production is a protected stage (ADR-0312): a single actor needs an explicit promote_direct policy allow, and none exists by default, so production is normally reached through nova promote propose + nova promote approve.
For agent type going to staging/production: verifies a passing eval result exists. Blocks if not (unless --force).
Runs UPDATE assets SET status = 'staging', promoted_at = ..., promoted_by = 'cli-user' (the CLI records the fixed actor cli-user; a named human sign-off is the approval record written by nova approve <name@version> --approver alice).
Returns the updated record.

Nothing else happens. Your agent does not restart. Your prompt does not reload. No traffic is rerouted. No container is restarted. The capture hooks and runners are unchanged.

The status is governance metadata — a reliable signal your tooling can read and act on. NovaFabric gives you the signal; your platform decides what to do with it. (Policy checks and maker-checker approval are backed by OPA/Rego and signed decision logs — see Eval-Gated Promotion.)

How external tooling consumes the status

The pattern is: your CI/CD pipeline (or any script) reads the status from NovaFabric and makes a deployment decision based on it.

# In a GitHub Actions deploy step or Argo CD sync hook:
# (`nova inspect` prints a human-readable panel, not JSON — read the record
# through the Python API instead.)
STATUS=$(python -c 'from novafabric.registry.service import get_asset; print(get_asset("my-agent", "v1.2.0")["status"])')

if [ "$STATUS" = "production" ]; then
  kubectl set image deployment/my-agent container=my-agent:v1.2.0
else
  echo "Asset not in production — deploy blocked (status=$STATUS)"
  exit 1
fi

NovaFabric doesn't own your deployment. It owns the evidence that the asset is ready for deployment.

Use cases

Use case 1 — CI/CD gate for an LLM-powered feature

You have a customer-facing summarisation agent. Your CI pipeline runs evals on every PR. Only if evals pass and a team lead approves does the agent reach production, which triggers your Helm chart to update the image tag.

# .github/workflows/deploy-agent.yml (simplified)
- run: nova eval agent summariser@${{ github.sha }}
- run: nova promote propose summariser@${{ github.sha }} --to production --identity ci-bot
# ... the team lead countersigns from their own machine (a separate, trust-listed key):
#     nova promote approve summariser@<sha> --identity team-lead
# ... a later job, once the registry reports `production`:
- run: helm upgrade summariser ./chart --set image.tag=${{ github.sha }}

production is a protected stage, so a single-actor nova promote direct --to production in CI is refused by default (ADR-0312).

Use case 2 — Safe prompt rollout without a full redeploy

You update a system prompt template. You don't want to redeploy the entire agent service — just swap the prompt version your agent reads at startup. The registry tells your agent loader which version is production.

# agent startup code
from novafabric.registry.service import get_asset

asset = get_asset("system-prompt-v2", version="latest")
if asset["status"] != "production":
    raise RuntimeError(f"Prompt not promoted to production (status={asset['status']})")

prompt_text = json.loads(asset["spec_json"])["spec"]["template"]

If the prompt has a bug and you archive it, the next agent restart fails fast rather than silently loading a bad prompt.

Use case 3 — Audit trail for a regulated environment

Your company needs to demonstrate that no AI model reached production without a human sign-off. Every promotion through pending_approval creates an approval record (approver, timestamp, note) and an audit log entry. At audit time:

python -c 'import json; from novafabric.registry.service import get_asset
a = get_asset("risk-scorer", "v3.1.0")
print(json.dumps({k: a[k] for k in ("status", "promoted_by", "promoted_at", "forced_promotion")}))'
# → { "status": "production", "promoted_by": "cli-user", "promoted_at": "2026-03-15T10:22:01Z", "forced_promotion": false }

No manual tracking spreadsheet. The registry is the record.

Use case 4 — Blast-radius control when an agent misbehaves

An agent in production starts producing bad outputs after an upstream model update. You archive it:

nova promote direct my-agent@v2.0.0 --to archived

Your deployment hook detects the status change, rolls back to the previous production version (my-agent@v1.9.0), and pages the team. The capsules from the bad version are still fully intact for forensic replay — nothing is deleted.

Use case 5 — Multi-team handoff on an HPC cluster

A research team develops and validates an agent on their workstation. When they're ready for the ML engineering team to integrate it into the batch pipeline, they promote it to staging. The ML team's Slurm job template only launches agents whose status is staging or production:

# check_asset_status.sh — called from sbatch prolog
STATUS=$(NOVAFABRIC_DB_PATH=/shared/nova/registry.db python -c \
  'import sys; from novafabric.registry.service import get_asset; print(get_asset(sys.argv[1], sys.argv[2])["status"])' \
  "$AGENT_NAME" "$AGENT_VERSION")
if [[ "$STATUS" != "staging" && "$STATUS" != "production" ]]; then
  echo "PROLOG FAIL: $AGENT_NAME@$AGENT_VERSION is not ready (status=$STATUS)" >&2
  exit 1
fi

The registry on the shared filesystem (/shared/nova/registry.db) is the handoff protocol between the two teams — no Slack messages, no wiki pages.

Asset Registry

The registry is a local SQLite database at ~/.novafabric/registry.db (override: NOVAFABRIC_DB_PATH). It stores:

Table	Contents
`assets`	Every registered asset — name, version, type, status, full spec JSON, git SHA, promotion history
`eval_results`	Pass/fail eval results per asset, per suite
`approvals`	Human sign-off records: approver, timestamp, note

The registry is entirely local. There is no server, no credentials, and no network dependency for local mode.

In server mode (nova server start, experimental — not the single-user nova serve dashboard), the same data lives in the server's database (SQLite by default, Postgres via backend: postgres) and is accessible via the REST API under /v0/assets, guarded by bearer-token auth (OIDC, offline tokens, API keys or the local token) and an RBAC model (reader < writer < admin, plus an orthogonal auditor). Server mode is strictly additive: local mode never requires it.

Eval-Gated Promotion

nova eval agent <agent@version> discovers evaluation suites declared in the agent spec (spec.evals) and resolves them via Python entry points in the novafabric.evals group. Results are stored in the eval_results table.

Promotion to staging or production queries eval_results for a passing result. If none exists, promotion is blocked unless --force is used. A forced promotion is recorded as such (forced_promotion = true) and appears in nova inspect output and the audit log — the escape hatch is visible, not silent.

Standard eval suites ship OCI-pinned for reproducibility: GAIA, SWE-bench, AgentBench, MMLU, TruthfulQA, and a Smoke suite (v0.9; the novafabric.evals entry points in pyproject.toml). Promotion can be Rego-gated to block on a regression against a prior result.

Evidence Bundle

An Evidence Bundle is the compliance primitive: a signed, self-contained ZIP built by nova export-evidence that embeds

a copy of the Run Capsule,
its lineage edges,
the JSON schemas,
Ed25519-signed in-toto DSSE attestations, and
a manifest.json with a SHA-256 for every file, plus the verification recipe

(full list).

Its defining property is offline verifiability: a recipient can verify it with only sha256sum plus an ed25519 verifier — no NovaFabric runtime required. That is what makes the capsule a durable, portable artifact you own rather than a row in someone else's database. A capsule missing its redaction-proof.json cannot be exported (see Secret Scanning and Redaction).

Shipped trust-layer surfaces (v0.4+): ed25519-signed Evidence Bundles, in-toto DSSE attestations, secret scanning with verifiable redaction proofs, OPA/Rego policy gates with maker-checker promotion, and WORM storage adapters (S3 Object Lock / Azure immutable blob / GCS Bucket Lock, with legal holds).

Compliance-export cohort (experimental, ADR-0107 + related). Beyond the general-purpose Evidence Bundle, nova export-compliance and a long tail of dedicated nova export-* commands render capsule/lineage evidence into regulator-shaped artifacts: EU AI Act (Annex IV, Art.12 record-keeping, Art.50 marking + C2PA/SynthID assertion, Art.53 GPAI hash-chained form, Art.73 incidents with a deadline clock chained to Art.72 post-market monitoring), ISO 42001/42005, NIST AI RMF, the NIST GenAI/CSA profile, GDPR (Art.30 RoPA, Art.17 erasure), CycloneDX AI-SBOM, and more. Every exporter is a pure projection over already-captured evidence — it renders facts, never adjudicates compliance — and each is marked with an evidence_source provenance tag (operator_asserted / capsule_verified / unverifiable, specified in the maintainers' private design/ tree) so a reader can tell what was actually observed from what an operator merely declared.

Experimental — NovaSeal signing core. The in-process NovaSeal core shipped in v0.10+ (experimental): DSSE signing (ECDSA P-256), opt-in RFC 3161 trusted timestamps (no default TSA and no network call unless tsa_url is set — ADR-0292), and an append-only SQLite Merkle log, verified offline by nova verify (signature_ok / timestamp_ok / log_integrity_ok, plus timestamp_trust), plus the maker-checker nova seal propose/approve/verify chain (ADR-0059). Still PLANNED / FUTURE DESIGN per ADR-0041: the dedicated network signing service, qualified timestamps, Sigstore-keyless signing as the default path, WORM-backed retention at the seal layer, and PQC/ML-DSA. The most stable portable proof today remains the ed25519 / in-toto DSSE Evidence Bundle described above.

SDK Decorator

The @agent decorator (from novafabric.sdk.agent import agent) is an in-process alternative to nova capture. It wraps an agent function with the same capture hooks used by the CLI, recording all LLM calls into a capsule. Without capsule_dir, it emits OTel spans only — this is the v0.1 observability mode.

ULID

Run IDs and replay IDs are Universally Unique Lexicographically Sortable Identifiers. They embed a millisecond timestamp as the high bits, so capsule directories naturally sort chronologically without a separate index.

Summary and next steps

You now have the vocabulary NovaFabric is built on:

Run Capsule — the ULID-named directory that is the source of truth for one execution, written on success and failure.
Capture — zero-code-change instrumentation via a sitecustomize.py loader, per-SDK hooks, and a wire-level safety net down to urllib3.
Replay — five explicit modes (forensic, mocked, semantic, exact, intervention); only mocked and intervention re-execute, and intervention remains experimental, used for counterfactual replay.
Diff — field-by-field structural comparison, wireable as a CI regression gate with --assert-no-regressions.
Diagnose — structural (no-LLM) causal-graph attribution over a failed run, with an experimental counterfactual root-cause search that replay-proves rather than merely ranks a hypothesis.
Lineage — a rebuildable provenance graph answering provenance, blast-radius, replay-chain, and time-travel, with OpenLineage emission; SQLite is the local default, with four experimental at-scale backends (Kuzu, Postgres, AGE, JanusGraph) for larger graphs.
Registry + lifecycle — versioned assets with a six-state lifecycle where promotion is governance metadata only.
Evidence Bundle — offline-verifiable signed export; the NovaSeal signing core (DSSE + RFC 3161 timestamps + Merkle log, nova verify) is experimental, as is the Accountability Spine (energy receipts, ledger, safety case) and the EU AI Act / ISO / NIST compliance-export cohort, while the dedicated signing service and seal-layer WORM retention remain PLANNED.

Where to go next:

CLI reference — every command and flag: docs/cli-reference.md
Writing a hook plugin — docs/integrations/writing-a-hook-plugin.md
Model-call field reference — design/spec/model-call-v0.md (private)
Runner design and anti-patterns — ADR-0025
Trust-layer and sealing design intent — ADR-0041