The pipeline: capture → seal → replay → diff → audit
Architecture, as built › Pipeline
The five verbs are separate nova commands that all work on one capsule
directory. Seal is the exception: it runs automatically at the end of
nova capture when NovaSeal is configured. Any later step can run on another
machine, years later, from a copy of the directory.
Diagram source (Mermaid)
sequenceDiagram
autonumber
actor U as User / CI
participant C as nova capture<br/>(CaptureOrchestrator)
participant W as Workload subprocess<br/>(+ sitecustomize hooks)
participant D as Capsule directory
participant S as NovaSeal
U->>C: nova capture -- python agent.py
C->>D: open <ULID>/ (inputs/, outputs/, empty *.jsonl)
C->>W: runner starts the command
W-->>D: model / tool / trace records (hooks)
W-->>C: exit code, stdout, stderr
C->>D: env.lock, scan + redact, replay.yaml, capsule.yaml, lineage.jsonl
C->>D: residual rescan → redaction-proof.json
C->>D: evidence_digests → capsule.yaml (rewritten, gate-checked)
opt NovaSeal configured
C->>S: seal(manifest)
S-->>D: .seal/manifest.dsse, .tsr, log-entry.json
end
U->>D: nova verify · nova replay · nova diff · nova export-evidence · nova audit1 · Capture: nova capture (works today)
cli/capture.py:capture_cmd builds a capture/orchestrator.py:CaptureOrchestrator
and calls run(). If the warm-capture daemon is reachable and no flags need the
in-process path, the command goes to the daemon instead (see
warm-capture-daemon.md). run() works through these
stages in order:
- Pre-flight gates, before anything is written: required asset status
(
--require-asset-status), deployment environment, A/B variant, session membership. - Open the capsule. It gets a new ULID (
capture/_ulid.py), andcapture/capsule.py:CapsuleWriter.open()creates the directory withinputs/,outputs/and empty call streams. - Start recording.
capture/event_recorder.py:EventRecorderstarts, and the environment snapshot (capture/env.py) is taken on a background thread so it overlaps the workload. - Run the workload. The selected runner (
runners/:localby default; alsodocker,kubernetes,slurm,lsf,pbs) writes thesitecustomize.pyhook loader onto the child'sPYTHONPATH. In the child,capture/hooks/__init__.py:install_allpatches the supported SDKs and HTTP clients. - Redact. After
env.lockis written,capture/secrets.py:SecretScannerV0.scan_and_redactalways runs over every file that exists at that point: the call and event streams,env.lock,assets.jsonl, and every file underinputs/andoutputs/(recursively; symlinks are not followed). Matches in text are redacted in place. A binary that carries a key-shaped match, or a file whose name carries one, is dropped. Any configured maskers (masking/, experimental) then run over the same files. - Write the manifest and lineage. The orchestrator writes
replay.yaml. The manifest is redacted as a data structure (built-in rules, then maskers, then the built-in rules again) beforecapsule.yamlis written, so a key on the command line is not written to disk.lineage/_writer.py:LineageWriterthen writeslineage.jsonl, which is scanned and masked straight away, and the edges are indexed. - Residual pass and proof. If the event recorder dropped any events,
capture-health.jsonis written first, so it is rescanned and digested like every other file.SecretScannerV0.residual_scanthen rescans every file in the finished capsule exceptcapsule.yaml, the proof and.seal/. That includes files written after step 5 (replay.yaml, the C2PA marker) and files a masker rewrote. A match is redacted and recorded like any other finding. Each file's after-hash is updated to the bytes on disk, and the pass is recorded inresidual_check.redaction-proof.jsonis then written, once. - Digest and gate.
evidence_digestsrecords a SHA-256 and size for every evidence file, including the proof. Beforecapsule.yamlis rewritten, the final manifest is checked against the rules (SecretScannerV0.assert_manifest_clean). If anything matches, the manifest is redacted, the capsule is not sealed, and capture prints an error. The workload's exit code is unchanged. - Seal, if configured. See step 2 below.
Steps 5–8 work today. Detection is rule-based (gitleaks-core-v0, pack 0.7.0),
so a secret in a format no rule matches is not found. For example, a bare AWS
secret access key is matched only when its key name is next to it, and PEM
private keys, JWTs and passwords are not matched at all. The full list:
what the scanner does not detect.
A failed workload still produces a complete capsule with status: failure. A
workload the runner could not start at all (for example a mistyped command, which
runners/_local.py reports as command not found: <name>) is recorded with
error.type: WorkloadNotStarted rather than NonZeroExit (since v0.105.0). If a NovaFabric
component fails, the failure is recorded and the workload continues.
Steps 6–9 are one shared function, capture/finalize.py (write_redacted_manifest
then finalize_capsule). Framework adapters, the @agent decorator and OTLP trace
ingest (otel/genai_ingest.py) call the same two steps through
finalize_in_process_capsule, after writing env.lock and replay.yaml and
running the step-5 scan (since v0.105.0). Their capsules
therefore get the residual pass, lineage.jsonl, evidence_digests and, when a
signing profile exists, a seal. Configured maskers are a nova capture option
and do not run there. A finalization failure in that path never fails the wrapped
call (or, for OTLP ingest, drop the ingested data): the capsule is left unsealed
and metadata.finalization_error says why.
See Run Capsule anatomy.
Capture also has proxy paths for clients that cannot be hooked in-process:
nova api-proxy (proxy/api_proxy.py) and nova mcp-proxy
(proxy/mcp_proxy.py).
Details: Run Capsule anatomy.
2 · Seal: automatic when configured (experimental)
capture/finalize.py:seal_capsule runs as the last step of capture (nova capture, a framework adapter, the @agent decorator or OTLP trace ingest), but
only if a NovaSeal configuration exists (NOVAFABRIC_SEAL_CONFIG, else
$NOVAFABRIC_HOME/novaseal.yaml, default ~/.novafabric/novaseal.yaml). Without one, sealing is skipped. A sealing
failure prints a warning and never fails the capture. A seal produces three
files in .seal/:
- a DSSE signature over the manifest,
- an RFC 3161 timestamp token,
- an entry in an append-only Merkle log.
nova seal propose / approve / verify add an optional maker-checker
signature chain on top (cli/seal_propose.py).
Details: Sealing and verification.
3 · Replay: nova replay (works today; intervention experimental)
cli/replay.py:replay_cmd → replay/_engine.py:ReplayEngine.run. The default
mode, mocked, re-runs the captured command with the recorded OpenAI and
Anthropic responses (and recorded SDK errors, raised again as the SDK's own
exception) and the recorded MCP tool results served from the capsule; other tools
run live, and outbound connections are reported. A capsule that records no
command to re-run (a framework-adapter or @agent-decorator (novafabric.sdk.agent) capsule, or one
imported from OpenTelemetry) is refused before anything is spawned
(CapsuleNotReplayable). What v0.104.0 served versus v0.105.0 is in
Replay modes › Release scope. forensic, semantic and exact analyse the capsule
without re-running it. Every mode writes
$NOVAFABRIC_HOME/replays/<ulid>/replay_result.yaml (or under -o <dir>). intervention is the only mode
that also writes a new capsule.
Details: Replay modes.
4 · Diff: nova diff A B (works today)
cli/diff.py:diff_cmd → diff/_engine.py:DiffEngine.compare. It takes two
capsule paths or run IDs and compares:
| Facet | How it matches | "Changed" means |
|---|---|---|
| Environment | Fixed keys from env.lock: Python version and interpreter, OS, architecture. An env.lock or capsule.yaml that exists but cannot be read or parsed raises CapsuleFileError (exit 2) |
Value differs |
| Model calls | diff/_align.py: a parent_span_id unique on both sides pairs exactly; the rest pair by sequence position, anchored on identical requests so an inserted or deleted call does not shift later pairs. Unpaired calls count as added or removed |
Provider (gen_ai.system), request model, messages or any other recorded gen_ai.request.* parameter (temperature, seed, …; recorded on one side only counts) differ, or the response choices differ |
| Tool calls | Exact (tool_name, arg_hash) match, each call used once, then position among calls with the same tool name |
result or arguments differ |
| Outputs | Every regular file under outputs/, recursively, by capsule-relative path; symlinks skipped and never followed (the evidence-digest rule) |
SHA-256 differs, or the path exists on one side only |
The output format is text, json or github-annotation. Whether the report
has any difference is one property, DiffReport.has_changes (a changed, added
or removed entry in any section). --assert-no-regressions exits 1 on it,
which turns the command into a CI gate; the text and github-annotation
formatters read the same property, so an added- or removed-only diff is an
error annotation, never a notice, and the json report carries it as a
top-level has_changes. Exit 1 means only "the comparison found a
difference"; a comparison that could not be made (a capsule ref that does not
resolve, an unknown asset ref, a usage error) exits 2 (ADR-0303). A record
line the engine cannot read (not UTF-8, not JSON, not a JSON object) is skipped,
counted per side in skipped_malformed_lines, and warned about on stderr; under
--assert-no-regressions it exits 2 as well, because the comparison is
incomplete (ADR-0303 Amendment 1). An unreadable or malformed capsule.yaml or
env.lock exits 2 in every format, with or without a gate flag, and
--assert-same-shape exits 2 when graph reconstruction skipped a malformed
source line (ADR-0303 Amendment 2). nova diff also accepts two name@version
asset references and diffs their specs field by field, in any of the three
output formats.
This contract (ADR-0303 and its Amendments 1 and 2) shipped in v0.105.0; in v0.104.0 an
unresolvable capsule ref exits 1, malformed lines are dropped silently, a malformed
env.lock crashes the diff, and request parameters are not compared.
| Exit | Meaning (cli/diff.py: EXIT_DIFFERENCES, EXIT_CANNOT_COMPARE) |
|---|---|
0 |
no difference, or a difference reported without a gate flag |
1 |
--assert-no-regressions found a difference (has_changes), or --assert-same-shape found a shape change; nothing else |
2 |
the comparison could not be made: an unresolvable capsule ref, an unknown asset ref, a usage error, --environment excluded a capsule, an unreadable or malformed capsule.yaml/env.lock (any format, gate or not), --assert-same-shape could not build a graph or skipped malformed graph-source lines, or --assert-no-regressions read skipped malformed lines (checked before has_changes) |
3 |
--significance found a statistically significant regression |
Diagram source (Mermaid)
flowchart LR
A[(capsule A)] & B[(capsule B)] --> R["_read_jsonl<br/>skip + count malformed lines"]
R --> L["logical_model_calls<br/>(ADR-0305)"] --> AL["diff/_align.py"] --> DR["DiffReport<br/>has_changes · skipped_malformed_lines"]
DR --> G{--assert-no-regressions}
G -->|lines skipped| E2["exit 2"]
G -->|has_changes| E1["exit 1"]
G -->|otherwise| E0["exit 0"]An OpenAI or Anthropic SDK call leaves two kinds of record in model-calls.jsonl:
the SDK hook's logical record and one httpx wire record per HTTP attempt, with no
response. Since ADR-0305 the wire records are marked transport and nova diff
aligns logical calls only, so a changed prompt is one changed pair; a capsule
captured before the marker existed is read the same way, through the reader-side
fallback described in Model-call record roles.
5 · Audit: nova verify, nova export-evidence, nova audit (works today)
"Audit" names three commands that produce or check evidence:
| Command | What it does | Module |
|---|---|---|
nova verify <capsule> |
Checks the seal: DSSE signature, timestamp, Merkle inclusion, manifest binding and per-file digests. Exit 0 or 1. It also verifies an Evidence Bundle .zip: manifest_hash, every artifact digest, the attestation signatures and subjects (--public-key pins the signer). |
cli/verify.py |
nova export-evidence <capsule> |
Builds the Evidence Bundle ZIP: the capsule, a lineage subgraph, Ed25519-signed in-toto DSSE attestations (run, redaction, lineage), vendored schemas and a manifest.json |
evidence/bundle.py:EvidenceBundleBuilder |
nova audit map / report / bundle / coverage |
Maps capsule evidence to the controls of a regulatory profile (default nist-ai-rmf) |
cli/audit.py, compliance/audit/ |
The Evidence Bundle is the artifact you hand to a third party. Its
manifest.json contains its own verification recipe: recompute each artifact's
SHA-256, then check the Ed25519 signatures. Any SHA-256 tool and Ed25519
verifier can run that recipe without NovaFabric. A bundle cannot be exported from
a capsule that has no redaction-proof.json.
Read next
- Sealing and verification
- Prove a run to an auditor (tutorial)
- How capture works (tutorial)