0012 Fixtures Are Not Live Parity Evidence

Fixtures Are Not Live Parity Evidence

Status: accepted

Checked-in fixtures make the graph deterministic and let tests exercise hard domain scenarios without internet access, credentials, legacy access, or paid inference. Their usefulness does not make them evidence of current source shape, complete source breadth, live paging behavior, or production data quality.

Every source fixture is declared in lib/sources/fixture_manifest.yaml as one of:

• authored_domain_scenario
• source_shaped_test_data
• captured_response_excerpt
• mixed_real_and_authored
• project_owned_catalog

Each declaration carries a review note and an explicit live_parity_proof field. Current fixtures all set that field to false. Tests require every checked-in source fixture to be declared and forbid an undeclared evidence class.

Fixture-only gates prove deterministic code, graph composition, SQL contracts, provenance plumbing, and representative negative cases. Live activation requires separate evidence:

1. a bounded real-source smoke proving transport, parser, identifiers, and immutable capture;
2. an explicit complete bootstrap or source-native incremental window;
3. completeness and activation metadata proving a bound was not promoted as a full cohort;
4. production operations and consumer cutover evidence.

Considered Options

• Treat realistic fixtures as frozen source truth.
• Copy large production samples into the repository.
• Remove fixtures and require network access in CI.
• Keep small classified fixtures and maintain a separate live-evidence ladder.

Consequences

• make check and make verify-bootstrap remain deterministic and free of external spend.
• Passing P01-P12 does not claim live parity or source breadth.
• A fixture may contain authored expected projections only when its class and source contract make that explicit.
• Production and staging reject fixture source mode, preventing representative test data from becoming deployed evidence.