00 Index

Docs Index and Reading Order

Status: SHARP. Canonical reading order, product topology, and live maturity table are decided. Denoising state: sharp (noise -> rough -> sharp -> converged) Updated: 2026-07-26

What this repo is

bioloupe-data is a greenfield, Dagster-native replacement for the legacy Rails “Data Gov” platform (../bioloupe-data-gov): a pharmaceutical-intelligence data system that ingests sources, extracts evidence with provenance, resolves identities, derives explainable canonical state, applies reviewed curation overlays when correction is needed, and serves stable SQL read models (q_*) to agents. There is no API. 01-product-principles.md defines the immutable engineering decisions; 02-product-capability-contract.md defines the product promise and completion standard.

How to read these docs

1. 01-product-principles.md — immutable engineering decisions and anti-goals.
2. 02-product-capability-contract.md — product scenarios, boundaries, parity standard, and completion contract.
3. 03-repository-blueprint.md — buildable repository and package shape.
4. 04-dagster-asset-graph.md — shared asset grammar, keys, partitions, lineage, and provenance boundaries.
5. 05-agentic-curation-and-overrides.md — reviewed correction and override boundary.
6. 06-llm-extraction-and-verification.md — inference contracts, lifecycle, reuse, sync/batch ownership, and evaluation extensions.
7. 07-sql-query-layer.md — the only consumer interface (q_*).
8. 08-contracts-and-quality.md, 09-operations-observability.md, and 10-testing-strategy.md — publication gates, operations, and proof.
9. 11-legacy-salvage-and-backfills.md, 12-source-first-ingestion.md, and 13-schema-evolution.md — legacy reuse, primary-source capture, and deployed-schema changes.
10. 20-pipeline-families.md — capability registry and ownership map.
11. 29-32 — foundational identity spines.
12. 21-27, 34, and 35 — source/evidence domain slices.
13. 33-slice-development-landscapes.md — cross-slice therapeutic-program synthesis.
14. 28-slice-forecasting-tpp.md — downstream forecasting boundary and optional readiness support.

Domain glossary terms live in root CONTEXT.md. Architectural decisions live in docs/adr/ and should be read when touching the decision area.

Current ADRs also pin the inference lifecycle, commercial-deal boundary, and disease-taxonomy ownership: task contracts, execution profiles, and routes are separate; Postgres stores inference lifecycle state; Dagster owns cohorts and visible batch-ingestion runs; direct structured output is the initial protocol; news owns source deal events while organisation intelligence owns durable deal identity and rights; and ADR-0010 makes the immutable Bioloupe release, not legacy state, the runtime disease source. ADR-0011 keeps regulator-approved use distinct from professional standard of care, while ADR-0012 makes the fixture/live evidence boundary explicit.

System map

flowchart LR
  Sources[Source systems]
  Raw[Raw source snapshots]
  Normalized[Normalized source records]
  Evidence[Evidence claims + governed inference enrichments]
  Resolution[Resolution candidates + verifier artifacts]
  Canonical[Canonical knowledge assets]
  Curation[Reviewed curation overlays]
  Query["q_* SQL read models"]
  Agents[Read-only SQL agents]

  Sources --> Raw --> Normalized --> Evidence --> Resolution --> Canonical --> Query --> Agents
  Curation -. optional correction/suppression .-> Canonical

Doc status (live)

Doc	Denoising state
00-index	sharp
01-product-principles	sharp
02-product-capability-contract	sharp
03-repository-blueprint	implemented core; source partitions/corpus-wide publication gaps visible
04-dagster-asset-graph	partial runtime; trials protected, natural partitions/other families open
05-agentic-curation-and-overrides	decided core/partial operations; seven owner routes
06-llm-extraction-and-verification	converged design; offline eval implemented, paid provider activation gated
07-sql-query-layer	sharp/partial runtime; trials generation-protected, other families in-place
08-contracts-and-quality	implemented core; one protected family, declarative automation open
09-operations-observability	implemented core; disposable production stack proven, live activation gated
10-testing-strategy	implemented fixture gate/live unproven
11-legacy-salvage-and-backfills	sharp
12-source-first-ingestion	sharp
13-schema-evolution	blank/versioned preparation implemented; live predecessor/restore rehearsal open
20-pipeline-families	sharp registry
21-slice-clinical-trials	converged NCT core + governed eligibility gap-fill; ChiCTR live hardening is a source-breadth expansion
22-slice-regulatory-approvals	sharp core vertical; non-FDA live expansion remains
23-slice-publications	sharp core vertical; full-text/table expansion remains
24-slice-news-intelligence	sharp deterministic vertical + bounded provider/extractor expansion
25-slice-standard-of-care	fail-closed review vertical; live decomposition/guideline sources block activation
26-slice-organisation-intelligence	sharp identity/deal/rights vertical
27-slice-discovery-search	sharp core vertical; owner text coverage expands
28-slice-forecasting-tpp	sharp downstream boundary
29-slice-drug-identity	sharp foundational vertical
30-slice-disease-biomarker-identity	converged foundational vertical; source breadth expands
31-slice-target-technology-identity	sharp deterministic vertical + review-only edge-case inference
32-slice-endpoint-adverse-event-identity	sharp core vertical; coverage expansion remains
33-slice-development-landscapes	converged core vertical
34-slice-epidemiology	converged core vertical; CDC mortality profile
35-slice-commercial-intelligence	converged core vertical; SEC Inline XBRL profile

This table is a reader-orientation snapshot, not a separate planning ledger.

Build-from-docs entrypoint

A builder starts from the product contract (01/02), constructs the shared system (03-12), establishes identity owners (29-32), builds source/evidence slices (21-27, 34, 35), and then builds cross-slice synthesis (33) and optional downstream support (28). 20 is the ownership registry throughout, not an extra runtime layer.

Core verticals are implemented with deliberately different live-source breadth. The direction is fixed: immutable source artifacts, explicit inference and curation lifecycles, owner-spine identity, source-grained evidence, contextual temporal state, generated q_catalog, W3C PROV-shaped q_provenance, and SQL-only consumption. Product completion is judged against scenarios P01-P12 in 02, not against legacy schema size.