01 Product Principles

Product and Engineering Principles

Status: SHARP. The decisions below are binding. Denoising state: sharp (noise -> rough -> sharp -> converged) Updated: 2026-07-23

Purpose

State the immutable product/engineering decisions, the anti-goals, and the rationale that every other doc must obey. This is the conditioning prior for the whole corpus.

Immutable Principles

1. Greenfield, not migration. Design the system as if built clean today. We do not write a data-migration plan and do not port Rails/Thor/Sidekiq/ActiveAdmin structure. The legacy repo ../bioloupe-data-gov is evidence of intent, not a template.
2. Intent-faithful, not behavior-faithful. Preserve real domain capability and business truth; correct legacy accidents. Each correction is recorded inline as a legacy intent ledger row: legacy did X -> real intent Y -> greenfield does Z -> rationale/evidence.
3. Assets are durable nouns. Every Dagster asset is a persistent object (a table/view/model), never a task verb. (Dagster defines an asset as a persistent object such as a table or model — https://docs.dagster.io/guides/build/assets)
4. Evidence before assertion. Every downstream fact is traceable to an immutable source artifact or source record, extractor/model/prompt version, evidence claim, resolution/verifier receipt, canonical assertion, and curation overlay only when applicable. No high-impact fact without provenance.
5. Curation is first-class but separate from LLM extraction. LLMs structure/enrich/connect data in the normal pipeline; curator agents and humans write reviewed override/suppression decisions into an append-only log — not hidden admin columns. See 05-agentic-curation-and-overrides.md and 06-llm-extraction-and-verification.md.
6. SQL is the only consumer interface. Agents query stable q_* read models. See 07-sql-query-layer.md.
7. Slice-local modelling. Domain model and entity-resolution rules belong in the slice docs where the evidence lives; 20-pipeline-families.md synthesizes only cross-slice concepts.
8. Contracts before compute. Quality gates and schema contracts are defined before the assets they guard. See 08-contracts-and-quality.md.
9. Scenario parity, not schema parity. Replacement is measured by the product scenarios in 02-product-capability-contract.md, consumer cutover, and explainable source-backed answers — never by recreating legacy tables, routes, or jobs.
10. Identity is not contextual state. Stable identity spines own what a drug, disease, organisation, target, technology, endpoint, or adverse event is. Contextual and temporal slices own phase, activity, regulatory, ownership, commercial, and epidemiology state.
11. Source-native facts before synthesis. Trials, labels, publications, filings, and statistical releases retain their own grains and authority. Cross-slice models compare and derive from accepted canonical facts without blending incompatible observations.
12. Time has two clocks. Preserve when a fact was effective in the world and when Bioloupe observed it. Current state is a reproducible temporal projection, not a mutable flag or an undocumented recency window.

Anti-Goals (what must never be built)

• No API / REST / GraphQL / serializers / application server. No HTTP consumer contract of any kind.
• No data-migration tooling from the legacy database.
• No mechanical 1:1 port of legacy Thor tasks, Sidekiq jobs, ActiveAdmin screens, or Scenic view names.
• No fake confidence scores. Matching/verifier/curation trust is a discrete provenance category, not a continuous score (legacy EntityMatcher already treats trust as discrete provenance — specified inside the relevant slice docs and summarized in 20-pipeline-families.md).
• No silent auto-accept of fuzzy matches. (Legacy gotcha: bare flexifind without a verifier auto-accepted a trigram hit — corrected inside slice-local entity-resolution sections and summarized in 20-pipeline-families.md.)
• No reliance on the legacy dual status/qa_status fields. (Legacy README marks these deprecated/never-effective; replaced by the decision log in 05.)

Legacy Intent Ledger

Legacy did X	Real intent Y	Greenfield does Z	Evidence
100+ ActiveAdmin screens for curation	humans must review every fact	append-only curator-agent proposals + human-reviewed high-impact overrides	../bioloupe-data-gov/app/admin/, README “Human-in-the-Loop”
status + qa_status columns largely unused	track review lifecycle	explicit decision events with provenance	../bioloupe-data-gov/README.md (fields marked deprecated)
Scenic SQL views feed an /api	give consumers stable read shapes	q_* SQL read models, no API	../bioloupe-data-gov/db/views/, app/controllers/api/
NCIt-first intervention match auto-accepted top trigram	resolve intervention to correct concept	drug-first + LLM-verified resolution	../bioloupe-data-gov/CLAUDE.md matching gotchas
drug-level phase/status and parallel current/all-trials views	show present and historical development progress	contextual therapeutic programs, milestones, and temporal state axes	../bioloupe-data-gov/db/views/drug_development_programs_v11.sql; ADR 0008
disease statistics and brand sales collapsed reporting context into convenient fields	support disease and commercial profiles	full-grain epidemiology and commercial observations with explicit selection summaries	34-slice-epidemiology.md; 35-slice-commercial-intelligence.md