30 Slice Disease Biomarker Identity
Pipeline Slice: Disease / Biomarker Identity Spine
Section titled “Pipeline Slice: Disease / Biomarker Identity Spine”Status: CONVERGED FOUNDATIONAL VERTICAL. Versioned Bioloupe taxonomy capture, approval/CDx mention extraction, cache-first NCIt/MeSH authority capture, semantic promotion, disease-biomarker relation publication, shared name resolution, six query models, checks, and provenance are implemented. Segment/treatment-line and source breadth remain additive coverage. Denoising state: converged (noise -> rough -> sharp -> converged) Updated: 2026-07-23
Purpose
Section titled “Purpose”Define the shared disease / biomarker identity spine that every therapeutic slice resolves into, the sibling of 29-slice-drug-identity.md. Legacy Data Gov treats diseases as one of the four knowledge-graph pillars: a directed acyclic graph (a disease has multiple parents, e.g. NSCLC is both “Lung Cancer” and “Solid Tumor”), anchored to NCI Thesaurus codes, narrowing from broad categories to biomarker-defined subtypes, with biomarkers as first-class hierarchical entities (../bioloupe-data-gov/docs/index.md, ../bioloupe-data-gov/docs/data-model.md “Domain 3: Diseases and biomarkers”, ../bioloupe-data-gov/app/models/disease.rb, biomarker.rb). This slice owns that identity so downstream resolved_disease_key(s) and resolved_biomarker_key(s) references have a real producer.
It deliberately separates grains: disease concept, disease hierarchy edge, biomarker concept, disease-biomarker prevalence link, patient segment (base and biomarker), and treatment line. It does not own approvals, trial design, or SOC assertions — only disease/biomarker identity and the links into it.
Current Decided Shape
Section titled “Current Decided Shape”flowchart LR Taxonomy["versioned Bioloupe taxonomy\nnodes + product DAG"] Sources["source artifacts\n+ NCIt / MeSH authority"] Evidence["disease / biomarker mentions\n+ prevalence claims"] Resolution["match mentions to curated identities\nor create review candidates"] Curation["reviewed taxonomy overrides"] Canonical["canonical.disease curated taxonomy\n+ canonical.biomarker + segments + treatment lines"] Query["q_diseases family"] Taxonomy --> Canonical Sources --> Evidence --> Resolution Resolution -. review candidates .-> Curation Curation --> Canonical --> Query Resolution --> Canonical
Source Inventory
Section titled “Source Inventory”| Source | Access | Identifier | Role |
|---|---|---|---|
| NCI Thesaurus (NCIt) | reference ontology / download or EVS API | nci_code | Core disease + biomarker identity anchor. NCI EVS publishes NCIt as a reference terminology/biomedical ontology with codes, terms, definitions, and relationships (https://evs.nci.nih.gov/); current downloads include OWL and tab-delimited releases with is-a relations, synonyms, and definitions (https://evs.nci.nih.gov/evs-download/thesaurus-downloads). Legacy diseases.nci_code is unique/not-null. |
| MeSH | reference vocabulary / RDF API | MeSH descriptor | Disease synonym/search and literature-indexing support, not the primary cancer-identity authority. NLM describes MeSH as a controlled, hierarchical vocabulary for indexing/cataloging biomedical information and provides RDF/API/download access (https://www.nlm.nih.gov/mesh/meshhome.html, https://id.nlm.nih.gov/mesh/). |
| Regulatory labels + CDx artifacts | approval evidence | label disease phrase, CDx biomarker term | High-authority mention/evidence for approved-use and companion-diagnostic contexts. These terms resolve into this spine; at most they can trigger lazy NCIt lookup for an exact, domain-valid, unambiguous authority concept. They do not define the spine by themselves. |
| Clinical trial registries / publications / news / SOC sources | evidence demand | condition, subgroup, biomarker text | Downstream mention streams that consume this spine. They may propose candidates and review demand; owner identity still lives here. |
| Versioned Bioloupe disease taxonomy | checked-in reviewed source release | release id, NCIt code, display name, synonyms, hierarchy level, parent/child edges | Runtime source of the product disease universe and rollup DAG. Exact bytes are immutable, releases cannot be reused with different content, and one active release publishes to canonical.disease / q_diseases. |
| Segment and treatment-line vocabularies | reviewed product vocabulary | segment/line keys and boundaries | Reusable patient-context vocabulary. These remain explicit curated additions rather than fields inferred from disease names. |
| Legacy Data Gov disease/biomarker rows | migration reconciliation | legacy ids, nci_code, synonyms, segments | Product-intent and coverage evidence only. Stage under migration.*; do not promote legacy rows directly into canonical disease/biomarker tables. |
Read-only Data Gov sampling on 2026-07-23 found 2,264 diseases (552 simplified, all NCIt-coded), 609 biomarkers (252 NCIt-coded), and 1,288 disease-biomarker rows. Of those relation rows, 931 had no prevalence; non-null values ranged from 0.051 to 96, mixing fractions and percentages, and 728 non-null potential_values payloads were JSON strings rather than arrays. These are hard-earned coverage and failure-shape signals, not source authority. Greenfield therefore stores prevalence only as a validated fraction in [0,1], stores potential values as text[], and never copies legacy relation rows into canonical data.
Source-Authority Decision
Section titled “Source-Authority Decision”NCIt-grounded concepts are the first-class identity authority for disease rows and one source of biomarker grounding. MeSH is a useful controlled vocabulary for literature/search aliases and disease indexing, but it is not the canonical cancer identity spine when an NCIt concept exists. Approval labels, CDx tables, trial conditions, publication cohorts, news, and SOC sources are mention/evidence producers: they create source-grounded disease/biomarker terms that must resolve through this owner spine. They can prove source usage, add aliases, and create review candidates; they do not define canonical.disease or the q_diseases disease universe by themselves.
Legacy Data Gov remains useful for reconciliation and product-intent checks: it tells us which NCI-coded diseases, biomarkers, curated-tree rows, segments, treatment lines, and prevalence links mattered to the product. It is not a runtime dependency or source authority. The greenfield path is versioned Bioloupe taxonomy + NCIt/MeSH validation -> canonical spine -> q_diseases family, while source-artifact mentions resolve into that spine or create review candidates.
Versioned Product Taxonomy
Section titled “Versioned Product Taxonomy”sources/bioloupe_taxonomy/disease_taxonomy captures one reviewed, checked-in release as immutable dataset, node, and edge artifacts. Reusing a release id with changed bytes fails. canonical/curated_disease_taxonomy publishes only the active exact release; source mentions and legacy sidecars cannot silently expand it.
The first checked-in release was recovered from the useful product intent behind legacy simplified=true rows and required parent closure, then detached from the legacy database and reviewed as a Bioloupe-owned artifact. That provenance explains the initial selection; it does not make legacy state authoritative or queryable at runtime.
The product hierarchy is likewise versioned with the release. It is not the full NCIt parent graph: NCIt relationships may propose reviewed changes, but they do not silently alter product rollups.
This source has no cron. A reviewed repository release and explicit active-release change are the refresh event; the normal downstream graph then rematerializes from those exact bytes.
The disabled migration.legacy_disease_taxonomy_seed sidecar remains available only for explicit reconciliation runs. Non-published legacy rows may become review demand when source usage or authority checks justify them; they never feed the default graph.
Slice-Local Domain Model
Section titled “Slice-Local Domain Model”Distinct grains (do not collapse):
| Concept | Grain | Legacy evidence | Notes |
|---|---|---|---|
| Disease | one curated disease concept | diseases (name, nci_code, synonyms) | canonical taxonomy identity; all_descendants precomputed for subtree queries. |
| Disease hierarchy edge | one curated parent->child relation | disease_parents | Versioned Bioloupe product DAG; multiple parents allowed; not a single tree and not the raw NCIt hierarchy. |
| Biomarker | one gene/protein/variant marker | biomarkers (name, synonyms, ncit_code, hierarchical parent_id) | known_values (Positive/Negative/High); embeddable for semantic search. |
| Disease-biomarker link | one disease+biomarker relevance assertion, with optional prevalence | disease_biomarkers (prevalence, potential_values) | Which markers matter in which disease. Prevalence is nullable unless the same evidence claim supplies a validated fraction. |
| Base segment | one stage/subtype patient partition | disease_base_segments | SOC/forecasting patient stratification. |
| Biomarker segment | one biomarker-defined patient partition | disease_biomarker_segments, biomarker_segment_biomarker | finer stratification on top of base segments. |
| Treatment line | one ordered line-of-therapy context | treatment_line, disease_treatment_lines | 1L/2L+/maintenance ordering per disease. |
Invariants
Section titled “Invariants”- one canonical disease per curated taxonomy concept, anchored by
nci_code; synonym variants point to it; - the hierarchy is a versioned Bioloupe product DAG (multiple parents) and is never silently replaced by the full NCIt hierarchy;
- biomarker identity is separate from disease identity but linked through a relevance assertion whose prevalence is optional and evidence-backed;
- segments reference a disease and (for biomarker segments) one or more biomarkers; they are partitions, not new diseases;
- disease/biomarker resolution is exact-first then verifier-checked — never bare trigram auto-accept; legacy keeps diseases on a dedicated matcher with broader/narrower hierarchy awareness, intentionally not the generic
EntityMatcher(../bioloupe-data-gov/CLAUDE.md“Diseases are intentionally NOT in EntityMatcher”).
Edge Cases That Define the Model
Section titled “Edge Cases That Define the Model”| Scenario | Wrong model | Required model |
|---|---|---|
| Label says “solid tumors with EGFR mutation” | mint one disease called solid tumors with EGFR mutation | disease phrase parks or resolves to a broader disease only with accepted basis; EGFR is a biomarker mention/value, not part of disease identity. |
| Source says “HER2-positive breast cancer” | create a separate disease for every marker-defined phrase | disease = breast cancer; biomarker = HER2 with value positive; segment may represent the patient partition. |
| Trial/source phrase is a stage or treatment line only | mint disease from metastatic, 2L, or maintenance | stage/line/setting is context on a disease or segment; no standalone disease identity. |
| Biomarker term is used as a target | resolve it in the biomarker spine | route target-role mentions to 31-slice-target-technology-identity.md; this spine only owns biomarker/readout role. |
| Legacy row has a useful synonym but no source artifact | promote it as canonical evidence | use as reconciliation/review candidate only; canonical alias requires source authority, source artifact, or reviewed curation. |
Slice-Local Entity Resolution
Section titled “Slice-Local Entity Resolution”| Subject kind | Source term examples | Candidate target types | Publishable basis | High-impact? |
|---|---|---|---|---|
disease_link | NSCLC, relapsed AML, breast cancer | canonical Disease / NCIt disease concept | published disease requires curated inclusion; source mentions may link by NCIt exact/cross-reference, context verification, semantic/LLM verifier, or reviewed override | yes |
biomarker_link | EGFR exon 20 ins, PD-L1 TPS ≥ 50%, HER2-positive | canonical Biomarker + value | exact/cross-reference/context-verified or reviewed override | yes |
disease_hierarchy_link | ”is NSCLC a solid tumor?“ | curated parent/child DAG edge | active Bioloupe taxonomy release, reviewed override, or reviewed NCIt-suggested candidate | medium |
segment_assignment | stage III, biomarker-defined cohort | base/biomarker segment | accepted segment boundary or reviewed override | medium/high |
Hierarchy-aware resolution is the key difference from drug identity: a narrower disease may need to roll up to a broader parent for analytics, so broader/narrower relationships are part of acceptance, not an afterthought.
Evidence Claim Schemas
Section titled “Evidence Claim Schemas”disease_identity_claim
Section titled “disease_identity_claim”A source-grounded assertion that a term denotes a specific disease, with source locator and extractor/model version.
biomarker_identity_claim
Section titled “biomarker_identity_claim”A marker-identity claim (name/synonym/NCIt) with allowed values.
disease_biomarker_prevalence_claim
Section titled “disease_biomarker_prevalence_claim”A claim that a biomarker occurs in a disease at some prevalence, with source authority.
disease_hierarchy_claim
Section titled “disease_hierarchy_claim”A claim of a parent/child relationship between two diseases (DAG edge).
Source-Derived Population Assets
Section titled “Source-Derived Population Assets”The first buildable pass should mirror the clean drug-identity split without copying source_drugs-style slop:
| Asset | Reads | Writes | Rule |
|---|---|---|---|
evidence/disease_biomarker_claims | approval indication/CDx claims first; then trial/publication/SOC/news mention streams | evidence.disease_biomarker_claims | Extract source-grounded disease terms, biomarker terms, biomarker values, line/setting/segment hints, and source locators. No canonical rows are written here. |
sources/disease_biomarker_authority/lookup_snapshots | distinct source-derived disease/biomarker terms | sources_disease_biomarker_authority.lookup_snapshots, provenance.source_artifacts | Cache-first NCIt/MeSH captures for missing lookup terms. The repository selects pending disease/biomarker authority requests before any live max_live_records bound; the authority_source resource owns fixture/live mode and HTTP adapters. Store raw payloads and releases; do not recreate legacy diseases/biomarkers. |
evidence/disease_biomarker_authority_claims | authority snapshots | evidence.disease_biomarker_claims | Parse exact authority matches into external_id, source_alias, and hierarchy-edge candidate claims. Closest/fuzzy source results go to review, not canonical. |
sources/bioloupe_taxonomy/disease_taxonomy | reviewed checked-in taxonomy bytes | immutable release, node, edge, and source-artifact rows | Validate uniqueness, edge closure, acyclicity, and release immutability before publication. |
canonical/curated_disease_taxonomy | active Bioloupe taxonomy release | canonical.disease, aliases, hierarchy, query provenance | Rebuild the exact active product taxonomy; no default legacy read and no source-mention expansion. |
canonical/source_disease_biomarker_identity_claims | accepted source/authority identity claims + active curated taxonomy | disease aliases and accepted biomarker identities | Source mentions can attach validated aliases to existing curated diseases; semantically valid unique NCIt biomarkers can publish. They do not expand the disease universe. |
canonical/disease_biomarker_links | paired accepted disease and biomarker claims from the same immutable artifact and parent source claim | canonical.disease_biomarker, query provenance | Publish only uniquely resolved claim-local pairs. Keep prevalence null unless supported; keep potential values as a validated subset of the biomarker text[] domain. |
canonical/disease_biomarker_tables | identity schema owner | disease, hierarchy, biomarker, prevalence, segment, and line tables | One DDL owner for the spine. Population arrives through explicit curated/source assets; table existence is not domain evidence. |
Minimal first slice: approval labels and CDx already produce disease_context_term, biomarker_terms, and biomarker_term, so they are the cheapest real mention stream. Once this spine has rows, resolution.regulatory_entities can resolve approval_indication_disease_link, approval_biomarker_link, and companion_diagnostic_link into real owner keys.
Rejected alternatives:
- Import legacy
diseases/biomarkersdirectly into canonical. Fast, but it makes old Data Gov rows source authority and hides provenance gaps. - Read the legacy simplified set on every run. It preserves historical coupling and lets mutable legacy state redefine the product taxonomy. The reviewed release is copied into this repository once and versioned here instead.
- Mint from every source disease phrase. Fast, but creates phrase-shaped identities like “metastatic EGFR-mutant NSCLC after platinum”, which are disease + biomarker + line/context, not one concept.
- Use MeSH as the primary cancer identity spine. Useful for literature indexing and aliases, but NCIt is the NCI cancer terminology/ontology carrying the source codes and relationships legacy already centered on.
- Make biomarker values separate biomarkers. “HER2” is the biomarker; “positive/high/mutated” is a value/readout or segment qualifier.
Implemented Approval/CDx Vertical
Section titled “Implemented Approval/CDx Vertical”The implemented path makes approval/CDx source mentions resolve to curated disease/biomarker owner keys while retaining unknown, ambiguous, fuzzy, and wrong-domain terms as evidence/review signals. It never reads legacy canonical rows as source authority.
Approvals emit disease_context_term, biomarker_terms, and CDx biomarker_term, giving a short source-first path through Dagster: source artifact -> evidence claim -> authority snapshot -> accepted owner identity or retained review signal. Ambiguous and wrong-domain authority hits never become identity.
flowchart LR ApprovalEvidence["evidence.indication_claims\n+ regulatory_safety_claims"] SpineClaims["evidence.disease_biomarker_claims"] AuthoritySnapshots["sources_disease_biomarker_authority.lookup_snapshots"] AuthorityClaims["evidence.disease_biomarker_authority_claims"] Taxonomy["sources_bioloupe_taxonomy\nactive immutable release"] Curation["reviewed taxonomy decisions"] Canonical["canonical.disease\ncanonical.biomarker\naliases"] Names["resolution.entity_names"] RegulatoryResolution["resolution.regulatory_entities"] Query["query.q_diseases\nquery.q_biomarkers\nq_approval_indications"] Taxonomy --> Canonical ApprovalEvidence --> SpineClaims --> AuthoritySnapshots --> AuthorityClaims AuthorityClaims -. review candidates .-> Curation --> Canonical --> Names AuthorityClaims --> Names Names --> RegulatoryResolution --> Query Canonical --> Query
Assets
Section titled “Assets”| Asset | Group | Reads | Writes | Acceptance rule |
|---|---|---|---|---|
sources/bioloupe_taxonomy/disease_taxonomy | source_bioloupe_taxonomy | checked-in reviewed release | immutable taxonomy source records + artifacts | Release id/content is immutable; nodes and edges are unique, closed, and acyclic. |
canonical/curated_disease_taxonomy | canonical | active Bioloupe taxonomy release | disease identities, aliases, hierarchy, provenance | Publish exactly the active release; no legacy connection or mention-driven expansion. |
evidence/disease_biomarker_claims | evidence | evidence.indication_claims, evidence.regulatory_safety_claims | evidence.disease_biomarker_claims | One row per non-empty disease or biomarker source mention. Disease, biomarker, value, line/setting, and source locator stay separate. |
sources/disease_biomarker_authority/lookup_snapshots | source_disease_biomarker_authority | distinct normalized terms from disease/biomarker claims | sources_disease_biomarker_authority.lookup_snapshots, provenance.source_artifacts | Cache-first exact NCIt lookup for disease and biomarker terms; MeSH lookup only for disease alias/search support. AuthoritySourceRepository owns pending-request selection and source-artifact writes; authority_source owns fixture/live capture. No legacy reads. |
evidence/disease_biomarker_authority_claims | evidence | authority snapshots | evidence.disease_biomarker_authority_claims | Parse exact external-id, preferred-name, synonym, and parent-edge claims. Wrong-domain, ambiguous, no-match, and fuzzy/closest matches remain coverage/review signals, not canonical facts. |
canonical/source_disease_biomarker_identity_claims | canonical | accepted authority claims + curated taxonomy | aliases and accepted biomarker identities | Attach validated disease aliases only to an existing curated NCIt disease. Publish an unambiguous semantically valid NCIt biomarker; retain all other outcomes as evidence/review signals. |
canonical/disease_biomarker_links | canonical | accepted claim-local disease/biomarker pairs | canonical relevance links + provenance | Both terms must resolve uniquely from the same parent claim and artifact; values must be allowed. |
existing canonical/disease_biomarker_entity_names | canonical | canonical disease/biomarker tables | resolution.entity_names | Rebuilds the owner-spine lookup surface after canonical population. |
existing resolution/regulatory_entities | resolution | resolution.entity_names, approval evidence terms | resolution.regulatory_entities | Existing approval disease/biomarker subjects now resolve to owner keys when exact accepted names/aliases exist. |
There is no generic mention-queue abstraction in this slice. Approval/CDx terms resolve through the existing resolution.regulatory_entities path once the spine has names. A cross-slice queue is justified only by a genuinely shared verifier lifecycle, not by repeated table names.
Evidence Table Shape
Section titled “Evidence Table Shape”evidence.disease_biomarker_claims:
| Column | Meaning |
|---|---|
claim_id | Stable hash from source artifact, source claim id, field path, normalized term, and extractor version. |
source_claim_id | Parent indication or CDx safety claim. |
source_record_key | Normalized source record key from the regulatory source record. |
source_family | Source family such as fda, ema, kegg, or cde. |
entity_kind | disease or biomarker. |
claim_kind | source_mention, source_alias, external_id, or hierarchy_edge. |
term / normalized_term | Source phrase and deterministic normalized phrase. |
role | disease or biomarker; target-role mentions are out of scope for this spine. |
value | Marker readout such as positive/high/mutated when present; not part of biomarker identity. |
context_term | Disease context for biomarker claims when available. |
source_artifact_key / source_locator | Immutable source artifact and exact field/span locator. |
extractor_version | Deterministic extractor version. |
review_reason | Optional reason a source mention cannot promote, such as compound_phrase, stage_only, line_only, target_role, or no_authority_match. |
sources_disease_biomarker_authority.lookup_snapshots mirrors the drug authority snapshot shape with entity_kind, source_family (ncit or mesh), lookup term, normalized term, source record id, source release, source URL, immutable URI, artifact hash, raw payload, and fetched timestamp.
evidence.disease_biomarker_authority_claims uses the same stable claim shape and adds external_id_source, external_id, preferred_name, authority_release, authority_basis (preferred_name_exact, synonym_exact, mesh_alias, hierarchy_edge, fuzzy_candidate, domain_mismatch, no_match), and promotion_state. Exact NCIt claims that survive the entity-kind semantic filter and are unambiguous can validate a match or create a review candidate; source mentions alone do not create the published disease taxonomy.
Promotion States
Section titled “Promotion States”| State | Can publish taxonomy? | Meaning |
|---|---|---|
accepted_authority | disease: existing curated inclusion; biomarker: yes | NCIt preferred-name/synonym exact match for exactly one semantically valid concept. Disease identity remains product-taxonomy controlled; a valid biomarker can publish directly. |
alias_only | alias only | MeSH descriptor or source synonym can attach to an existing NCIt-grounded disease; it cannot create the disease. |
review_required | no | Fuzzy, closest, no-match, compound, or hierarchy-sensitive result needs reviewed curation before canonical use. |
rejected_context | no | Term is a value, stage, line, setting, target-role mention, or other non-identity context. |
The source-derived matching asset may read source mentions, authority claims, existing curated identities, and reviewed curation overlays. It must not read migration.*, sources.legacy_data_gov.*, or the legacy Data Gov database connection.
Canonical and Mention Rules
Section titled “Canonical and Mention Rules”- Curated canonical disease ids are deterministic from NCIt code; slugs come from the accepted NCIt preferred name.
- Canonical biomarker ids are deterministic from NCIt code; values such as positive/high/mutated stay on claims, links, or segments.
- NCIt exact preferred-name or exact synonym claims can attach to an existing curated identity or propose a reviewed addition only after entity-kind semantic filtering: disease claims require disease-like NCIt concepts, while biomarker claims require gene/protein/receptor/enzyme-like marker concepts. Laboratory procedures, laboratory/test results, findings, organisations, and other wrong-domain concepts park for review.
- MeSH exact descriptor claims can attach aliases to an already NCIt-grounded disease, but cannot mint a disease by themselves.
- Legacy Data Gov disease/biomarker ids cannot insert or update canonical identity in this asset.
- A disease-biomarker relation is promoted only from disease and biomarker mentions sharing one parent source claim and immutable artifact; co-occurrence elsewhere is not enough.
prevalenceis a fraction in[0,1]or null, never an unlabeled percent/string;potential_valuesistext[]and must be a subset of the biomarker’s allowed values.- Source phrases that combine disease + biomarker + line/setting park as separate terms; no compound disease identity is minted.
review_requiredandrejected_contextrows are published only as evidence/review signals; query-layer identity views must not expose them as accepted disease or biomarker rows.
Checks
Section titled “Checks”schema_contractandprovenance_coverageon new evidence and query-visible rows.- Source-artifact immutability for every source mention and authority snapshot.
- Bioloupe taxonomy release immutability, node/edge closure, uniqueness, and DAG acyclicity.
- No canonical disease without NCIt code unless reviewed curation explicitly accepts it.
- No canonical biomarker without NCIt code unless reviewed curation explicitly accepts it.
- No MeSH-only canonical disease mint.
- No
migration.*,sources.legacy_data_gov.*, or legacy DB read in source-derived mention population. - No
review_required/rejected_contextauthority claim reachesq_diseasesorq_biomarkersas accepted identity. - Existing referential-integrity gates prove approval disease/biomarker keys resolve to
q_diseases/q_biomarkers. - Disease-biomarker link values are typed and domain-valid; ambiguous or cross-claim pairs publish nothing.
Acceptance Tests
Section titled “Acceptance Tests”- Extractor test: approval indication with
HER2-positive breast canceryields diseasebreast cancer, biomarkerHER2, valuepositive, not a compound disease. - CDx test: companion diagnostic biomarker rows become biomarker source mentions with exact source locators.
- Authority parser test: NCIt exact preferred-name/synonym payloads create external-id and alias claims; fuzzy/no-match payloads do not.
- Canonical test: NCIt-grounded claims attach aliases or create reviewed-addition candidates; MeSH-only and legacy-only claims do not publish query rows.
- Resolution test: after canonical population and
entity_namesrebuild,resolution.regulatory_entitiesresolves approval disease/biomarker terms to owner keys. - Relation test: the KRAZATI label’s claim-local NSCLC and KRAS G12C mentions publish one relevance link, while unpaired or ambiguous mentions publish none.
- Scenario acceptance: P03 proves NSCLC joins taxonomy, biomarker, approval, SOC, trial, program, and news surfaces; P09 separately proves burden observations retain comparable contexts.
- Contract test:
q_diseases,q_biomarkers,q_approval_indications, andq_companion_diagnosticskeep schema/provenance gates green on a tiny fixture. - Dagster definition test:
dg check defsloads the new source/evidence/canonical assets with resource keyspostgres,source_http, andauthority_source.
Beyond this vertical’s current source breadth: full NCIt bulk ontology import, segment/treatment-line population, more trial/publication/news/SOC relation evidence, and evaluated LLM-assisted fuzzy matching. Legacy identity remains reconciliation-only.
Curation Override Subjects
Section titled “Curation Override Subjects”| Subject | Why high-impact | Example |
|---|---|---|
disease_merge_override | wrong split fragments a disease’s evidence | merge duplicate disease rows. |
disease_split_override | wrong merge contaminates two diseases | split an over-merged disease. |
disease_reparent_override | wrong hierarchy breaks rollups | fix NSCLC parentage. |
biomarker_identity_override | wrong marker corrupts segments/indications | force HER2 vs HER3 disambiguation. |
segment_boundary_override | over/under-splitting changes SOC/forecasting | merge/split a biomarker segment. |
Canonical Assets
Section titled “Canonical Assets”canonical.disease— one curated disease concept with NCIt anchor + precomputed descendants.canonical.disease_hierarchy— curated product DAG parent/child edges.canonical.biomarker— one marker with allowed values + hierarchy.canonical.disease_biomarker— disease+biomarker relevance links with optional prevalence.canonical.disease_segment— base and biomarker segments.canonical.disease_treatment_line— ordered line-of-therapy contexts.resolution.disease_candidates/resolution.biomarker_candidates— proposed links with basis + verifier state.
q_diseases Family
Section titled “q_diseases Family”All read-only SQL models in the query schema; visible in generated q_catalog, provenance-required rows resolve through q_provenance.
| Model | Grain | Agent question |
|---|---|---|
q_diseases | one published curated disease from canonical.disease | disease search by name/synonym/NCIt; hierarchy level, curated-tree flag, descendants. |
q_disease_hierarchy | one curated parent/child edge | broader/narrower navigation and product rollups. |
q_biomarkers | one biomarker | marker search with allowed values and hierarchy. |
q_disease_biomarkers | one disease+biomarker link | which markers matter in a disease, with prevalence. |
q_disease_segments | one base/biomarker segment | patient stratification options for SOC/forecasting. |
q_disease_treatment_lines | one disease+line context | line-of-therapy options per disease. |
resolved_disease_key(s) and resolved_biomarker_key(s) used across 21-35 are defined here as stable keys into q_diseases/q_biomarkers (e.g. disease:<slug>, biomarker:<slug>). Downstream slices reference these keys; they do not mint disease/biomarker identity.
segment_key and treatment_line_key used by SOC and forecasting support surfaces are also produced here through q_disease_segments and q_disease_treatment_lines. A segment is a disease-scoped patient-context vocabulary row, not a SOC assertion; SOC owns recommendations for a segment, while this spine owns the reusable disease/biomarker/line context that makes those recommendations comparable.
q_diseases contract seed
Section titled “q_diseases contract seed”| Field | Required content |
|---|---|
| Purpose | Canonical disease discovery and the resolution target for every disease mention. |
| Grain | One published curated disease concept. |
| Stable row key | resolved_disease_key (disease:<slug>), used by q_provenance.q_row_key. |
| Required columns | resolved_disease_key, disease_name, nci_code, synonyms (text[]), hierarchy_level, parent_keys (text[]), descendant_keys (text[]), in_curated_tree, assertion_state, provenance_key. |
| Search affordances | trigram/full-text on disease_name/synonyms; GIN on synonyms/parent_keys/descendant_keys; btree on hierarchy_level. |
| Provenance rule | provenance_grain=bundle; identity bundle expands to source ontology artifact, resolution/verifier receipt, canonical assertion, and curation overlay only when an override exists. |
| Refresh | materialized view/table; full refresh, targeted refresh on reviewed identity/hierarchy overrides. |
| Blocking checks | schema_contract, provenance_coverage, identity-uniqueness (one canonical row per concept), acyclic-hierarchy check. |
Checks and Quality Gates
Section titled “Checks and Quality Gates”- every
resolved_disease_key/resolved_biomarker_keyin anyq_*model exists inq_diseases/q_biomarkers(enforced corpus-wide by thereferential_integritygate in08-contracts-and-quality.md); - every
segment_keyandtreatment_line_keyin anyq_*model exists inq_disease_segments/q_disease_treatment_lines; - the curated disease hierarchy is acyclic (DAG) and
descendant_keysare consistent with edges; - no two curated canonical diseases share an
nci_code; - biomarker values come from the allowed-value set;
q_diseasesfamily schema matches generatedq_catalog; provenance-required rows resolve throughq_provenance.
Concepts Promoted to 20-pipeline-families.md
Section titled “Concepts Promoted to 20-pipeline-families.md”- Disease/biomarker identity is a foundational spine consumed by all therapeutic slices; the cross-slice “Disease / indication” concept resolves to this spine’s
resolved_disease_key. - Disease vs hierarchy edge vs biomarker vs prevalence vs segment vs treatment line are distinct grains.
Owned Elsewhere
Section titled “Owned Elsewhere”- Disease burden observations, comparable series, and reviewed summaries are implemented in
34-slice-epidemiology.md; this spine owns only disease identity and reusable patient-context vocabularies. - SOC owns treatment assertions for a segment, while this spine owns the segment and treatment-line keys.
- Target-role mentions resolve through
31-slice-target-technology-identity.md, not biomarker identity.