Skip to content

29 Slice Drug Identity

Pipeline Slice: Drug / Therapeutic Product / Intervention Identity Spine

Section titled “Pipeline Slice: Drug / Therapeutic Product / Intervention Identity Spine”

Status: SHARP FOUNDATIONAL VERTICAL. Regulatory active-substance/brand claims, cache-first NCIt/RxNorm enrichment, canonical identities, aliases, external ids, brands, regimens, shared name resolution, five query models, checks, provenance, and legacy reconciliation sidecars are implemented. Contextual development state is deliberately absent from this spine. Denoising state: sharp (noise -> rough -> sharp -> converged) Updated: 2026-07-23

Define the shared drug / therapeutic product / intervention identity spine that every other slice resolves into. In legacy Data Gov, drugs are explicitly the center of the knowledge graph: drugs are tracked at INN/compound level, the same compound appears under many names (“Pembrolizumab”, “KEYTRUDA”, “MK-3475”, “lambrolizumab”), and trials, approvals, publications, news, SOC, organisations, and forecasting all hang off drug identity (../bioloupe-data-gov/docs/index.md, ../bioloupe-data-gov/docs/data-model.md “Domain 1: Drugs”). This slice owns that identity so downstream resolved_drug_key(s) references have a real producer.

This spine deliberately separates grains that legacy collapsed into one drugs table plus satellites: active substance/ingredient, branded/marketed product, regulatory product/application, combination/regimen, trial/publication intervention component, and non-drug intervention. Flattening these is the documented source of matching bugs (../bioloupe-data-gov/app/services/drug_matching_service.rb formulation/isotope guards; ../bioloupe-data-gov/docs/publication_issues_tracker.md issue 50 “DrugLinker false-matches non-drug interventions to drugs”). It also separates stable identity from contextual development state: phase, clinical activity, regulatory state, and historical progression belong to therapeutic development programs in 33-slice-development-landscapes.md, not to a drug row.

Drug identity is a foundational spine, not a downstream slice. It produces canonical drug nouns and the resolution surface that trials/approvals/publications/news/SOC/organisations consume. It does not own approvals, trial design, therapeutic development state, or SOC assertions — only identity and the links into it.

flowchart LR
  Sources["source artifacts\n+ authority vocabularies"]
  Evidence["drug identity / alias / product claims"]
  Resolution["intervention resolution candidates + verifier"]
  Canonical["canonical.drugs + ingredients + products + regimens + external ids"]
  Curation["reviewed merge/split/identity overrides"]
  Query["q_drugs family"]

  Sources --> Evidence --> Resolution --> Canonical --> Query
  Curation -. correction / merge / split .-> Canonical
SourceAccessIdentifierRole
ChEMBL (project-deprecated legacy remainder)mapping-only if salvagedchembl_idDeprecated for this greenfield repo, not modelled as a live ingestion source. Legacy carries ~3.7k source_drugs.source='chembl' rows and drugs.chembl_id, but the product decision is to stop carrying ChEMBL as a go-forward source. Upstream ChEMBL itself remains active (EMBL-EBI describes ChEMBL as a current curated bioactivity database and published ChEMBL 36 as live in 2025: https://www.ebi.ac.uk/chembldb/ and https://www.ebi.ac.uk/about/news/updates-from-data-resources/chembl-36/). Therefore greenfield may retain old ChEMBL ids only as deprecated historical mappings; do not add new ChEMBL ingestion.
NCI Thesaurus (NCIt)reference vocabularyncit_id / NCIt codedrug and non-drug concept vocabulary; NCI describes NCIt as its core reference terminology and biomedical ontology (https://www.cancer.gov/about-nci/organization/cbiit/vocabulary). Procedures/devices/radiation live here (../bioloupe-data-gov/app/models/source_drug.rb, ncit_concepts).
RxNormreference vocabularyrx_norm_idnormalized clinical-drug naming; NLM says RxNorm provides normalized names for clinical drugs and links to common drug vocabularies (https://www.nlm.nih.gov/research/umls/rxnorm/index.html). Legacy source_drugs rows use source='rxnorm'.
Adislegacy/proprietary mapping unless source contract is re-establishedadis_idDevelopment-stage drug naming/synonyms (source_drugs.source='adis') from a paid/proprietary source family, not a public source-first adapter. Springer describes AdisInsight as a drug R&D intelligence database with platform/API/data-feed access (https://adisinsight.springer.com/). Salvage existing ids/names as mapping-only traceability; do not mint canonical drugs from Adis unless a current source contract, release/version, and ingestion adapter are supplied.
Regulatory productsfrom approvals slicebrand/application/ingredientbrand/product identity evidence (brand_drugs, FDA/EMA/KEGG/CDE); FDA Orange Book data files identify application products (https://www.fda.gov/drugs/drug-approvals-and-databases/orange-book-data-files) and Purple Book lists approved biological products, biosimilars/interchangeables, and reference products (https://purplebooksearch.fda.gov/).
Free-text mentionstrials/publications/newsintervention stringsidentity demand, resolved into this spine, not new identity.

Live shape sampled from datalake (2026-06): active drugs 13,967, active brand_drugs 13,118, active source_drugs 19,237 (Adis 6,909, NCIt 6,589, ChEMBL 3,736, RxNorm 2,003), trial_arm_interventions≈95.2k (each carrying drug_id xor ncit_concept_id), publication_interventions≈60.1k (drug-linked ≈39.6k, ncit-linked ≈47.6k). This confirms two resolution targets — canonical drug and NCIt concept — not one.

Source liveness decision. Presence of source_drugs rows is not proof of an active source. A 2026-06 read-only legacy sample shows source_drugs is load-bearing as a historical cross-reference table, but not clean evidence that every source should be re-ingested: active source_drugs counts were Adis 6,909, NCIt 6,589, ChEMBL 3,736, RxNorm 2,003; linked active drugs were Adis 6,739, NCIt 6,447, ChEMBL 3,601, RxNorm 1,940; 5,506 active drugs had no source-vocabulary FK at all. The liveness split is:

SourceLegacy DB signalRuntime signalGreenfield decision
NCItrecent updates through 2026-06-18; 6,447 linked active drugsactive NCIt concept linking/conversion in ../bioloupe-data-gov/app/models/ncit_concept.rb and ../bioloupe-data-gov/app/admin/ncit_concepts.rblive reference vocabulary; the implemented path enriches an existing regulator-derived identity and routes collisions to review.
RxNorm1,940 linked active drugs; source_data present on nearly all rows but last updated 2024-11-18RxNormService and RxNormMatching::Task still write SourceDrug.rxnorm rows (../bioloupe-data-gov/app/services/rx_norm_service.rb, ../bioloupe-data-gov/app/tasks/rx_norm_matching/task.rb)candidate_live public reference vocabulary; the implemented path enriches an existing identity rather than minting from a lookup response.
ChEMBL3,601 linked active drugs, but source rows are old/static and source_data is emptyno current source-drug importer; README says ChEMBL DB is read-only reference and not source of truth (../bioloupe-data-gov/README.md)mapping_only_remainder; expose only as deprecated/mapping external ids, never as identity mint source.
Adis6,739 linked active drugs, all source rows created/updated 2024-06-20 with no source_datano active importer found; only FK/admin/search use and a static total_drugs_adis.json artifactmapping_only_remainder until a current licensed source contract exists; do not treat Adis as an authoritative minting source.

The implemented mint policy is deliberately narrower than legacy source_drugs: an exact regulator active-substance/ingredient claim or reviewed curation can create the base identity. NCIt/RxNorm snapshots enrich that existing identity with official ids and aliases; ambiguous or colliding authority results route to review. ChEMBL and Adis may appear only in mapping/legacy external-id surfaces unless a future source contract deliberately promotes them.

No source_drugs rule. Legacy source_drugs is a collapsed cross-reference/blob table and must not exist in the greenfield model. Its useful meaning is split into typed rows: vocabulary/source identifiers go to canonical.drug_external_ids, names go to canonical.drug_aliases, and source payload snapshots stay in source/migration tables.

No legacy canonical producer rule. Legacy Data Gov public.drugs, public.source_drugs, and public.brand_drugs are not primary sources. Stage them under migration.* only so old ids, aliases, source-code mappings, and review context can be matched against source-derived identities. They must not insert or update canonical.drugs, canonical.drug_aliases, canonical.drug_external_ids, or canonical.drug_brands by themselves. Canonical minting remains source/curation controlled: FDA/EMA/KEGG/CDE active-substance claims or reviewed curation create base identities; NCIt/RxNorm enrich them; Adis/ChEMBL stay mapping-only unless a current source contract promotes them.

Legacy Data Gov did not get its canonical drugs from FDA approvals. It had one old bulk construction path, then several later maintenance/linkage paths. The old bulk path lived mostly in ../bioloupe-ai-solutions, not in the Rails workflow:

flowchart TD
  Adis["Adis scrape/export + legacy source_drugs(source=adis)\npaid/proprietary names, synonyms, profiles"]
  Chembl["ChEMBL local DB / synonym dump\nmolecule ids, preferred names, synonyms, phase"]
  MatchA["match_chembl_to_adis_by_name\nAdis names/synonyms -> ChEMBL candidates"]
  MatchB["match_adis_to_chembl\nbest ChEMBL molecule per Adis drug"]
  Ncit["NCIt EVS API search\ncodes, names, synonyms"]
  Bundle["chembl_adis_to_ncit.json\nlegacy matched identity bundle"]
  Import["import_drugs / data-lake import-drugs\nupsert source_drugs + insert/update drugs"]
  SourceDrugs["source_drugs\nsource=adis|chembl|ncit"]
  Drugs["drugs\nBioloupe id + source FK ids + all_synonyms"]

  Adis --> MatchA
  Chembl --> MatchA
  MatchA --> MatchB
  Adis --> MatchB
  MatchB --> Bundle
  Ncit --> Bundle
  Bundle --> Import
  Import --> SourceDrugs
  Import --> Drugs
  SourceDrugs --> Drugs

Evidence:

  • ../bioloupe-ai-solutions/data-collection/build-drug-list.sh runs match_chembl_to_adis_by_name, match_adis_to_chembl, match_chembl_adis_to_ncit, then import_drugs.
  • ../bioloupe-ai-solutions/data-collection/src/drugs_matches/match_chembl_to_adis_by_name.js reads Data Gov source_drugs where source='adis', combines Adis names/synonyms with ChEMBL synonym data, and assigns legacy B ids.
  • ../bioloupe-ai-solutions/data-collection/src/drugs_matches/match_adis_to_chembl.js queries ChEMBL molecule_dictionary / molecule_synonyms by Adis names and prefers exact preferred-name and higher-phase matches.
  • ../bioloupe-ai-solutions/data-collection/src/drugs_matches/match_chembl_adis_to_ncit.js calls the NCIt EVS search API for the combined synonym set and writes chembl_adis_to_ncit.json.
  • ../bioloupe-ai-solutions/data-collection/src/drugs_matches/import_drugs.js and ../bioloupe-ai-solutions/data-lake/import-drugs/index.js upsert source_drugs for adis, chembl, and ncit, then insert/update drugs.
  • Legacy artifacts include a 5.4k-line drugs_list_from_adis.csv / list_of_unique_drugs.csv; the 118k-line clinical-trial drug-name list was used for exploration/enrichment, not as the final Rails drugs minting path.

Rails-era paths after that bulk construction are mostly linkage and maintenance:

Legacy pathReadsWritesMints drugs?Greenfield interpretation
Old Adis + ChEMBL + NCIt bulk importAdis scrape/export or source_drugs(source='adis'), ChEMBL local DB/synonyms, NCIt EVS searchsource_drugs(adis/chembl/ncit), drugsyesHistorical construction evidence only. Do not port mechanically; model current source artifacts instead.
NCIt concept conversion/admin linkingncit_concepts / NCIt payloadssource_drugs(ncit), drugs.ncit_id, sometimes a new drugs rowyes in legacyUseful authority-linking precedent. In the implemented Dagster path this becomes NCIt snapshot -> verified authority claim -> external id/alias enrichment or review on an existing source-derived drug.
RxNorm matchingexisting drugs, RxNorm/RxNav service resultssource_drugs(rxnorm), drugs.rx_norm_id, RxNorm hierarchy rowsmostly no; links existing drugsPublic vocabulary enrichment/linkage after an identity exists, unless a future explicit RxNorm source policy allows minting.
Salt / parent-child cleanupexisting source_drugs hierarchiesdrugs.parent_id, occasional parent/child drug rowscleanup-derivedResolution/curation operation, not bootstrap. It can propose split/parent links with verifier evidence.
Regulatory matching: FDA/EMA/KEGG/CDE/Purple Bookregulatory product/application recordssource record drug_id, drug_approvals.drug_id, brands/approval factsnoApproval/product evidence. It resolves to the drug spine; it does not define the spine.
Brand and label workflowsdrug_approvals, labels, active substances, brand namessource-grounded active-substance and brand claimsyes for exact active-substance identity; no for a brand name aloneBrand/product layer owned by this spine plus the approvals slice.
ActiveAdmin/manual editshuman-entered Drug / SourceDrug recordsdrugs, source_drugs, locksyes, manuallyReviewed curation overlay with provenance; not hidden extraction logic.

Therefore the greenfield source graph treats the legacy catalog as reconciliation evidence, not as a bootstrap catalog. FDA approvals, labels, Purple Book, EMA, KEGG, and CDE provide active-ingredient/product/brand claims that create or resolve the base spine. NCIt/RxNorm snapshots then add authority ids and aliases. Legacy rows preserve continuity only after matching a source-derived canonical identity or becoming reviewed curation input.

Legacy brand_drugs is a derived rollup/cache over regulatory approvals, not a source-authoritative brand catalog. Data Gov builds approval rows first, then groups them by drug_approvals.brand_name, canonicalizes through existing BrandDrug.find_canonical, and stores one selected approval per region (usa_approval_id, eu_approval_id, jp_approval_id, cn_approval_id) (../bioloupe-data-gov/app/workflows/drugs_workflow.rb, ../bioloupe-data-gov/lib/tasks/regulatory/brands.thor, ../bioloupe-data-gov/app/models/drug_approval.rb). Regulatory brand names enter through DrugApproval mappers:

Approval sourceRegionBrand field used by Data Gov
FdaDatum / openFDAUSAfda_datum.brand_name, filtered to product records with the same brand name.
EmaApprovedDrugEUmedicine_name.
KeggApprovedDrugJPbrand_name.
ChinaApprovedDrug / CDECNnormalized product_name via Regulatory::CdeBrandNameNormalizer.
PurplebookRecordUSAproprietary_name, falling back to proper_name when proprietary name is blank or placeholder.

The legacy model is intentionally not copied because it collapses four grains into one mutable row:

  1. brand/trade-name identity (name, aliases);
  2. marketed/regulatory product facts (active_ingredient, dosage_form, strength, route, company, atc_code, inn_key);
  3. representative regional approvals (one best approval per USA/EU/JP/CN, selected by approval status/originality/label presence);
  4. operational search/review/sales anchor (news_drug_mentions, brand_sales, ActiveAdmin review surfaces).

Read-only legacy sampling on 2026-06-30 confirms this is dirty enough to require salvage semantics: active brand_drugs = 13,118; linked to a legacy drug = 3,663; rows with zero regional approval links = 3,897; rows with aliases = 1; duplicate normalized brand-name groups = 303 covering 726 rows; approval brand names linked to multiple legacy drugs = 14; rows where a selected regional approval points at a different drug than brand_drugs.drug_id = 8. Active approval brand-name source counts were FdaDatum USA 27,322, EmaApprovedDrug EU 2,326, ChinaApprovedDrug CN 2,046, KeggApprovedDrug JP 900, and PurplebookRecord USA 687.

Greenfield treatment:

  • migration.legacy_drug_brand_baseline stages public.brand_drugs as a provenance-bearing salvage baseline, including all four regional approval links as structured context, not as the canonical model shape.
  • canonical.drug_brands is only brand/trade-name identity linked to a canonical drug. It must not copy approval/product attributes as brand identity.
  • Approval/product facts stay with the regulatory approvals/product slice and can expose many-to-many brand-to-approval relationships in query models. Do not model one fixed approval column per region.
  • Unmatched legacy brand rows remain staged for review or future source-derived backfill. They do not mint canonical drugs or brand links by themselves.

The complete implementation is source-first. Legacy rows are a reconciliation side input, never the canonical producer:

flowchart TD
  Ncit["NCIt source snapshot\nsource vocabulary"]
  RxNorm["RxNorm/RxNav source snapshot\nsource vocabulary"]
  Reg["FDA/EMA/KEGG/CDE approval snapshots\nproduct/brand/ingredient claims"]
  Mentions["trial/publication/news/SOC mentions\nidentity demand"]
  Legacy[(Legacy Data Gov RDS\nread-only)]
  Bridge["migration.legacy_drug_baseline\nold ids + stale mappings"]

  Claims["evidence.drug_identity_claims\nalias/id/brand/product claims"]
  Resolve["resolution.drug_intervention_candidates\n+ verifier/review"]
  CanonDrug["canonical.drugs"]
  CanonAlias["canonical.drug_aliases"]
  CanonIds["canonical.drug_external_ids"]
  CanonBrand["canonical.drug_brands"]
  CanonCombo["canonical.drug_combinations\n+ components"]
  Names["resolution.entity_names\nentity_type=drug"]
  Query["q_drugs family"]

  Ncit --> Claims
  RxNorm --> Claims
  Reg --> Claims
  Mentions --> Resolve
  Legacy --> Bridge -. reconciliation only .-> Resolve

  Claims --> Resolve
  Resolve --> CanonDrug
  Resolve --> CanonAlias
  Resolve --> CanonIds
  Resolve --> CanonBrand
  Resolve --> CanonCombo
  CanonDrug --> Names
  CanonAlias --> Names
  CanonBrand --> Names
  CanonDrug --> Query
  CanonAlias --> Query
  CanonIds --> Query
  CanonBrand --> Query
  CanonCombo --> Query

Source-derived implemented assets:

AssetReadsWritesPurpose
evidence/drug_identity_claimsnormalized.approval_source_recordsevidence.drug_identity_claimsExtract primary-source active-substance, source-alias, and brand-name claims from regulatory approval/product records. Product brand claims link only when the source row has exactly one active substance.
sources/drug_authority/lookup_snapshotsevidence.drug_identity_claims substance termssources_drug_authority.lookup_snapshots, provenance.source_artifactsCache-first official NCIt/RxNorm captures for observed drug terms. The repository selects missing (source_family, normalized_term) requests before any live max_live_records bound; the authority_source resource owns fixture/live mode and HTTP adapters. Raw authority payloads stay in source tables; this is not a recreated source_drugs table.
evidence/drug_authority_claimssources_drug_authority.lookup_snapshotsevidence.drug_identity_claimsParse exact authority matches into external_id claims and source aliases. No fuzzy or closest-result promotion.
canonical/source_drug_identity_claimsapproval/product evidence.drug_identity_claimscanonical.drugs, canonical.drug_aliases, canonical.drug_brandsPromote source-derived approval/product claims into the base canonical drug and brand identity surface. Legacy rows are not read by this asset, and NCIt/RxNorm lookup snapshots are not a prerequisite for approval-product resolution.
canonical/drug_authority_external_idsevidence.drug_authority_claims, canonical/source_drug_identity_claimscanonical.drug_external_ids, canonical.drug_aliases, resolution.review_queueEnrich existing same-normalized source-derived drugs with official NCIt/RxNorm external ids and source aliases; external-id collisions route to review. It does not mint approval-local identity, copy legacy source_drugs, or block canonical.drug_brands / resolution.regulatory_entities.

Legacy reconciliation assets:

AssetReads legacyWrites localPurpose
sources/legacy_data_gov/drug_baselinepublic.drugs d plus left joins to public.source_drugs for d.ncit_id, d.rx_norm_id, d.adis_id, d.chembl_idmigration.legacy_drug_baselineReconcile old Bioloupe ids, names, and stale source-code mappings to source-derived canonical drugs; never mint canonical rows.
sources/legacy_data_gov/drug_brand_baselinepublic.brand_drugs plus linked drug_approvals contextmigration.legacy_drug_brand_baselinePreserve legacy brand review/search context while treating region approvals and product facts as source context, not canonical brand shape.
sources/legacy_data_gov/drug_combination_baselinepublic.combination_drug_components, public.drugs.parent_id, drugs.is_combinationmigration.legacy_drug_combination_baselineReconcile legacy combination/regimen shells to source-derived component drugs; unmatched rows stay staged.
sources/legacy_data_gov/drug_target_baselinepublic.drug_target_actions joined to legacy drugs/targetsmigration.legacy_drug_target_baselineTransitional reconciliation input until the target/technology spine fully owns this.

All four assets use the disabled-by-default legacy_data_gov_source, are bounded by its max_import_records config, enforce read-only database sessions, and keep source_locator + content_hash. Their output is useful for audit, continuity matching, and coverage reports only. The drug baseline query must filter out soft-deleted rows from publication surfaces by default while still allowing an audit mode to stage deleted_at rows for reconciliation reporting.

Topology Decision: Base Identity vs Authority Enrichment

Section titled “Topology Decision: Base Identity vs Authority Enrichment”

Approval-derived active-substance and brand claims are the base identity surface: they create the canonical drug/brand rows and the resolution.entity_names entries that approvals, trials, publications, SOC, and news can resolve against. NCIt/RxNorm are authority enrichment sources for external ids after a source-derived drug exists. The source asset is intentionally thin: it asks AuthoritySourceRepository for pending missing requests, asks authority_source for fixture/live capture, then persists immutable snapshots and source-artifact receipts. This keeps the Dagster graph faithful to the domain: FDA/EMA/KEGG/CDE product records can resolve their own product_name/generic_name without waiting for live NCIt/RxNorm lookups, while q_drugs.primary_external_ids still remains source-backed by official vocabulary snapshots.

Rejected alternative: one monolithic canonical/source_drug_identity_claims asset that waits on NCIt/RxNorm before rebuilding brand/name resolution. That couples a useful enrichment path to approval publication and makes an authority lookup a hidden precondition for source-product resolution. Legacy Data Gov did the equivalent coupling through mutable source_drugs FKs and brand_drugs rollups (../bioloupe-data-gov/app/models/source_drug.rb, ../bioloupe-data-gov/app/models/brand_drug.rb, ../bioloupe-data-gov/app/models/drug_approval.rb); greenfield keeps those grains separate.

migration.legacy_drug_baseline schema:

ColumnTypeMeaning
legacy_drug_idbigint primary keypublic.drugs.id.
bioloupe_idtextLegacy stable B... id, nullable.
drug_nametext not nullpublic.drugs.name; rows with blank names are rejected to ops.rejected_records.
inn_nametextpublic.drugs.inn_key.
all_synonymsjsonb not null default '[]'Raw legacy synonym array for reconciliation/review; not direct alias authority.
legacy_statustextLegacy approval/workflow status (status).
legacy_qa_statustextLegacy QA status.
drug_statustextLegacy lifecycle summary retained for reconciliation only; not canonical development truth.
development_phasetextLegacy development summary retained for reconciliation and program-gap analysis only; not an identity key or canonical phase.
drug_typetextLegacy regulatory/product-class facet used by ownership-suggestion filters; staged separately and not forced into modality.
therapeutic_areasjsonb not null default '[]'Legacy therapeutic-area array used by ownership-suggestion and disease/indication handoff; not a canonical drug-identity field.
is_combinationboolean not null default falseLegacy is_combination.
deleted_attimestamptzSoft-delete marker; normally excluded from reconciliation dashboards.
external_idsjsonb not null default '{}'Source-code map recovered from left-joined source_drugs: {ncit, rxnorm, adis, chembl}. Values are source codes, not legacy FK ids.
source_aliasesjsonb not null default '{}'Optional source-row names/synonyms recovered from joined source_drugs, used only as candidate alias evidence.
source_locatorjsonb not null{source_family:'legacy_data_gov', legacy_table:'public.drugs', legacy_pk, bioloupe_id} plus joined locator metadata.
content_hashtext not nullHash over the staged legacy fields used for idempotency/drift.
captured_attimestamptz not null default now()Capture time.

The source asset must reject only row-local failures to ops.rejected_records (source_family='legacy_data_gov', stage='legacy_drug_baseline'). Required source-stage reason codes: legacy_drug_blank_name, legacy_drug_bad_synonyms. Cross-row identity conflicts belong to source-derived resolution/review, where normalized-name and external-id ownership maps exist. Do not create a one-off reject table.

AssetReadsWritesRule
canonical/source_drug_identity_claimsSource-derived drug identity claims from approvals/productscanonical.drugs, canonical.drug_aliases, canonical.drug_brandsMint/link only from approved source policies and verifier/curation receipts; does not depend on NCIt/RxNorm enrichment.
canonical/drug_authority_external_idsOfficial NCIt/RxNorm external-id and alias claims for already source-derived drugscanonical.drug_external_ids, canonical.drug_aliases, resolution.review_queueAttach source-contract-backed external ids and aliases after the base identity exists; collisions route to review rather than overwriting.
canonical/source_drug_combinationscombination/regimen claims, component drug identitiescanonical.drug_combinations, canonical.drug_combination_componentsCombination shells reference component drugs; never collapse components into one substance.
canonical/drug_identity_namescanonical drug/alias/brand tablesresolution.entity_namesRebuild the resolver surface after source-derived canonical rows materialize.

The target/technology spine owns target authority capture, target/mechanism claims, and canonical.drug_target_action; drug identity contributes only the resolved drug endpoint.

Canonical row ids and keys:

  • drug_id is deterministic from the source-authority identity claim, e.g. source family + source id + release-grain policy, so reruns are idempotent without using legacy primary keys.
  • resolved_drug_key is an opaque drug:<slug-or-bioloupe-id> surface. A legacy B... id may be retained only after reconciliation links it to a source-derived identity.
  • source_fact_key points to the source artifact/evidence claim that justified the canonical assertion, not to public.drugs.
  • confidence is a categorical provenance basis such as source_cross_reference, verifier_passed, or reviewed_curation, not a numeric score.

Canonical write policy:

  • upsert canonical.drugs by deterministic source-derived drug_id;
  • never update a canonical row from a legacy migration table;
  • insert aliases with ON CONFLICT DO NOTHING after deterministic normalization;
  • insert external ids with ON CONFLICT (source, external_id) DO NOTHING; if the same source id already maps to a different drug, route the claim to review instead of overwriting;
  • treat duplicate normalized primary names as review unless the duplicate has the same vetted source id map or an approved merge curation decision.
Legacy inputGreenfield use
drugs.idretained only in source_locator.legacy_pk and reconciliation reports.
drugs.bioloupe_idold stable product key to attach to a source-derived resolved_drug_key after match/review.
drugs.namecandidate display-name evidence for matching; never the sole canonical primary-name authority.
drugs.inn_keycandidate alias/INN evidence after it matches a source-derived drug.
drugs.all_synonymscandidate alias evidence after source-derived match or reviewed curation; no direct alias insert.
drugs.drug_status, drugs.statuslegacy workflow context for review queues and continuity dashboards; no direct assertion-state policy.
drugs.development_phase, drugs.drug_typepreserved in migration.legacy_drug_baseline for reconciliation and coverage analysis only; do not overload canonical.drugs.modality or use as ownership/development truth.
drugs.therapeutic_areaspreserved in migration.legacy_drug_baseline for organisation-ownership candidate generation and later disease/indication reconciliation; not written as canonical drug identity.
drugs.is_combination, combination_drug_components, parent_idcandidate combination evidence; canonical combinations require source-derived component drugs or reviewed curation.
joined NCIt/RxNorm source rows codereconciliation key to current NCIt/RxNorm snapshots; canonical external id only when the current source artifact confirms it.
joined Adis/ChEMBL source rows codemapping-only/deprecated traceability after a source-derived drug exists; never a minting source.
joined source row name / synonymscandidate alias evidence after matching; never a source_drugs table.
brand_drugs.name, approval brand_namebrand candidate evidence; canonical brand rows come from approval/product source records and must resolve to a source-derived canonical drug.
Source-derived statecanonical.drugs.assertion_stateUse
active source identity later withdrawn/deprecated by source or curationdeprecatedHidden from publication q_drugs unless explicitly requested.
vetted NCIt/RxNorm/source-artifact identity with verifier/review supportresolvedPublished identity.
source-supported but unresolved/conflicting identityprovisionalSearchable for review, but cannot auto-merge or receive legacy aliases without evidence.

Sources never write directly into canonical.drugs by pretending to be legacy rows:

  1. NCIt/RxNorm source snapshots create vocabulary records and identity/alias/external-id claims.
  2. Resolver links claims to existing source-derived identities by exact source id/name/alias.
  3. Resolver mints only from an exact accepted active-substance claim or reviewed curation when no existing normalized identity matches.
  4. Curation can merge/split/repoint identities through reviewed append-only decisions.
  5. Adis/ChEMBL ids imported from legacy remain mapping-only external ids.

Latest legacy Data Gov runs drug ownership suggestions immediately after therapeutic-area refresh (../bioloupe-data-gov/app/workflows/drugs_workflow.rb) and filters ownerless drugs by drug_type, drug_status, development_phase, therapeutic_areas, plus approval/trial evidence (../bioloupe-data-gov/app/services/ownership_suggestions/drug_sync_service.rb). Greenfield preserves those legacy fields in migration.legacy_drug_baseline only to explain legacy coverage and compare candidate populations. The organisation slice rebuilds ownership candidates from explicit source roles and temporal relationship evidence; legacy lifecycle summaries are not an acceptance basis.

Drug identity still does not own ownership suggestions. 26-slice-organisation-intelligence.md owns temporal asset ownership (canonical.asset_ownerships, q_asset_ownerships) and the reviewed/verified current-owner decision. This spine only supplies the resolved drug key and source-backed drug context.

Blocking checks:

  • no_source_drugs_table — no greenfield schema creates a source_drugs table.
  • no_legacy_drug_canonical_producer — no Dagster asset writes migration.legacy_drug_baseline into canonical drug tables.
  • source_drug_identity_claims_present — source approval/product records produce evidence.drug_identity_claims before canonical promotion.
  • legacy_drug_reconciliation_coverage — staged legacy rows report matched/unmatched/review counts against source-derived canonical identities.
  • source_external_id_coverage — NCIt/RxNorm ids in canonical.drug_external_ids point to current source snapshots, not just legacy joined codes.
  • identity_uniqueness — normalized names remain unique or the claim is routed to review.
  • legacy_reject_dlq — all skipped baseline rows are present in ops.rejected_records, not silently dropped.
  • no_separator_component_auto_link — intervention resolution never splits /, +, comma, semicolon, or and and publishes one arbitrary component as the whole intervention.

Minimal implementation tests:

  • unit test that staged legacy drug rows have no canonical-producing asset;
  • unit test that regulatory active-substance rows create source-derived substance, alias, and brand claims;
  • unit test that multi-substance product rows do not brand-link to one arbitrary component;
  • unit test that NCIt/RxNorm claims enrich existing identities and route missing/colliding identities to review;
  • unit test that a source-supported but unresolved active drug becomes provisional;
  • unit test that ChEMBL/Adis external ids are deprecated/mapping-only;
  • unit test that an external-id collision routes to review and does not overwrite the existing canonical mapping;
  • unit test that "drug A + drug B" resolves only to an existing combination/regimen row or remains unresolved/review, never to just one component;
  • definition/SQL test that no source_drugs relation is created;
  • DB smoke test materializing source-derived drug identity -> q_drugs, with legacy reconciliation remaining sidecar-only.

Distinct grains (do not collapse):

ConceptGrainLegacy evidenceNotes
Active substance / drugone compound at INN leveldrugs (name, inn_key, all_synonyms)canonical identity; combination_drug_components self-join for fixed-dose combinations.
Drug aliasone name variant for a drugdrugs.all_synonyms, brand_drugs.aliasesevery variant must trace to one drug.
External idone source-scoped identifierdrugs.ncit_id/rx_norm_id/adis_id, plus legacy chembl_id when salvagedSource locators, not global identity. NCIt/RxNorm are vetted public reference ids; Adis/ChEMBL are mapping-only legacy/proprietary ids unless future source contracts promote them.
Branded productone marketed brand/trade name linked to a canonical drugbrand_drugs.name, brand_drugs.aliases, and linked approval brand_name valueslegacy brand_drugs is a rollup/cache; product attributes and selected regional approvals stay with approval/product facts.
Combination / regimena set of component drugs as a treatment unitcombination_drug_components, chemo_combinationa combination is not a new molecule.
Intervention componentone arm/protocol intervention with dose/route/scheduletrial_arm_interventions, publication_interventionsresolves to drug, NCIt concept, combination, or unresolved.
Non-drug interventionprocedure/device/radiation/behavioral/placeboNCIt concepts via ncit_concept_idmust not resolve to a drug (issue 50).
Target / mechanism / modalitywhat a drug acts on / how it is builtdrug_target_action, target, technologyidentity and hierarchy are owned by 31-slice-target-technology-identity.md; this spine owns only the drug endpoint of the relation.
  • one canonical drug per compound; alias/external-id/brand variants point to it, never duplicate it;
  • combination/regimen reference component drugs and never replace them;
  • an intervention component resolves to exactly one winning identity (drug xor NCIt concept) or stays unresolved — mirrors the legacy single-FK rule (publication_interventions.drug_id or ncit_concept_id, never both; ../bioloupe-data-gov/CLAUDE.md);
  • external ids (ncit/rx_norm, plus mapping-only adis/deprecated chembl mappings when salvaged) are source-scoped and not interchangeable;
  • generic class terms (“statins”, “PD-1 inhibitor”, “CAR-T”) must not auto-resolve to a specific product (drug.rb MATCH_STOPLIST / generic_class_term?).

This spine owns the canonical resolver that trials/approvals/publications/news/SOC consume. It is exact-first, then LLM-verified fuzzy, with discrete provenance basis categories — never a bare trigram auto-accept (../bioloupe-data-gov/app/services/entity_matcher.rb, ../bioloupe-data-gov/CLAUDE.md “never bare flexifind”).

Subject kindSource term examplesCandidate target typesPublishable basisHigh-impact?
intervention_drug_linkpembrolizumab, KEYTRUDA, MK-3475, imatinib mesylatecanonical Drug (incl. broader salt/formulation match)exact synonym, source cross-reference, context-verified, verifier-passed fuzzy/LLM, reviewed curation overrideyes
intervention_concept_linksurgery, EBRT, PET/CT, placebo, exerciseNCIt non-drug conceptNCIt exact/semantic-scoped, verifier-passed LLM, reviewed overrideyes
brand_product_linkKeytruda, Tagrissocanonical BrandDrug -> Drugnormalized brand/alias exact, source cross-reference, reviewed overrideyes
drug_alias_promotionterm confirmed = drug across many sponsorsnew alias on canonical Drugoffline reviewed promotion only (entity_matching promote_synonyms)yes
combination_membership”pembrolizumab + chemo”combination -> component drugssource-exact component listing or verifier/reviewed overridemedium/high

Resolver guards that must survive into greenfield (all from drug_matching_service.rb):

  • formulation guard — liposomal/pegylated/etc. must not collapse onto the plain molecule;
  • radioisotope guard — a different isotope is a different agent (Y-90 ≠ Ga-68);
  • sponsor/context disambiguation — owner organisation breaks ties only when the name cannot identify the product;
  • broader match — salt/dose/formulation variant may match the base drug as match_type=broader;
  • whole-intervention guard — the resolver resolves the full intervention string as one term and does not split on separators to choose a single component. Latest legacy Data Gov removed that fallback in ../bioloupe-data-gov/app/services/intervention_resolver.rb because it corrupted dosing units and collapsed fixed-dose combinations to arbitrary constituents. A combination/regimen either resolves to a combination/regimen identity or stays unresolved for review.

A source-grounded assertion that a name denotes a specific substance/product, with source locator and extractor/model version. Promoted to canonical identity only after resolution/verifier acceptance.

A name-variant-for-drug claim (synonym/brand/code) with source authority; feeds alias resolution, not silent merge.

A source-scoped id mapping (ncit/rx_norm, plus mapping-only salvaged ids such as adis and deprecated chembl) with the source release/version and liveness state it came from.

A claim that a trial/publication intervention string resolves to a drug or NCIt concept, with basis category and verifier state.

Reviewed overrides correct identity after pipeline inference; they are not the default path (separate from LLM extraction per 05-agentic-curation-and-overrides.md).

SubjectWhy high-impactExample
drug_merge_overridewrong split fragments a compound’s evidencemerge two drugs rows for the same compound.
drug_split_overridewrong merge contaminates two compoundssplit an over-merged drug.
intervention_identity_overridewrong link poisons every downstream program viewforce “MK-3475” -> pembrolizumab, or reject a non-drug from a drug match.
brand_to_drug_overridewrong brand link corrupts approvals/competitive landscapere-point a brand to the correct drug.
alias_promotion_overridebad alias becomes a permanent false matchapprove/revoke a synonym promotion.

Legacy preserves manual curation against automated overwrite via Lockable/lockable_attributes and durable manual matches (entity_match_caches.provenance='manual'); greenfield expresses this as reviewed overlays, not Rails locks.

Durable nouns (group canonical, per 04-dagster-asset-graph.md):

  • canonical.drugs — one compound (INN-level) with identity assertion state; no global development phase/status.
  • canonical.drug_aliases — one alias per drug.
  • canonical.drug_external_ids — source-scoped id mappings.
  • canonical.drug_brands — branded/marketed products linked to a drug.
  • canonical.drug_combinations / canonical.drug_combination_components — combination/regimen membership.
  • resolution.drug_intervention_candidates — proposed intervention->identity links with basis + verifier state.

Target/mechanism/technology identity is owned by 31-slice-target-technology-identity.md; this spine publishes resolved drug keys used by those relations.

All read-only SQL models in the query schema; visible in generated q_catalog, provenance-required rows resolve through q_provenance.

ModelGrainAgent question
q_drugsone canonical drugdrug search by name/synonym/INN; identity assertion state, modality summary, and key external ids. Contextual development state is joined through q_development_programs.
q_drug_aliasesone aliasname/synonym lookup and identity trace.
q_drug_external_idsone source-scoped idcross-reference to NCIt/RxNorm plus mapping-only Adis/ChEMBL traceability; only source-contract-backed ids may be promoted into q_drugs.primary_external_ids.
q_drug_brandsone branded product with source approval/product contextbrand -> drug resolution, marketed-name search, and compact FDA/EMA/KEGG/CDE source-product backing.
q_drug_regimensone combination/regimen + componentswhat a combination contains.

Drug-target relations are owned and published by the target/technology spine as q_drug_target_actions; the drug identity family does not expose a second transitional copy.

resolved_drug_key(s) used across 21-35 is defined here as a stable key into q_drugs (e.g. drug:<slug>). Downstream slices reference this key; they do not mint their own drug identity.

FieldRequired content
PurposeCanonical drug discovery and the resolution target for every drug mention.
GrainOne canonical compound.
Stable row keyresolved_drug_key (drug:<slug>), used by q_provenance.q_row_key.
Required columnsresolved_drug_key, drug_name, inn_name, synonyms (text[]), modality, is_combination, primary_external_ids (jsonb: vetted live ids, currently ncit/rxnorm), legacy_external_ids (jsonb: mapping-only ids such as adis/chembl), assertion_state, provenance_key.
Search affordancestrigram/full-text on drug_name/synonyms; GIN on synonyms; btree on assertion_state/modality.
Provenance ruleprovenance_grain=bundle; identity bundle expands to source vocabulary artifact, resolution/verifier receipt, canonical assertion, and curation overlay only when an override exists.
Refreshmaterialized view/table; full refresh, targeted refresh on reviewed identity overrides.
Blocking checksschema_contract, provenance_coverage, identity-uniqueness (one canonical row per compound).
  • every resolved_drug_key in any q_* model exists in q_drugs (enforced corpus-wide by the referential_integrity gate in 08-contracts-and-quality.md);
  • no two canonical drugs share a normalized identity (uniqueness like drug.rb normalize_name);
  • an intervention component never resolves to both a drug and a non-drug concept;
  • generic class terms do not resolve to specific products;
  • no q_drugs field is documented or consumed as a global clinical phase, clinical activity, regulatory, or “highest phase ever” state;
  • q_drugs family schema matches generated q_catalog; provenance-required rows resolve through q_provenance.

Concepts Promoted to 20-pipeline-families.md

Section titled “Concepts Promoted to 20-pipeline-families.md”
  • Drug identity is a foundational spine consumed by all therapeutic slices; the cross-slice “Drug / intervention” concept resolves to this spine’s resolved_drug_key.
  • Substance vs brand vs combination vs intervention vs non-drug concept are distinct grains.
  • resolved_drug_key is the deterministic drug:<normalized-slug> key for the current contract; changing it requires an explicit key migration and compatibility plan.
  • Combination/regimen identity is first-class through q_drug_regimens; downstream slices must not encode regimen membership in free text.
  • Clinical phase, activity, regulatory state, and historical maximum are contextual development-program facts owned by 33-slice-development-landscapes.md, never drug identity columns.