32 Slice Endpoint Adverse Event Identity

Pipeline Slice: Endpoint / Adverse-Event Identity Spine

Status: SHARP CORE VERTICAL. Versioned endpoint and NCI CTCAE capture now runs source -> normalized -> evidence -> canonical -> four provenance-backed q_* models. The remaining work is coverage expansion and downstream mention resolution, not an unowned source population gap. Denoising state: sharp (noise -> rough -> sharp -> converged) Updated: 2026-07-23

Purpose

This spine owns the shared identities for:

• what was measured: clinical endpoints such as OS, PFS, ORR, and DoR;
• what harm was observed: standardized adverse-event terms such as anemia or pneumonitis;
• how aggregate severity was reported: explicit CTCAE grade bands.

Trials, publications, and standard-of-care evidence resolve into these identities. They continue to own their result values, groups, denominators, seriousness fields, and source text.

Legacy Data Gov established the useful product intent through endpoints, endpoint_groups, and adverse_events. Greenfield preserves that intent while correcting two legacy accidents:

1. an adverse-event term does not inherently belong to one endpoint;
2. regulatory seriousness is not a CTCAE severity grade.

The second distinction is explicit in FDA/ICH safety guidance: severe describes intensity, while serious is determined by outcomes or required action. CTCAE independently defines Grades 1-5 as term-specific severity. Sources: FDA serious adverse events, ICH E2A guidance, and NCI CTCAE v6.

Decided Shape

flowchart LR
  EndpointSource["Versioned internal endpoint catalog"]
  CtcaeSource["Official NCI CTCAE v6 workbook"]
  Artifacts["Immutable source artifacts + active capture pointers"]
  Normalized["Typed source-faithful records"]
  Evidence["Endpoint identity + AE term claims"]
  Canonical["Endpoints + groups + AE terms + grade bands"]
  Names["Shared exact-name projection"]
  Query["q_endpoints / q_endpoint_groups / q_adverse_event_terms / q_ae_grade_vocabulary"]
  Demand["Trial / publication / SOC mentions"]
  Review["Verifier or reviewed curation"]

  EndpointSource --> Artifacts
  CtcaeSource --> Artifacts
  Artifacts --> Normalized --> Evidence --> Canonical --> Names
  Canonical --> Query
  Demand --> Names
  Names --> Review
  Review --> Canonical

Authoritative catalog rows do not need an artificial fuzzy-resolution asset before publication. They create source-grounded identity claims directly. Fuzzy downstream mentions remain reviewable signals and never auto-link.

Source Contracts

Source	Captured artifact	Completeness	Role
Bioloupe endpoint catalog	checked-in versioned JSON with definitions, synonyms, taxonomy, and references	complete for its declared release; currently 15 common oncology endpoints	authoritative initial endpoint identity catalog
NCI CTCAE v6.0 / MedDRA 28.0	exact official XLSX bytes plus parsed clean-copy rows	live mode captures the full workbook unless max_live_records explicitly requests a bounded smoke	authoritative CTCAE term names, LLT codes, SOCs, definitions, and five term-specific grade slots
CTCAE fixture	checked-in representative JSON subset	deliberately incomplete; 9 rows	deterministic offline development and CI
Result-bearing slices	source endpoint/AE phrases	source-specific	identity demand only; they do not mint endpoint or AE identity
Independent MedDRA product	not captured	licensing decision pending	optional future PT/hierarchy enrichment; CTCAE-carried LLT fields remain source-faithful

The official CTCAE download is owned by NCI: CTCAE and AE reporting.

Fixture/live seam

ClinicalVocabularySourceResource owns:

• mode = fixture | live;
• the official CTCAE workbook URL;
• max_live_records, which applies only to bounded live smoke;
• the nested shared source_http resource.

The Dagster source asset never branches on an environment flag. Default definitions stay offline. Live mode is launch-time run configuration.

Immutable artifacts and current selection

sources_clinical_vocabulary.catalog_artifacts stores content-addressed exact bytes and never updates them. active_capture_artifacts is a small operational pointer to the exact endpoint and CTCAE artifacts selected by the latest successful capture.

This separation is intentional:

• artifact history is immutable and replayable;
• switching fixture -> live -> fixture changes current selection correctly;
• a bounded smoke cannot become current merely because its artifact timestamp is newest;
• downstream canonical publication never guesses current state from fetched_at.

Domain Grains

Concept	Grain	Stable key / fields	Ownership
Endpoint	one canonical clinical endpoint	canonical_endpoint_key, name, abbreviation, definition, references	this spine
Endpoint group	one taxonomy root or subgroup	canonical_group_key, parent group	this spine
Endpoint alias	one phrase resolving to one endpoint	normalized phrase + alias kind	this spine
Adverse-event term	one CTCAE-compatible term identity	resolved_ae_term_key, CTCAE release, MedDRA LLT code/SOC, definition	this spine
Term-specific grade definitions	five nullable CTCAE descriptions on one AE term	grade 1 through 5 array	this spine
Aggregate grade band	one closed range over CTCAE grades 1-5	category + inclusive minimum/maximum	this spine
Seriousness	source/regulatory classification of an observed event	serious/non-serious and outcome criteria	owning result slice, not this vocabulary
Safety observation	one source result group + AE term + count/denominator	trial/publication-specific key	trials/publications, not this spine

Closed grade bands

Category	Minimum	Maximum
all_grade	1	5
grade_1_2	1	2
grade_3_4	3	4
grade_gte3	3	5
grade_5_fatal	5	5

sae is intentionally absent. Unknown grade phrases remain null. A caller must model seriousness separately rather than coercing serious into a grade.

Invariants

• endpoint identity is separate from endpoint result values;
• adverse-event term identity is separate from endpoint identity, observation grade, and seriousness;
• every published endpoint and AE term traces to the active immutable source artifact and its deterministic extraction claim;
• CTCAE grade definitions remain term-specific; unavailable grades remain null array elements;
• every grade band has 1 <= minimum_grade <= maximum_grade <= 5;
• exact names and aliases may auto-link; fuzzy matches route to review;
• generic descriptors such as primary endpoint are rejected as identities;
• ambiguous OR never silently becomes ORR;
• retired catalog rows become deprecated instead of leaking into current q_* views;
• source history remains append-only even when current canonical publication changes.

Resolution and Curation

Subject	Accepted automatic basis	Otherwise
endpoint_link	exact canonical name, abbreviation, or synonym	verifier/review; ambiguous and fuzzy mentions do not auto-link
adverse_event_term_link	exact standardized name, source code, or reviewed synonym	verifier/review
ae_grade_assignment	deterministic phrase mapping to one closed grade band	null plus review when material
ae_seriousness_assignment	explicit source seriousness field or outcome criteria	remains in the owning observation slice

Useful reviewed curation subjects are endpoint remap/merge/split, AE-term remap, and grade-band correction. Curation is an append-only overlay; it does not mutate source artifacts or hide inside catalog extraction.

Implemented Asset Chain

Layer	Asset	Durable output
sources	sources/clinical_vocabulary/catalog_snapshots	exact artifact bytes, typed rows, active capture pointers, source-artifact receipts
normalized	normalized/clinical_vocabulary_records	endpoint and CTCAE source-faithful records
evidence	evidence/clinical_vocabulary_claims	deterministic endpoint identity and AE-term claims plus extraction runs
canonical	canonical/endpoint_adverse_event_tables	canonical DDL and five grade bands
canonical	canonical/clinical_vocabulary_catalog	selected endpoints, groups, aliases, AE terms, provenance links
canonical	canonical/endpoint_adverse_event_entity_names	this spine’s exact-name/alias projection
query	four q_* assets	stable SQL-only consumer surface

The old resolution/endpoint_adverse_event_candidates definition was deleted because it only created an empty buffer and performed no resolution. The remaining resolution module contains tested policy functions; a Dagster asset should return only when real downstream mention demand exists.

Query Contracts

Model	Grain	Key	Purpose
q_endpoints	one published endpoint	canonical_endpoint_key	names, definitions, aliases, taxonomy, references
q_endpoint_groups	one published taxonomy node	canonical_group_key	hierarchical endpoint navigation
q_adverse_event_terms	one published AE term	resolved_ae_term_key	CTCAE/MedDRA-carried term identity and term-specific grade definitions
q_ae_grade_vocabulary	one aggregate grade band	grade_category	closed min/max grade ranges; no seriousness conflation

All four contracts require schema_contract, provenance_coverage, and positive-row checks. Published rows carry a provenance_key that resolves through the shared provenance ledger.

Verification

The slice’s deterministic suite proves:

• fixture and mocked-live resource behavior without external calls;
• exact workbook parsing and preservation of five grade slots;
• explicit fixture incompleteness versus complete live semantics;
• all source/normalized/evidence/canonical/query asset keys;
• absence of the deleted no-op resolution asset;
• contract vocabulary and grade bounds;
• idempotent DDL upgrades from an older local schema;
• real Postgres source-to-query materialization;
• a second identical materialization preserves row and provenance counts.

No paid LLM call is required to build or test this spine.

Remaining Scope

1. Expand the reviewed endpoint catalog beyond the initial high-value set while keeping each definition and synonym source-referenced.
2. Add demand-driven trial/publication/SOC endpoint and AE-term link assets when those slices publish real unresolved mention queues.
3. Decide whether licensed MedDRA PT/hierarchy enrichment adds enough value beyond CTCAE-carried LLT identity.
4. Add corpus-level referential checks once downstream slices emit resolved_ae_term_key.

These are explicit coverage/integration frontiers. The core vocabulary capture and publication path is implemented.