34 Slice Epidemiology

Pipeline Slice: Epidemiology and Disease Burden

Status: CONVERGED CORE VERTICAL. Fixture/live CDC capture, source-faithful normalization, exact disease resolution, canonical observations and series, reviewed summaries, query models, checks, and database acceptance tests are implemented. Denoising state: converged (noise -> rough -> sharp -> converged) Updated: 2026-07-23

Purpose

Publish source-grounded disease-burden observations without collapsing geography, population, period, denominator, method, or uncertainty into one convenient number.

This slice answers:

How common or severe is this disease, in which population and place, during what period, measured how, and according to which source?

It supports disease profiles and downstream forecasting inputs. It does not own forecast assumptions or projections.

Boundary Decisions

An epidemiology value is an observation in context, not an attribute of disease identity.
Counts, rates, percentages, survival estimates, and projected values are different statistic kinds.
Incidence, prevalence, mortality, survival, and population are different metrics with explicit definitions.
Geography, population, sex, age, race/ethnicity, disease segment, and period are part of comparability.
Crude and age-adjusted rates never silently mix; standard population and denominator remain visible.
Observed, estimated, and projected values remain distinguishable.
A preferred/current summary is a versioned evidence-selection decision over observations, not a rewrite of source truth.

Legacy Intent and Correction

Data Gov exposed disease statistics for disease overview and forecasting. The useful intent was to make disease burden available beside treatment and development intelligence. The legacy shape often reduced a statistic to a selected value or display string, which obscured comparability and source method.

Greenfield keeps the useful product capability while replacing “one disease, one number” with explicit observation and series grains.

Source Inventory

Source family	Authority / use	Capture unit
CDC United States Cancer Statistics	US incidence, mortality, population-stratified statistics and methods	versioned table/download/API response with release metadata
NCI SEER	cancer incidence, survival, prevalence, mortality, and statistical products	versioned release/table/export artifact
national cancer registries and official statistics agencies	jurisdiction-specific disease burden	exact release/table/document artifact
peer-reviewed publication	evidence when an official statistical source does not cover the needed disease/context	exact publication/table/figure with population and methods locator
reviewed licensed source	optional source under an explicit license and retention contract	immutable licensed artifact with access policy metadata

Primary references:

CDC USCS: https://www.cdc.gov/cancer/data/index.html
CDC incidence and death-rate methods: https://www.cdc.gov/united-states-cancer-statistics/technical-notes/incidence-death-rates.html
NCI SEER data and software: https://seer.cancer.gov/data-software/

Search snippets, unsourced web summaries, and legacy selected values are not source authority.

Domain Model

Noun	Grain	Meaning
Epidemiology Metric Definition	one metric/version	Defines incidence, prevalence, mortality, survival, diagnosed population, or another measure and its required dimensions/units.
Epidemiology Observation	one source-reported statistic in one complete context	The immutable normalized fact, including metric, statistic kind, disease, geography, population, period, method, value, unit, and uncertainty.
Epidemiology Series	one comparable sequence definition	Groups observations only when metric, disease concept, geography level, population strata, statistic basis, unit, denominator, and adjustment method are compatible.
Preferred Series Selection	one reviewed policy decision for a use case	Selects a source/series for disease overview or forecasting input and records priority, rationale, validity, and alternatives.

Observation dimensions

Every observation supports these fields when applicable:

resolved_disease_key, optional segment_key, and source disease term/code;
metric definition and statistic kind;
geography key/type and source geography label;
population sex, age range, race/ethnicity, and other source-backed strata;
period start/end, period kind, and source release;
value, lower/upper bounds, confidence level, standard error, and sample/count basis;
unit, denominator, crude/adjusted basis, standard population, and method;
observed/estimated/projected basis;
source artifact, exact table/cell/row/figure locator, observed time, assertion state, and provenance key.

Null means the source did not provide or the field is not applicable. Null must not be converted to “all populations,” zero, or a default period.

Identity and resolution

disease and segment keys come from 30-slice-disease-biomarker-identity.md;
geography uses a source-scoped code plus a canonical geography key when one is available;
source metric labels resolve to versioned metric definitions, not free-text column names;
age/sex/race strata retain source values even when a normalized facet is available;
ambiguous disease or population mapping is quarantined or reviewed, never guessed from a nearby category.

Asset Flow

official release/document
  -> immutable source artifact
  -> source-faithful statistical rows
  -> metric/disease/geography resolution
  -> epidemiology observations
  -> comparable series
  -> optional reviewed preferred-series projection
  -> q_epidemiology_*

Canonical assets:

canonical.epidemiology_metric_definitions;
canonical.epidemiology_observations;
canonical.epidemiology_series;
reviewed preferred-series decisions remain curation overlays with provenance.

Query Models

`q_epidemiology_observations`

Contract field	Required meaning
Purpose	Full-fidelity disease-burden facts for filtering, comparison, and downstream calculation.
Grain	One source-reported statistic in one complete context.
Stable row key	`epidemiology_observation_key`.
Required columns	observation/metric/series keys; disease/segment; geography; population strata; period; statistic/value/bounds/unit/denominator; adjustment/method; observed-estimated-projected basis; source release; assertion state; provenance key.
Provenance	field/bundle provenance to the exact source table/cell/row/figure and normalization receipts.
Search/indexes	disease + metric + geography + period; population facets; source/release; observed basis.

`q_epidemiology_series`

One row per comparable series definition with first/latest period, observation count, metric and dimension signature, source family, method, unit, denominator, release cadence, coverage gaps, and provenance key. It lets consumers compare like with like before reading values.

`q_disease_epidemiology_summary`

One row per disease + geography + population context + summary profile. It exposes selected latest incidence/prevalence/mortality/survival observations only when a versioned preferred-series policy exists.

Every selected value carries its observation key, selection policy/revision, alternatives count, selected_as_of, and provenance key. The summary never strips units or methods and never becomes the source of truth for observations.

Deterministic and Inference Boundary

Deterministic parsers own structured statistical files, type/unit parsing, period normalization, series signatures, and consistency checks.

Inference is allowed only for difficult document/table interpretation where structured data is unavailable. It must return exact locators, preserve source labels, pass schema/unit/domain validation, and enter the normal inference acceptance lifecycle. Inference must not estimate a missing value, infer a denominator, or choose a preferred source by intuition.

Curation Subjects

disease/segment mapping correction;
metric-definition mapping correction;
geography/population-stratum mapping correction;
preferred-series selection for a named use case;
suppression of a source observation proven erroneous or superseded;
explicit equivalence decision between source series.

All selections are append-only, reviewable, and valid for a declared purpose and time range.

Blocking Checks

shared schema_contract, referential_integrity, source_locator, source_authority, and provenance_coverage gates;
epidemiology_observation_semantics, a slice-owned gate suite that verifies:
- complete metric/statistic/unit context;
- no count/rate/percentage coercion;
- no crude/age-adjusted, denominator, or standard-population mixing;
- valid period ordering and non-negative values where required by the metric;
- uncertainty bounds contain the point estimate when present;
- every series shares its declared comparability signature;
- preferred summaries resolve to active observations and an active selection policy;
- projected values never appear as observed values.

Acceptance Scenarios

Scenario	Expected result
crude and age-adjusted incidence for the same disease/year	separate observations and series
US count and rate for the same period	separate statistic kinds; no arithmetic substitution
source reports a five-year period	period remains five-year; it is not assigned to the final year
overall and biomarker-positive populations	separate contexts; segment mapping is explicit
later source release restates an earlier value	both observations retained; supersession/release relation visible
disease overview needs one current value	selected summary points to the chosen observation and selection policy; alternatives remain queryable
forecasting requests prevalence for an uncovered geography	readiness reports missing input; this slice does not fabricate a projection

Forecasting Boundary

This slice owns sourced epidemiology observations and comparable series. 28-slice-forecasting-tpp.md may expose readiness over them. Downstream forecasting owns extrapolation, scenario assumptions, patient-flow logic, and forecast outputs.

Implementation Map

Responsibility	Implementation owner
CDC fixture/live capture and paging	`lib/sources/epidemiology/` and `defs/sources/epidemiology/statistics.py`
Source snapshot storage	`sql/sources/epidemiology/`
Normalize, evidence, and exact resolution	`defs/normalized/epidemiology.py`, `defs/evidence/epidemiology.py`, `defs/resolution/epidemiology.py`
Canonical observations, comparable series, reviewed summaries	`defs/canonical/epidemiology.py` and `sql/canonical/epidemiology/`
Agent-facing read models and gates	`defs/query/epidemiology.py`, `sql/query/epidemiology/`, and `lib/checks/contracts_schema/epidemiology__*.yaml`
Fixture and Postgres acceptance tests	`tests/test_slice_epidemiology.py`

The initial operational profile is CDC Data Query System malignant-neoplasm mortality. It proves the full source-to-query contract without claiming that one CDC dataset covers incidence, prevalence, survival, SEER, or other national registries. New datasets add explicit normalization profiles behind the same observation contract.

The checked-in release contains three 2020 US mortality observations for the NCIt-anchored C9305 Malignant Neoplasm concept: all-ages age-adjusted, all-ages crude, and age-adjusted All races, Hispanic. They become three observations and three comparability series because adjustment and population strata are part of the signature. All retain bounds, standard error, period, geography, source artifact, and locator. No disease-summary row is published in the clean fixture because there is no reviewed preferred-series selection. P09 verifies these distinctions and the absence of an invented summary.

Fixture mode is the deterministic default. A bounded, free live smoke run is selected at launch:

resources:
  epidemiology_source:
    config:
      mode: live
      max_live_records: 10

Omit max_live_records for a complete selected-dataset capture.