05 Agentic Curation And Overrides

Agentic Curation and Overrides

Status: DECIDED CORE / PARTIAL OPERATIONS. The curation boundary, event store, effective overlay, and owner-routed Dagster sensor are implemented. Seven canonical owner routes are definition-load validated. Row/partition targeting, all documented slice subject kinds, conflict-safe publication for the seven routed families, and a reviewer product remain incomplete. The generic generation mechanism is implemented and used by clinical trials, but no current curation owner route targets that protected family. Live/consumer state: not production-proven; no consumer cutover evidence. Denoising state: sharp (noise -> rough -> sharp -> converged) Updated: 2026-07-25

Purpose

Specify how facts are corrected, overridden, suppressed, or reopened after the normal source -> evidence -> resolution -> canonical pipeline has produced candidate state. Curation is a separate change-control layer: curator agents usually discover and propose changes, humans review high-impact changes, and Dagster observes the reviewed decision log to rematerialize affected assets. Dagster never owns the decision; it reacts to it.

This is intentionally distinct from 06-llm-extraction-and-verification.md. LLM extraction is a normal pipeline capability for structuring, enriching, classifying, and connecting source data. Curation is the editorial/review mechanism for changing the effective truth when the pipeline output is wrong, ambiguous, disputed, or intentionally overridden.

Boundary: curation is not extraction

Capability	Lives in	Writes	Meaning
Deterministic parsing	sources.*, normalized.*, evidence.*	source-faithful records / evidence claims	Convert source records into structured, replayable facts.
LLM structuring/enrichment	evidence.*, resolution.* via shared lib/llm	evidence claims, classifications, candidates, verifier artifacts	Normal pipeline work: extract, normalize, classify, connect, and verify source-grounded data.
Entity resolution acceptance	resolution.* and canonical assets	accepted links with basis + verifier state	Decide whether a source term can publish as a canonical link under policy.
Agentic curation / override	curation.*	reviewed decision events + effective override state	Correct, suppress, supersede, or reopen effective facts after pipeline interpretation.

Rules:

• LLM outputs do not become curation decisions merely because a model produced them.
• A verifier/judge output may block or support canonical publication, but it is still pipeline verification, not curation.
• Curation decisions may reference LLM/evidence/resolution artifacts as evidence, but the decision itself is written by a curator agent or human reviewer.
• Human review is required for high-impact curation changes unless a slice explicitly marks the subject kind as low-risk and auto-reviewable.

Roles

• Curator agents — inspect q_*, q_provenance, conflicts, failed checks, and external evidence; propose correction/override/suppression events with rationale and evidence refs.
• Human reviewers — approve, reject, or revise high-impact curation proposals; human-reviewed decisions outrank agent-only decisions.
• Optional reviewer agents — pre-screen proposals, find counterevidence, and prepare review packets; they do not replace human review for high-impact changes.
• Dagster observer — watches publishable curation state and resolves each implemented subject_kind to one definition-load-validated canonical owner job. Owners are currently rematerialized at slice scope because their assets are not partitioned by curation subject.

Decision Store (system of record)

Append-only event log in the new DB; the effective curated state is derived, never edited in place.

curation.decision_events       -- append-only proposals/reviews/overrides
curation.effective_overrides   -- derived effective curation state; never directly edited
curation.conflicts             -- derived unresolved curation-conflict queue
curation.review_queue          -- optional view over pending high-impact proposals

curation.effective_overrides, curation.conflicts, and curation.review_queue are projections over curation.decision_events owned by the curation store. Dagster declares/observes them as external assets in defs/curation/* but does not materialize, mutate, approve, or reject decisions.

Minimum decision_events row:

decision_id            opaque append-only caller identifier
event_sequence         database-owned monotonic observation cursor
idempotency_key        deterministic actor+subject+payload key; unique (dedupes retries)
subject_kind           e.g. drug_identity_override, trial_scope_override, indication_text_override
subject_key            stable entity/assertion/query-fact key (resolve-then-write)
decision_type          propose | approve | reject | override | suppress | unsuppress | supersede | reopen
decided_value          asserted replacement/effective value (nullable for reject/suppress)
prior_decision_id      supersession pointer (nullable)
actor_kind             curator_agent | reviewer_agent | human
actor_id               agent name+version or human id
proposal_source        conflict_queue | failed_check | agent_audit | human_request | external_signal
review_state           not_required | pending_human_review | human_approved | human_rejected | superseded
reviewed_by            human id or review authority (nullable)
reviewed_at            timestamp (nullable)
review_note            human/reviewer explanation (nullable)
rationale              required justification from proposer/actor
evidence_refs          pointers to source artifacts, evidence claims, q_provenance receipts, URLs, or notes
provenance_basis       manual_review | source_exact | cross_reference | contradiction | correction | policy_override
status                 active | superseded | rejected
created_at             timestamp
decided_at             timestamp

Properties: append-only, idempotent, provenance-complete, identity-stable ((subject_kind, subject_key) is the typed stable identity so re-extraction cannot orphan an applied override), and review-aware (high-impact unreviewed proposals do not publish).

Dagster Integration (decided)

• The decision store is modelled as an external/observable upstream, not something Dagster materializes — matching Dagster’s external-assets pattern for data produced outside its control. (https://docs.dagster.io/guides/build/assets/external-assets)
• A sensor detects new decisions that are human-approved or explicitly low-risk and auto-reviewable. It cursors on database-owned event_sequence, de-duplicates by typed subject, and emits deterministic run_key values. Caller decision ids are never treated as clocks. (https://docs.dagster.io/guides/automate/sensors)
• An explicit subject-kind registry routes implemented overlays to seven jobs: news, publications, SOC, organisation identity, organisation deals, epidemiology, and commercial. Unknown or documented-but-unimplemented kinds fail the sensor tick without advancing the cursor. A successful no-op is forbidden.
• Each job selects the effective overlay/conflicts projection, one canonical asset that actually consumes the overlay, and Dagster-derived descendants. This is owner-slice targeting today, not row/partition targeting.
• Curator/reviewer systems may also push reviewed decisions as runless events via the external-assets REST API instead of being polled. (https://docs.dagster.io/api/dagster/external-assets-rest)

sequenceDiagram
  autonumber
  participant Curator as Curator agent
  participant Human as Human reviewer
  participant Store as curation.decision_events
  participant Sensor as Dagster curation sensor
  participant Checks as Blocking checks
  participant Query as q_* read models

  Curator->>Store: propose override/suppression + evidence_refs + idempotency_key
  Human->>Store: approve/reject/revise high-impact proposal
  Sensor->>Store: poll active reviewed decisions after cursor
  Sensor->>Checks: launch owner-routed job with deterministic run_key
  Checks-->>Query: validate rebuilt owner/query descendants

Precedence and Conflict Resolution (decided)

Deterministic merge -> one effective curation overlay per (subject_kind, subject_key):

1. Review tier: human_approved > not_required agent decision > pending_human_review (not publishable for high-impact) > human_rejected (not effective).
2. Actor tier within review tier: human > curator_agent > reviewer_agent.
3. Basis tier: manual_review > source_exact > cross_reference > contradiction > correction > policy_override, then most recent decided_at.
4. Supersession: a row with prior_decision_id replaces its ancestor; only status=active participate.
5. Unresolvable conflict target: emit to curation.conflicts; each routed in-place family still needs a candidate-generation gate that keeps the previous served generation for high-impact conflicts. The current projection records the tie and uses decision_id as a deterministic final ordering key, but it does not provide last-known-good protection for those seven families.

Publication Gate (target and current boundary)

Curation affects canonical/q_* publication only when the effective curation overlay is reviewed enough for the subject kind:

1. Implemented review gate — curation.effective_overrides admits human-approved decisions and not_required decisions only for subject kinds explicitly marked auto-reviewable.
2. Implemented in-run checks — @asset_check(blocking=True) can stop selected downstream assets in an owner job. (https://docs.dagster.io/guides/test/asset-checks)
3. Implemented mechanism, partial adoption — immutable candidate generations, pre-promotion checks, atomic stable-view movement, and rollback exist for the complete clinical-trials query family. The seven current curation owner routes still publish their query descendants in place, so a failed post-materialization check cannot prove their previous rows remain served.
4. Implemented trigger, incomplete publication gate — the curation sensor launches one deterministic owner job after reviewed events. Each routed family still needs candidate validation inside the same transaction as promotion (or an explicitly accepted maintenance recovery contract); changing to declarative automation alone would not create that guarantee.

Rejected, superseded, pending, and unreviewed high-impact decisions do not enter the effective overlay. review_state='not_required' is publishable only for subject kinds explicitly listed as low-risk and auto-reviewable, and it must remain auditable in q_provenance.

Expected Tests

• duplicate curator-agent retry inserts one decision only (idempotency_key);
• pending high-impact proposal does not affect canonical/query assets;
• human-approved override changes the effective overlay and launches exactly one owner job;
• two subject kinds sharing one subject_key remain independent;
• arbitrary decision ids cannot reorder or skip events because the sensor uses event_sequence;
• an unimplemented subject kind fails closed without advancing the cursor;
• human rejection leaves the previous effective assertion unchanged;
• supersession chain yields exactly one active effective override;
• human-approved override outranks agent-only low-risk decision;
• sensor cursor + run_key prevent duplicate downstream runs;
• replaying the decision log rebuilds curation.effective_overrides deterministically.

Still required for completion: one DB-backed correction/reversal proof per routed family, explicit routes plus actual overlay application for every high-impact subject kind claimed by slice docs, partition-aware targeting where scale warrants it, conflict blocking, reviewer operations, and atomic last-known-good publication.

Slice Policy Extension Points

• Each owning slice declares which subject kinds are high impact and therefore require human approval. Undeclared subject kinds default to high impact.
• A reviewer agent may prioritize or annotate an expensive queue, but its output is another reviewable receipt. It cannot replace the required human decision for high-impact subjects.