Forecasting Cross Repo Contract

Cross-repo forecasting API contract test

Producer: bioloupe-data-gov (this repo) · branch feat/data-collection-forecasting Consumer: bioloupe-forecasting — a separate React/TypeScript app (sibling checkout ../bioloupe-forecasting) · branch fixes/client Key files: producer guard test/integration/api/forecasting/contract_test.rb · consumer check bioloupe-forecasting/src/test/integration/api-contract.test.ts · fixtures bioloupe-forecasting/src/test/fixtures/api/*.json

The forecasting tool is two applications talking over HTTP: data-gov produces JSON, and the separate bioloupe-forecasting app consumes it and hard-validates every response with Zod. This contract test pins the exact JSON shape data-gov emits, so a serializer change trips a test at the source instead of breaking the consumer’s runtime validation in production.

Contents

• Why this exists (post-mortem)
• Two-layer architecture
• Covered surface
• Refresh runbook
• Limitations (honest)
• Adding a new surface later

Why this exists (post-mortem)

A disease_id cleanup on the public statistics serializer dropped two fields (disease_id, disease_name) from GET /api/forecasting/statistics. The consumer’s Zod schema still required both as .nullable() — which permits null but requires the key to be present — so every row failed safeParse, the app threw “Invalid statistics data received from server”, and it would not load. The fix was consumer-side: the dead fields were removed (bioloupe-forecasting commit c342ce3).

Nothing caught it before users did:

• The consumer’s shape tests validate hand-written fixtures (src/lib/schemas/*.test.ts), never real server output — so they stayed green while the real data broke. Checking the part against an old photo of the part.
• The producer’s tests do not run automatically. data-gov has no test CI (only a deploy workflow + paused bots), so a shape-pinning test only fires when a developer runs it.

Root cause: nothing structurally linked the two repos, and the only cross-repo “tests” validated fiction. This contract test fixes both halves — a producer-side lock that fires the moment the shape changes, and consumer fixtures that are real serializer output instead of fiction.

Two-layer architecture

flowchart TB
  S["Real serializers\n(single source of truth)"]
  G["contract_test.rb\nproducer guard + refresh mode"]
  F["src/test/fixtures/api/*.json\n(real producer output)"]
  K["api-contract.test.ts\nconsumer check"]
  S -->|"seed (rolled back) → authed in-process request → capture + assert shape"| G
  G -->|"refresh mode writes JSON (when CONTRACT_FIXTURES_DIR set)"| F
  F -->|"safeParse through the REAL Zod schemas"| K

Layer 1 — producer guard (test/integration/api/forecasting/contract_test.rb). Seeds representative rows inside the test transaction (auto-rolled-back — net read-only, never mutates dev-DB rows), makes real authed in-process requests, then asserts an exact key-set + per-key type + nullability + envelope against a Ruby type-map that mirrors each consumer Zod schema. Its job is to page a human to refresh the fixtures on any shape change — including a safe additive one, because with no producer CI the strict trip is the only forcing function. Every assertion carries a runbook message.

Layer 2 — consumer check (bioloupe-forecasting/src/test/integration/api-contract.test.ts). Reads the captured fixtures and safeParses them through the same Zod schemas the runtime hooks use (src/lib/schemas/). This is Pact-style consumer-driven verification — tolerant of additive fields, since Zod strips unknown keys.

The no-drift property: the asserted body and the written fixture are the same captured response, so the guard’s type-map and the consumer’s fixtures can never disagree about what the producer emits. Reality (real output) is checked against both definitions, in two languages.

Covered surface

Locked surface = statistics + forecasting model CRUD + pricing.

Surface	Endpoint	Producer serializer	Response shape	Consumer schema	Coverage
Statistics	GET /api/forecasting/statistics	Api::Forecasting::StatisticsController#serialize (+ #synthesized_type_row for type)	{data: [Statistic]}	ApiStatisticsResponseSchema	Full row + envelope
CRUD index	GET /api/forecasting	Api::ForecastingController#index	{forecasting_models: [{id,name,created_at,updated_at}]}	ForecastingIndexResponseSchema	Full
CRUD create	POST /api/forecasting	Api::ForecastingController#create	{model_id, model_name}	ForecastingMutationResponseSchema	Full
CRUD update	PUT /api/forecasting/:id	Api::ForecastingController#update	{model_id, model_name}	ForecastingMutationResponseSchema	Full
CRUD show	GET /api/forecasting/:id	Api::ForecastingController#show	{model_values: <blob>}	presence-only	Envelope key only
Pricing	GET /api/forecasting_pricings	Api::ForecastingPricingsController#serialize_forecasting_pricing	{data: [Pricing], meta: {total_count}}	PricingApiResponseSchema	Full row + envelope (serializer shape unchanged by the publish gate — no published key; row-filtering only)

Statistics synthesizes the type row. Since Decision C, type is no longer a stored row — #index appends one synthesized type row per ready indication (#synthesized_type_rows, value from ForecastingIndication.type_map / .type_for_label) and excludes stored type rows from the feed. The synthesized row is shape-identical to #serialize, so the same STATISTIC_CONTRACT / ApiStatisticSchema covers it; contract_test.rb asserts the contract over every data row, including the synthesized one.

Transition rates are destination-keyed (display convention, no contract impact). A transitionRate row stores the rate on its destination line (line2/met2/…) — the rate flowing into that line from the prior one. The first line of each sequence carries none (100% is structural; the consumer defaults a missing rate to 100). The producer admin displays transitions as NL → (N+1)L, but the stored line key is unchanged — so this is purely a producer display concern with zero contract impact (no key/type/nullability change; Zod’s line: z.string().nullable() is unaffected). The one data exception is CML, whose transitionRate rows are re-keyed onto their destination lines in the seed CSV — every row shifts one destination line down (line1→line2, line2→line3, line3→line4; the line4 row is the former line3 data shifted, not net-new) — a deliberate data correction restoring the sourced 1L → 2L = 46.6%, already applied to prod via the console. The consumer absorbs it through its DISTINCT line roster derivation with no shape/Zod change; the CSV edit only stops a future wipe-and-load re-import from regressing CML.

Indications taxonomy SSOT. The synthesized type value ("Solid Tumor" / "Hematology") is read from the forecasting_indications table’s disease_type column (ForecastingIndication), not a hardcoded Ruby module — the old ForecastingStatistics::AnalyticsDiseases classification was removed. The table is the single source of truth for both the readiness gate (ReadinessChecker) and the synthesized feed row, so the public /statistics row shape and value are byte-identical to the pre-table behavior and the consumer’s Zod schema needs no change. Cutover runbook: after deploying the table, run the one-off thor seed_forecasting_indications:seed once to upsert the taxonomy and auto-publish currently-ready indications.

Pricing publish state is server-only. Whether a (brand_name, indication) tuple is published lives on the forecasting_pricing_publications table and gates which rows the client feed (and per-row show) serves — it is never serialized (mirrors deleted_at). The per-row shape and values stay byte-identical, so the consumer’s PricingEntrySchema needs no change. Cutover: the PO bulk-publishes the seeded rows once from the Pricing console (no backfill task).

show / model_values is envelope-key-only by design. model_values is consumer-owned data the producer just echoes back (written by the client). Validating its interior would make the guard fail on legitimate client-side schema evolution the producer has no control over. The contract guarantees only that the model_values key is present (the consumer throws “Model data missing from response” if absent — that is what we protect).

Pricing locks a non-null price. monthly_wac_price is numeric NOT NULL (0 null rows), so the serializer never emits a null there and the consumer’s non-nullable z.string() is correct as-is. Pricing stays in scope to lock that guarantee: if a future migration made the column nullable, the producer guard goes red instead of the Pricing tab dying in production. The refresh deliberately seeds a null-variant row so the fixtures exercise every .nullable() path. The price_as_of column (added 2026-06) follows the same :date/nullable contract as date_approved — serialized ISO, consumed as z.string().nullish(); the contract fixtures seed both a populated and a null variant. “Outdated” is a console-only badge derived from price_as_of (> 1 year) with no payload impact, and the publish gate is membership-only (it reduces rows, never changes the shape).

Out of scope (by decision, not omission):

• DELETE /api/forecasting/:id — returns {message}, which the consumer discards (not Zod-validated). No Zod break-class to guard.
• Pricing publish admin endpoints (POST/DELETE /api/forecasting_pricings/:id/publish, POST /api/forecasting_pricings/bulk_publish, GET /api/forecasting_pricings/publications) — admin-session console surfaces, not consumer-facing serializers (consistent with the statistics publish API being out of the covered table).
• session (GET /api/check_session) and feedback (POST /api/feedback) — cross-cutting auth/feedback surfaces, not forecasting-domain serializers. Deferred by the “forecasting domain only” scope decision. They are the obvious next candidates, and the harness makes them a small diff to add.

Refresh runbook

When you change a forecasting serializer:

1. Run the producer guard:

   bundle exec rails test test/integration/api/forecasting/contract_test.rb

   It goes red with the runbook message if the shape drifted from the type-map. (For fast local iteration against the dev DB, prefix SKIP_SCHEMA_MAINTENANCE=1 — see the Testing section of CLAUDE.md.)
2. If the change is intentional, update the matching *_CONTRACT type-map in contract_test.rb.
3. Re-run in refresh mode to rewrite the consumer’s fixtures with real output:

   CONTRACT_FIXTURES_DIR=../bioloupe-forecasting/src/test/fixtures/api \
     bundle exec rails test test/integration/api/forecasting/contract_test.rb

   Normal runs write nothing; only CONTRACT_FIXTURES_DIR enables the write. It aborts loudly if the directory is absent (never a silent half-write).
4. In the consumer, run the check:

   pnpm exec vitest run src/test/integration/api-contract.test.ts

   • Green → the change is consumer-compatible (e.g. additive — Zod strips it). Done.
   • Red → it is a real break: update the consumer Zod schema and the dependent app code (or reconsider the producer change). Both repos’ fixes land together in one coordinated release.

The fixtures are deterministic: refresh mode stabilizes DB-generated fields (id → sequential, timestamps → fixed) and sorts rows, so re-running refresh with no shape change produces byte-identical files (no churn).

Limitations (honest)

• The producer guard is discipline-gated — there is no producer CI (by decision). data-gov runs no test CI, so the guard fires only when a developer runs rails test. The design maximizes loudness (strict trip + runbook on every assertion) but cannot make it automatic.
• The consumer check IS CI-enforced. The consumer’s .github/workflows/deploy.yml runs pnpm test:coverage (CI: true) as a hard gate before build/deploy on every non-draft PR and main push, and vitest.config.ts’s include glob already matches api-contract.test.ts — so it runs with zero extra wiring. Caveat: it validates the static committed fixtures, so it catches consumer-schema regressions and corrupted fixtures, but not live producer drift. The producer guard is the always-current live signal; the consumer fixtures depend on the refresh discipline (which the guard’s red drives).
• Fixtures are a snapshot — the consumer check validates the last-refreshed output, not whatever the producer emits right now.
• Shape-only, not semantic/value drift. The contract locks key-set + type + nullability + envelope. A change that preserves the shape but alters meaning or units (e.g. dollars → cents, still a String) passes untouched. Detecting that needs value-level assertions, out of scope.
• Strict exact-key-set. A safe additive producer field trips the guard too — intentional, since the trip is the only thing that prompts a fixture refresh.

Adding a new surface later

About a five-line diff per surface (no framework):

1. Add the surface’s *_CONTRACT type-map + a test method (seed → capture → assert) in contract_test.rb, plus a refresh_fixture call for it.
2. Add the fixture + a safeParse assertion in api-contract.test.ts.
3. Add an inline # CONTRACT: comment above the serializer.
4. Refresh, confirm both sides green.

feedback and session are the obvious next candidates.