26 Slice Organisation Intelligence
Pipeline Slice: Organisation Intelligence
Section titled “Pipeline Slice: Organisation Intelligence”Status: SHARP. Organisation identity, source adapters, single-owner deal flow, temporal ownership, filings, financials, curation subjects, and the
q_organisationsfamily are implemented and DB-testable. Legal-entity hierarchy depth remains deliberately bounded. Denoising state: sharp (noise -> rough -> sharp -> converged) Updated: 2026-07-23
Purpose
Section titled “Purpose”Model organisations, aliases, legal identity, ownership periods, deals, filings, and finance/company attributes so agents can ask “who owns what, when, why, and according to which source?”
Legacy Data Gov treated organisation identity and ownership as a repeated cleanup pressure: company workflow, organisation histories, deal extraction from news, name changes, SEC filings, financial sync, drug ownership audits, merge tasks, and ownership consistency cleanup all point to the same domain problem (../bioloupe-data-gov/docs/news-and-intel.md, ../bioloupe-data-gov/docs/data-model.md, ../bioloupe-data-gov/app/workflows/companies_workflow.rb, ../bioloupe-data-gov/lib/tasks/sec_filings.thor, organisation cleanup tasks under ../bioloupe-data-gov/lib/tasks/one_off/).
Current Decided Shape
Section titled “Current Decided Shape”Organisation intelligence is a temporal identity and relationship layer. Do not model organisation as a single static company row: names change, companies merge, assets are licensed/divested, subsidiaries sponsor trials, and public-company financial identifiers are source-scoped.
flowchart LR SecFmp["SEC + FMP profiles"] OrgRaw["organisation snapshots"] OrgEvidence["identity / filing / financial / subsidiary claims"] OrgIdentity["canonical organisation identity"] NewsRaw["news articles"] NewsEvent["canonical.news_deal_events"] ParticipantLinks["organisation participant links"] BusinessDeal["durable business deal + lifecycle links"] Temporal["org relationships + asset ownership periods"] Query["q_organisations family"] SecFmp --> OrgRaw --> OrgEvidence --> OrgIdentity NewsRaw --> NewsEvent OrgIdentity --> ParticipantLinks NewsEvent --> ParticipantLinks --> BusinessDeal --> Temporal OrgIdentity --> Query NewsEvent --> Query Temporal --> Query
This split is intentional: news owns each immutable source event and its extracted terms; organisation intelligence owns exact participant links, durable cross-event Business Deal identity, and temporal consequences. No FMP fixture or second event table copies a news event.
Source Inventory
Section titled “Source Inventory”| Source | Greenfield role | Source identifiers | Legacy evidence | Web/reference evidence |
|---|---|---|---|---|
| SEC EDGAR | official filings and CIK/ticker-linked disclosure source | CIK, accession number, form type, filing date | SEC filing sync described in news/intel docs and sec_filings.thor | SEC data.sec.gov provides submissions and XBRL company facts APIs updated as filings are disseminated: https://www.sec.gov/edgar/searchedgar/accessing-edgar-data.htm |
| News/deal events | sole event source for acquisitions, licensing, mergers, rebrands | article key, deal_claim_key | news deal/name-change extraction creates organisation_histories | News owns canonical.news_deal_events; this slice consumes it after acceptance. |
| Financial Modeling Prep / market data provider | public-company profiles and provider-scoped metrics only | ticker, provider id, CIK where available | OrgFmpService, daily financial sync in legacy docs | Stable profile and balance-sheet endpoints are documented at https://site.financialmodelingprep.com/developer/docs/stable; preserve provider id, period, unit, and retrieval timestamp. |
| Clinical/regulatory/publication sources | sponsor/applicant/author affiliation/MAH/partner roles | source-local organisation terms | sponsors, approval organisations, publication affiliations in legacy models | Slice-local sources produce org-role evidence; organisation slice resolves durable identity. |
Source Decision
Section titled “Source Decision”Reviewed curation is change-control, not a source: merges, aliases, and ownership corrections enter through curation.decision_events (05-agentic-curation-and-overrides.md), reflecting recurring legacy identity-cleanup pressure (../bioloupe-data-gov/app/services/organisation_matcher.rb, ../bioloupe-data-gov/app/services/organisation_merger.rb, ../bioloupe-data-gov/lib/tasks/one_off/merge_organisations.thor, ../bioloupe-data-gov/app/admin/services/news_deal_review_service.rb).
SEC EDGAR is the authoritative source for public-company filing identity and XBRL facts, but not every biotech/academic/sponsor organisation is public. Therefore the canonical organisation model must support both public identifiers (CIK/ticker/LEI where available) and private/academic/regulatory organisations with verified aliases plus reviewed curation overlays and evidence-backed relationships.
FMP capture is a typed fixture | live | import resource. Live mode requires an explicit symbol
window and API key; max_live_records bounds only that live window. Fixture mode is the free,
deterministic default. FMP does not acquire or own deal records.
Latest legacy Data Gov builds drug ownership suggestions after therapeutic-area refresh (../bioloupe-data-gov/app/workflows/drugs_workflow.rb). The useful intent is to find assets with missing or stale ownership, not to infer ownership from a drug’s lifecycle summary. Greenfield generates candidates from resolved asset/program keys plus explicit approval-holder, trial-sponsor, filing, licensing, acquisition, and deal evidence. Development-program context comes from 33-slice-development-landscapes.md; the drug spine supplies stable identity only. Staged legacy drug_type, development_phase, drug_status, and therapeutic_areas may measure reconciliation coverage, but are never an acceptance basis. The suggestion queue is an intermediate review artifact, not a canonical ownership fact.
Read-only production shape evidence
Section titled “Read-only production shape evidence”A redacted aggregate sample on 2026-07-23 found 4,373 legacy organisation histories, 8,650 participants, and 15,217 drug-ownership rows. The useful intent is real, but the stored shape is not an authority:
- 88 exact type/participant/asset fingerprints repeated across 184 history rows; 48 crossed articles, 45 crossed dates, and 24 mixed executed/non-executed state;
- projected completion dates existed on non-executed acquisitions;
- 4,153 of 4,373 history rows still needed review;
- the legacy deal-reference field had only 13 distinct values across 1,941 populated rows, with one value repeated 1,689 times;
- territory arrays mixed names, codes, blanks, and default-like empty strings.
The temporal ownership table exposed a second scar. Of 15,217 sampled rows, 12,571 were marked current and none of those had an end date, while 2,638 rows marked non-current also had no end date. For the public KRAZATI/adagrasib case, legacy state used 2024-06-20 as the ownership-change date and left an open non-current Mirati row beside the current BMS row, while the official acquisition completion event is dated 2024-01-23. Related BMS/Mirati history was fragmented across event-shaped rows rather than derived from one lifecycle.
Therefore the greenfield does not import legacy cluster ids, review state, geography defaults, or current-owner flags. It preserves product intent through source events, exact/reviewed clustering, explicit lifecycle dates, normalized territory expansion, and replayable rights intervals.
The implemented dependency direction is:
immutable source event -> exact participant and asset resolution -> durable business-deal cluster -> completed/effective organisation relation -> non-overlapping asset ownership intervalObserved aliases remain on the source event, but a resolved asset is keyed and deduplicated by
resolved_drug_key; KRAZATI and adagrasib therefore produce one ownership row, not two.
Organisation exact matching may remove only trailing legal designators such as Inc or Company.
It does not reorder or fuzzily collapse names, and a legal-name core shared by multiple identities
remains ambiguous.
Asset Graph for This Slice
Section titled “Asset Graph for This Slice”flowchart TD Sources[[SEC / FMP]] Raw[sources_organisations.*_snapshots] EvidenceIdentity[evidence.organisation_identity_claims] EvidenceRel[evidence.organisation_relationship_claims] EvidenceFiling[evidence.organisation_filing_claims] EvidenceFinancial[evidence.organisation_financial_claims] Resolution[resolution.organisation_entities] Curation[[curation.effective_overrides]] Canonical[canonical organisation identity] NewsDeal[canonical.news_deal_events] ParticipantLinks[canonical.organisation_deal_participant_links] BusinessDeals[canonical.business_deals] LifecycleLinks[canonical.business_deal_event_links] Temporal[relationships / ownership periods] Query["q_organisations family"] Sources --> Raw --> EvidenceIdentity --> Resolution Raw --> EvidenceRel --> Resolution Raw --> EvidenceFiling --> Resolution Raw --> EvidenceFinancial --> Resolution Resolution --> Canonical --> ParticipantLinks NewsDeal --> ParticipantLinks --> BusinessDeals --> LifecycleLinks --> Temporal --> Query Canonical --> Query Curation -. reviewed merges / ownership overrides .-> Canonical
Slice-Local Domain Model
Section titled “Slice-Local Domain Model”| Entity | Grain | Canonical key recommendation | Notes |
|---|---|---|---|
| Organisation | one durable legal/business/research entity identity | org:<gold-uuid> | Rename-proof identity cluster, not a slug or current name. |
| Organisation alias/name | one name in a source/time context | org_alias:{org}:{name_hash}:{valid_or_seen} | Names can be former/current/trade/branch names. |
| External organisation id | one source-scoped identifier | org_external_id:{source}:{id} | CIK, ticker, LEI, source provider id, registry id. |
| Organisation relationship | one temporal relation between orgs | org_rel:{subject}:{predicate}:{object}:{valid_from} | parent/subsidiary, acquired_by, merged_into, renamed_to, partner. |
| Asset ownership relation | one org-to-drug/asset/program relation over time | asset_owner:{asset}:{org}:{valid_from} | Ownership changes must be time-valid. |
| News deal event (consumed) | one source lifecycle observation | org_event:{article}:{claim_hash} | Owned by news; never copied into an organisation event table. |
| Business Deal | one durable commercial transaction across lifecycle events | deal:{semantic_hash_or_reviewed_key} | Exact semantic clustering or reviewed assignment only. |
| Business Deal event link | one source event assigned to one Business Deal | {business_deal_key}:{deal_claim_key} | Exactly one assignment per accepted transaction event. |
| Deal participant link | one source participant position resolved to an org | {deal_claim_key}:{ordinal} | Stores name, role, exact match basis, and optional resolved_org_key. |
| Filing | one SEC/provider filing document | filing:{source}:{accession_or_id} | Exact filing version/source locator. |
| Financial fact | one company metric over a period/source | org_financial:{org}:{metric}:{period}:{source} | Market cap/revenue/cash/etc with source timestamp. |
Invariants
Section titled “Invariants”- Organisation identity is temporal. Current name, aliases, ownership, and parent/subsidiary relations can change.
- External ids are source-scoped. Ticker and CIK are not the same thing and can change or be absent.
- Merges and name changes are curation override subjects with effective dates, not destructive row edits.
- Asset ownership must preserve valid-from/valid-to and source evidence.
- A Business Deal and a Deal Lifecycle Event are different grains. One deal may have announcement, signing, effective, amendment, partial-termination, and termination events.
- An announced acquisition is not ownership.
owns_assetandacquiredpublish only after an expliciteffective/completedevent; announced deals remain visible inq_organisation_deals. - A projected completion date is a term, not an effective event date.
- Financing events and thin contract events do not auto-cluster; ambiguous cross-event identity is review work, not a fuzzy merge.
- Licences do not imply exclusive ownership.
licensed_toperiods remain independent unless an explicit end or superseding reviewed decision exists. - Lifecycle phase/status or therapeutic-area similarity alone cannot establish asset ownership.
- Public-company financial facts must retain period, currency/unit, source, and filing/provider timestamp.
- Raw SEC/FMP observations remain append-only. When a source corrects bytes under the same release, normalization selects exactly one version by source observation time, capture time, and snapshot id; downstream filings and repeated facts therefore cite the newest deterministic version.
- Every normalized organisation row resolves its artifact by source family, source-native id, and exact content hash. A merely newer artifact for the same CIK/provider id is never substituted.
Slice-Local Entity Resolution
Section titled “Slice-Local Entity Resolution”| Subject kind | Source terms | Candidate target types | Publishable basis | High-impact? |
|---|---|---|---|---|
organisation_identity_link | company name, sponsor, applicant, licensor | Organisation | exact legal name, ticker/CIK/LEI, verified alias, reviewed curation override | yes |
organisation_alias_assertion | old name, branch name, trade name | Organisation alias | source document, reviewed curation override, repeated source evidence | medium/high |
organisation_merge_decision | duplicate org rows/identity clusters | Organisation identity cluster | human-reviewed curation override or strong source evidence | yes |
organisation_relationship_assertion | subsidiary, parent, acquired by, renamed to | Organisation relationship | provider declaration or accepted news event with exact participant links; reviewed curation for corrections | yes |
asset_ownership_assertion | drug/program owner, license holder | Organisation + asset | regulatory/news/deal evidence + verifier or reviewed curation override | yes |
asset_ownership_suggestion | ownerless drug with approval/trial evidence and drug context | Organisation + drug/program candidate | candidate queue only; promote to asset_ownership_assertion after verifier/review | yes |
filing_org_link | CIK/ticker/company name | Organisation | exact CIK/ticker or reviewed curation override | medium/high |
Evidence Claim Schemas
Section titled “Evidence Claim Schemas”organisation_identity_claim
Section titled “organisation_identity_claim”| Field | Meaning |
|---|---|
source_family | SEC or FMP for this claim family |
source_locator | filing/article/source row locator |
name_text | raw organisation name |
external_ids | CIK, ticker, LEI, provider ids |
role_context | authoritative filer/provider profile or observed alias |
seen_at | source observation timestamp |
organisation_relationship_claim
Section titled “organisation_relationship_claim”| Field | Meaning |
|---|---|
subject_org_term | raw subject organisation |
relationship_type | provider-declared subsidiary/parent relation |
object_org_or_asset_term | raw target/object |
valid_from / valid_to | temporal fields when available |
terms | bounded source-specific relationship attributes |
evidence_text | source span |
asset_ownership_suggestion
Section titled “asset_ownership_suggestion”An intermediate review candidate, not a canonical fact. It may be generated for an ownerless resolved drug/regimen/development program from explicit approval-holder, trial-sponsor, filing, deal, licensing, acquisition, or other organisation-role evidence. It must identify the candidate relationship role and validity/territory context, keep source locators for every supporting claim, and promote to asset_ownership_assertion only after deterministic/verifier acceptance or reviewed curation. Legacy lifecycle summaries and therapeutic-area tags may appear in reconciliation metadata only; they cannot support acceptance.
Curation Override Subjects
Section titled “Curation Override Subjects”| Subject kind | Why overrideable | Example curation decision |
|---|---|---|
organisation_merge_decision | Duplicate entities corrupt ownership and search. | Merge “Bristol Myers Squibb” and “BMS” identity cluster. |
organisation_name_change | Name changes must preserve continuity. | OldCo renamed NewCo effective date D. |
organisation_relationship_assertion | Parent/subsidiary/acquirer semantics are temporal. | Org A acquired Org B; B remains subsidiary until date X. |
asset_ownership_assertion | Drug ownership changes affect pipeline intelligence. | Asset X licensed to Org Y for EU only. |
filing_identity_link | CIK/ticker mismatches corrupt financials. | CIK C links to Org O, ticker T as-of date D. |
financial_fact_assertion | Provider facts may be restated or period-specific. | Revenue metric accepted for FY2025 from filing/provider P. |
business_deal_cluster_decision | False merges silently corrupt lifecycle and rights. | Assign event E to explicit stable deal key D after reviewing all source artifacts. |
Canonical Assets
Section titled “Canonical Assets”| Canonical asset | Grain | Built from |
|---|---|---|
canonical.organisations | one durable org identity | identity claims + verifier receipts; reviewed merge overlays when present |
canonical.organisation_aliases | one alias/name context | alias claims + verifier/source receipts; curation overlay when present |
canonical.organisation_external_ids | one source id | SEC/provider/source ids |
canonical.organisation_relationships | one temporal org-org relation | provider relationship claims or effective accepted news events with exact participant links |
canonical.organisation_deal_participant_links | one participant position/link decision | news-owned canonical deal event + exact organisation name/alias lookup |
canonical.business_deals | one durable commercial transaction | exact semantic event fingerprint or reviewed cluster assignment |
canonical.business_deal_event_links | one event-to-deal assignment | news-owned lifecycle event + exact/reviewed clustering basis |
canonical.drug_ownership_periods | one temporal org-asset relation | effective/completed acquisition or licence events; curation overlay when present |
canonical.organisation_filings | one filing/document | SEC/provider filing snapshots |
canonical.organisation_financial_facts | one metric/period/source | SEC XBRL/provider data + checks |
canonical.news_deal_events (consumed, news-owned) | one deal/corporate event | accepted news deal evidence; never duplicated here |
q_organisations Family
Section titled “q_organisations Family”| Query model | Grain | Purpose |
|---|---|---|
q_organisations | one organisation | canonical org profile, current name, aliases, identifiers, public/private flags |
q_organisation_aliases | one alias | name search and identity trace |
q_organisation_relationships | one temporal relation | parent/subsidiary/rename/acquisition timeline |
q_asset_ownerships | one temporal asset ownership relation | drug/program owner over time |
q_organisation_deals | one durable Business Deal | lifecycle dates, all source events, participants, terms, assets, and provenance |
q_organisation_filings | one filing | SEC filing discovery and provenance |
q_organisation_financials | one metric/period/source | financial facts for agents |
q_organisation_relationships contract seed
Section titled “q_organisation_relationships contract seed”q_organisation_relationships is the temporal relationship surface. It must preserve validity intervals and source authority instead of overwriting “current” relationships.
| Contract field | Required meaning |
|---|---|
| Purpose | Answer: “What relationship exists between these organisations, during what period, according to which source?” |
| Grain | One subject organisation + relationship type + object organisation + validity interval + source/authority context. |
| Stable row key | org_rel:{subject_org_key}:{relationship_type}:{object_org_key}:{valid_from_or_observed}. |
| Required columns | organisation_relationship_key, subject_org_key, subject_name, relationship_type, object_org_key, object_name, valid_from, valid_to, observed_at, source_family, source_artifact_key, source_locator, source_authority_tier, resolution_basis, assertion_state, provenance_key. |
| Search affordances | btree filters on subject/object/type/date; trigram search over organisation names/aliases; facets on source family and relationship type. |
| Provenance rule | provenance_grain=bundle; field receipts for valid_from, valid_to, and relationship type when source-derived. |
| Blocking checks | schema_contract, provenance_coverage, source_artifact_immutability, temporal_validity, resolution_basis, curation_review_state when merge/ownership override applies. |
Relationship types should start conservative: parent_of, subsidiary_of, acquired, merged_into, renamed_to, licensed_to, partnered_with, owns_asset, formerly_known_as. Slice fixtures should decide which are exclusive over time.
P08 uses two named profiles instead of smuggling unrelated rows into one count. The BMS/Mirati
profile proves the 2024-01-23 completion event, one durable deal, one acquisition relationship,
and exactly one BMS ownership interval for drug:adagrasib. The Merck family profile resolves
officially listed subsidiaries, a trial sponsor, a pembrolizumab development program, regulator
holders, and reported sales through stable organisation keys. The fixture’s subsidiary names are
cross-checked against Merck’s SEC Exhibit 21:
https://www.sec.gov/Archives/edgar/data/310158/000162828024006850/ex21-subsidiarylistasof123.htm.
The representative fixture includes Merck Sharp & Dohme LLC and MSD K.K.. It deliberately does
not introduce the similarly named Merck Sharp & Dohme B.V. without a durable external identifier:
authoritative source status permits a new identity, but it does not make a fuzzy name collision safe
to auto-link or auto-merge. That case parks for review.
Downstream key contract
Section titled “Downstream key contract”resolved_org_key(s) used by downstream slices (trial sponsors/collaborators in 21, deal/event participants and regulatory-event holders in 24, applicant/holder/partner roles in 22) is defined here as a rename-proof key into q_organisations (org:<gold-uuid>). It is the same identity expressed by subject_org_key/object_org_key inside q_organisation_relationships. Downstream slices reference this key; they do not mint organisation identity. A resolved_sponsor_key is not a separate identity — it is a resolved_org_key observed in a sponsor/collaborator role, so trial/news org references must resolve into this spine rather than create a parallel sponsor namespace.
Checks and Quality Gates
Section titled “Checks and Quality Gates”Blocking:
- every high-impact merged organisation identity has verifier acceptance or reviewed curation override plus a provenance bundle;
- every external id is source-scoped and unique within that source namespace;
- every
resolved_org_key(includingsubject_org_key/object_org_key) in anyq_*model exists inq_organisations(enforced corpus-wide by thereferential_integritygate in08-contracts-and-quality.md); - temporal relationships have non-overlapping validity where exclusivity is required;
- asset ownership changes preserve prior owner/time interval instead of overwriting;
- announced acquisitions produce zero ownership/acquired assertions;
- every accepted transaction event is assigned to exactly one Business Deal;
- every
q_organisation_dealsrow resolves to all contributing source events throughq_provenance; - effective/terminated deal state requires an explicit effective/terminated lifecycle event;
- SEC financial facts retain CIK/accession/period/unit/source timestamp;
q_organisation_*schemas match generatedq_catalogand provenance-required rows/facts resolve throughq_provenance.
Warning:
- conflicting current names or tickers;
- overlapping ownership intervals;
- unresolved deal participants;
- stale financial data;
- source-only aliases with no canonical identity;
- provider facts lacking SEC cross-check.
Rejected Shapes
Section titled “Rejected Shapes”- FMP-owned
news_deals.json: wrong source owner and a second ingestion path for the same event. - A second
canonical.deal_eventstable: duplicates news event identity, terms, and lifecycle. - One deal per article forever: loses signing/completion/termination continuity and duplicates commercial identity.
- Legacy
news_deal_reference_uid: production cardinality shows it is a default-like field, not a cluster authority. - Fuzzy automatic event merging: a false merge corrupts lifecycle and downstream rights; exact false-negative splits are safer and reviewable.
- Ownership on announcement: confuses a proposed transaction with an effective legal state.
- Generic licensed-feed “live mode”: licensed news feeds use explicit imports until a concrete provider adapter exists.
Remaining Bounded Questions
Section titled “Remaining Bounded Questions”- Organisation hierarchy stays as typed temporal relationships, not nested-tree columns.
- Provider financial facts may publish with a provider authority label and warning when no SEC cross-check exists.
- Full bitemporality remains deferred until correction-history requirements exceed the current
valid_from/valid_to/ source-observation model. - Territory normalization needs an authority-backed geography adapter before raw source strings can become reusable include/exclude sets; current strings remain observed evidence.