Sources

A signal source belongs to an organization: a provider key, display name, JSON config (validated by that provider’s schema), enabled flag, schedule, and a country (NL, BE, DE, …) and language (nl, de, fr, en). Country and language are passed to classification, and the country selects the company registry used for entity resolution.

What a provider doesCopy Anchor Link

Every source implements SignalSourceProvider:

• fetchNew({ config, cursor, limit }) — page of raw publications + next cursor
• normalize(raw) — candidate signal fields
• externalId(raw) — stable id for dedupe

Cursors are stored per source. Ingest is started from the API (Vercel Cron every 30 minutes, or Run now in the app).

After fetchCopy Anchor Link

1. Insert or skip raw_publications
2. Insert signals with status ingested
3. Start processSignalWorkflow for new (or changed) signals

Rows are unique on (source_id, external_id), and a content hash skips unchanged payloads, so a provider may safely return a publication twice.

Failed ingest runs are visible on the source (last run, errors). Disable a source to stop cron from fetching it without deleting history.