Learning and evals

SignalDock learns from what happens after outreach, but it never changes your setup on its own. Outcomes become suggestions a person applies, and examples the writer reads.

OutcomesCopy Anchor Link

An opportunity counts as converted when it is marked won, a contact booked a meeting, or a reply was classified as interested or a meeting request. It counts as ended when it is marked lost, or every enrollment finished without converting. Mark outcomes on the opportunity page (Mark won with an optional deal value in euros, or Mark lost), through the API (POST /opportunities/{opportunityId}/outcome), or with the opportunities_setOutcome MCP tool.

Scoring suggestionsCopy Anchor Link

Every Monday the learning job compares the score components of converted opportunities with the rest. When a rule's share of the score differs clearly between the two groups, it suggests a new weight:

• It needs at least 12 closed opportunities, with 4 or more on each side.
• A suggestion moves the weight by the measured lift, never below 0 or above twice the current weight.
• Changes under 15% are not suggested.

Suggestions appear under Settings → Scoring → Suggested from outcomes. An organization owner or admin chooses Apply (the rule weight changes and an audit event is written) or Dismiss. While suggestions are open, the job does not add new ones.

Writer examplesCopy Anchor Link

Each review in the approval inbox is recorded: approved unchanged, edited, or rejected. When the writer drafts the next step, it gets up to three recent examples from your organization as style references. Drafts that led to a booked meeting come first, then positive replies, then edited drafts (using your edit), then approved drafts. Rejected drafts are never used.

EvalsCopy Anchor Link

Approved and edited drafts also form an eval set. The writer prompt stored with each draft is replayed against the current prompt version, and every result is graded:

A prompt version passes when at least 80% of cases pass and neither the pass rate nor the similarity drops more than 3 points below the last passed run. Run it before changing the writer prompt:

pnpm eval:writer --organization eventdock

Pass --limit 40 or --model <gateway-model> to change the sample or model. The command exits with code 1 when the gate fails, so it can run in CI. The dashboard shows the latest result under Agent quality.