Repository navigation
outreach-engine: build plan + M0–M7 + M8 (Gmail, Outlook, Slack, health checker, D5 jurisdictions) - #15
Draft
splitintech wants to merge 20 commits into
Draft
outreach-engine: build plan + M0–M7 + M8 (Gmail, Outlook, Slack, health checker, D5 jurisdictions)#15splitintech wants to merge 20 commits into
splitintech wants to merge 20 commits into
Conversation
Provider-neutral outreach orchestration spec: durable action queue with claimed/executing split and uncertain-outcome reconciliation, versioned playbooks, approvals bound to content hashes, deterministic importer, reply correlation, CLI/HTTP/MCP/Slack/Papr surfaces, milestones M0-M10, and a parallel non-blocking Papr Work upstream track. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
npm workspace with strict TypeScript, ESLint, Vitest and tsup; CI workflow on Node 22 and 24; scripts enforcing package boundaries, a secret scan and a 400-line file limit; README; ADRs 0001-0003; and a seed @splitin/outreach-contracts package so the pipeline builds and tests real code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Owner
Author
|
It failed on the first commit, which only added a markdown file, so it will fail on every PR until Dependency graph is turned on under Settings → Code security (admin only). No code change here can fix it, and I've left the shared workflow untouched. The check for this folder is Generated by Claude Code |
Implements milestone M1 of outreach-engine/BUILD_PLAN.md (§5, §14). @splitin/outreach-contracts (no runtime dependencies) - actions.ts: the action state machine from §6.1 as a single closed transition table (ACTION_TRANSITIONS), terminal and cancellable state sets, and assertTransition() throwing a typed InvalidTransitionError. The table encodes ADR 0002: `executing` is reachable only from `claimed`, and neither `executing` nor `uncertain` can be cancelled or rescheduled directly; uncertain actions leave only through reconciliation or human review. - errors.ts: the provider error taxonomy (11 classes) and one ErrorDisposition per class: retry never / backoff / after re-auth, terminal state (failed or review), and side effects (suppress recipient, bounce enrollment, mark account unhealthy or needing re-auth, engage the account kill switch, pause the campaign). The core will act on the class alone, so no provider logic lives in the engine. - capabilities.ts: ProviderPurpose (manual_correspondence, transactional, automated_outreach, marketing, bulk), CapabilitySnapshot, and purposePermitted(), which requires BOTH the adapter's terms and the operator-attested account purposes to allow a campaign's purpose. - providers.ts: capability-sized provider ports instead of the original plan's monolithic MailConnector: AccountPort, EmailSender (send + reconcile), MailboxReader (cursor-based), WebhookVerifier (raw body, verified before parsing), NotificationPublisher, ManualTaskProvider, and ProviderAdapter bundling them. SendResult distinguishes accepted, rejected (with ErrorClass and retryAfterMs) and unknown; ReconcileResult is found, absent (only when the provider can affirm it) or still_unknown. InboundMailEvent carries provider/RFC ids, references, DSN details and a bounded snippet for deterministic classification. - sql.ts: a synchronous SQLite-dialect SqlDatabase port. Transactions are synchronous by design, so a transaction can never span an await and provider calls always happen outside them (§6.6). The store package implements it; core and importer write SQL against it. The plan's package layout is updated to record this split. - crypto.ts: canonical JSON, sha256/HMAC helpers, constant-time hex comparison, and a dependency-free monotonic ULID. @splitin/outreach-fakes - FakeEmailProvider: deterministic in-memory email provider with a scriptable outcome queue (accept, reject with any ErrorClass and retryAfterMs, unknown_after_accept, unknown_before_accept, throw_after_accept). It records a delivery only when the effect really happened, which later lets the engine's tests assert "at most one delivery per action" directly. It also checks the secret resolved through ctx.secrets, supports optional provider-side idempotency, threads replies by In-Reply-To, simulates replies, DSN bounces and complaints, pages a mailbox by cursor, and signs and verifies HMAC webhooks with a 5-minute timestamp window, 1 MB cap and tamper detection. The reconcile mode can be switched to still_unknown to exercise the review path. - FakeNotifier: stands in for Slack with the same scriptable outcomes. - Conformance kit (EMAIL_SENDER_CONFORMANCE, runEmailSenderConformance): framework-agnostic cases every email adapter must pass: stable ids, no credential leakage in results, reconcile finds accepted sends with the same provider id, reconcile never "finds" an unsent message, timeouts after/before accept reconcile correctly, rate limits carry retryAfterMs, rejections use the taxonomy. Cases that need a forced outcome are skipped for real sandbox accounts. staticSecrets() helper. Tooling - vitest resolves @splitin/outreach-* to package sources, so tests never depend on a prior build. - The boundary check lets *.test.ts files import workspace packages declared in devDependencies, and reserves @splitin/outreach-e2e for the end-to-end suite. Tests: 31 passing (transition-table closure and invariants, taxonomy coverage, crypto vectors, ULID ordering/uniqueness, full conformance suite against the fake, delivery bookkeeping, webhook verification, mailbox paging, notifier). lint, typecheck, boundaries, secret scan and line limit all pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Implements milestone M2 of outreach-engine/BUILD_PLAN.md (§4, §6.6, §14).
@splitin/outreach-store-sqlite (new)
- schema.ts: the full initial schema from plan §4 as migration
0001_init. 25 STRICT tables covering workspaces/principals, contacts
and contact points (consent basis, jurisdiction, permitted channels),
mapping profiles and import batches/rows, templates, sequence and
campaign versions, audience snapshots, enrollments, scheduled actions
and attempts, messages, provider events and cursors, manual tasks,
approvals, suppressions, kill switches, rate buckets and the audit
log. Invariants are constraints, not application convention:
* one live enrollment per (workspace, campaign, contact) via a partial
unique index over status IN ('active','paused'), which fixes the
race the original plan left to app logic;
* action idempotency: UNIQUE (workspace_id, idempotency_key);
* provider event dedupe: UNIQUE (provider_account_id, provider_event_id);
* CHECK constraints on every state/kind/reason enum and foreign keys on.
Columns the execution core needs beyond the plan's sketch are
explicit: action purpose, recipient_norm, contact_point_id and
reconcile_count; per-attempt rate-budget reservations; the
confirmed_absent attempt outcome; message recipient_norm for
per-recipient gaps; event class/correlation/enrollment; workspace
settings; approval reason.
- audit_events is append-only: BEFORE UPDATE / BEFORE DELETE triggers
abort.
- migrate(): ordered migrations, each in its own transaction, recorded
with a sha256 of its SQL; an applied migration whose source changed
raises MigrationDriftError instead of silently diverging.
- driver.ts: wrapNativeDatabase() adapts any synchronous SQLite handle
(node:sqlite DatabaseSync or better-sqlite3) to the SqlDatabase port,
with statement caching, BEGIN IMMEDIATE outer transactions and
savepoints for nesting. Returning a Promise from a transaction
callback throws TransactionMisuseError, enforcing "never await inside
a transaction".
- open.ts: openSqliteDatabase() for the built-in node:sqlite driver (WAL,
busy_timeout, foreign_keys, synchronous=NORMAL, auto-migrate), and
fromBetterSqlite3() for hosts such as Papr Work that already load
better-sqlite3. The better-sqlite3 path is structural, so there is no
native dependency here; it is not exercised in CI yet because
installing the native module is left to hosts.
@splitin/outreach-contracts
- audit.ts: appendAudit() and verifyAuditChain(). Each row stores
prev_hash and hash = sha256(prev_hash || canonical(row)); writers
append inside their own transaction so BEGIN IMMEDIATE serializes the
chain. verifyAuditChain() recomputes it and reports the first broken
seq. It lives in contracts so the core and importer can both write
audit rows without depending on the store package.
Build
- tsup configs set removeNodeProtocol: false; esbuild otherwise rewrites
"node:sqlite" to "sqlite", which does not resolve. The ESM and CJS
bundles are smoke-tested against a real database.
Tests (11 new, 42 total): migrations idempotent and drift-refusing;
live-enrollment uniqueness while finished enrollments repeat;
idempotency-key and provider-event dedupe; STRICT typing and CHECK
enums; foreign keys; audit triggers abort UPDATE/DELETE; the chain
detects an edited row even with the trigger dropped; nested savepoint
rollback semantics; async callback refusal; two connections to one
file serialize writers (BEGIN IMMEDIATE, SQLITE_BUSY with no timeout).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Implements milestone M3 of outreach-engine/BUILD_PLAN.md (§6, ADR 0002), the part of the system that must not be rushed. @splitin/outreach-core (new), src/execution/ - enqueue.ts: enqueueAction(), the only way an external effect is created. Idempotent on (workspace, idempotency_key); freezes the payload and its canonical content hash; generates the RFC Message-ID for email kinds; audits creation. - actions-repo.ts: transitionAction(), the only way an action changes state. It enforces the contracts transition table, compares-and-sets on the expected state (a concurrent change raises StaleActionError instead of being overwritten), and appends a hash-chained audit row for every transition. - preflight.ts: claimActions() takes due actions under a short lease in one BEGIN IMMEDIATE transaction. preflight() is the last gate before any effect: in one transaction it re-validates the lease, runs the checks (kill switches, expiry, pluggable domain checks, account health, adapter capability, purpose permitted by adapter AND account, live-send allowlist gate), reserves rate budget, and commits `executing` together with a pending attempt BEFORE the provider is called. Verdicts are pass / defer / cancel / await_approval / review. - invoke.ts: calls the provider outside any transaction with a timeout. Anything an adapter throws is treated as `unknown`, never as a failure, because we cannot prove the request never left the process. - results.ts: records the outcome in its own transaction. Accepted means succeeded plus the outbound message row (or an open manual task). Rejected follows the error taxonomy: backoff with full jitter or honoured retryAfterMs, re-auth hold, max-attempts to failed, content_rejected to review. It applies account effects (unhealthy, reauth_required, provider-account kill switch) and releases rate budget for confirmed non-sends. Unknown means `uncertain`, never a retry; notifications, which cannot be reconciled, fail instead of retrying. A late definitive result for an action the sweeper already parked resolves it as reconciliation would. - recovery.ts: sweepExpiredLeases() puts expired `claimed` actions back to `scheduled` (no attempt started, so safe), and sends expired `executing`/`reconciling` actions to `uncertain`. reconcileUncertain() asks the adapter: found means succeeded, absent means retry if attempts remain, still_unknown means exponential re-check and, after 3 tries, `review`. - rate.ts: atomic fixed-window budgets (all-or-nothing inside preflight) and a per-recipient minimum gap across campaigns. - kill-switches.ts: global, workspace, provider-account and campaign switches, checked at preflight; engage and release are audited. - review.ts: human resolution of review items: sent (records success with the given provider id), not_sent_retry, or drop. - executor.ts: executeDue() and runExecutionPass() (sweep, reconcile, execute). Crash hooks at afterClaim / afterPreflight / beforeResult let tests kill the "process" at every step boundary. Manual tasks can optionally be announced to a host. - ActionEffects hooks (onSucceeded, onFailed, onCancelled, onErrorEffects) run inside the outcome transaction; the campaign domain (M4) plugs into them. Tests (29 new, 71 total) - Executor: happy path with attempt, message and a valid audit chain; not-due; enqueue idempotency; retryable backoff with budget release; retryAfterMs; max attempts; permanent recipient failure with effects; revoked auth holds the account; policy block engages the account kill switch; content rejection to review; global kill switch defer/release; expiry; allowlist gate plus review retry; purpose gate; fixed-window budget and recipient gap; uncertain notifications never retried; manual tasks. - Failure injection: timeout after accept reconciles to succeeded with no resend; timeout before accept retries only after affirmed absence; adapter crash after accept; undecidable outcomes go to review after 3 checks and are never resent; crashes after claim, after the preflight commit and after provider acceptance each recover with exactly one delivery; late results; a dead reconciler's lease is recovered. - Property test (fast-check, 60 runs): random provider behaviour (accept, transient, rate limit, bad recipient, unknown after/before accept, adapter throw) combined with random crash points over many rounds. Invariants: no action is ever delivered twice; every action ends succeeded/failed/review; succeeded holds exactly when delivered; the audit chain verifies. Mutation-checked: making reconciliation treat "found" as "absent" is caught with a minimal counterexample. - Concurrency: two workers on separate connections to one database file, 300 actions, interleaved passes: no action has two attempts and every message is delivered exactly once. A sweep never steals a live lease. Not yet covered: the plan's 4-process x 10k soak, which needs built artifacts in child processes. It is deferred to the CLI milestone (M7), where a worker binary exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Implements milestone M4 of outreach-engine/BUILD_PLAN.md (§7, §9, §14): turns playbooks into per-contact scheduled actions and plugs the domain rules into the M3 execution core. @splitin/outreach-core, src/domain/ (new) and src/engine.ts - auth.ts: AuthContext (workspace, principal, roles, surface, trace), always derived from an authenticated identity via authenticate(); unknown identities get nothing. Cumulative roles viewer < operator < approver < admin, requireRole(), and audit() for every domain mutation. - calendar.ts: send windows in the recipient's time zone with fallback, allowed weekdays and holidays. nextSlot() and addDuration() use the business calendar (P3D = three sending days) or elapsed time. Luxon handles DST, so slots never land on nonexistent local times. - playbook.ts: zod schema for the Playbook format in plan §7.1 (strict: unknown fields are errors), with policy defaults (approval mode, first-batch size, window, account/domain/campaign daily limits, recipient min gap, postal address, unsubscribe link or reply, expireAfter). Social steps can only be manual.task (ADR 0003). YAML or object input; errors carry field paths. - compile.ts: semantic checks against the database and adapters: unique step ids, a reply only after a send, templates exist on the right channel, send templates have subjects, no trailing waits, the purpose is permitted by both adapter and account, a postal address when required, and unsubscribe links only when a public URL is configured. - templates.ts: immutable versioned templates with a token allowlist (contact, organization, sender, attr.* and unsubscribe_url). Rendering fails closed on any missing value, escapes values in HTML, and strips control characters from untrusted contact data (no header or line injection). - materialize.ts: creates the next step's action, walking waits on the business calendar and snapping to the send window. Content is fully rendered and frozen before approval, with an automatic compliance footer (organization, postal address, unsubscribe link or reply instruction) and RFC 8058 List-Unsubscribe headers when the provider supports custom headers. Replies thread via In-Reply-To, References and the provider thread id. A missing value stops that enrollment with template_missing:<tokens> instead of sending a broken email. - campaigns.ts: createCampaign (validated, deduplicated sequence versions); prepareActivation snapshots the audience (excludes suppressed, no legal basis, already live in any campaign, duplicate contact) and computes a version hash over sequence, policy, audience, templates and account; commitActivation activates exactly the previewed and approved hash, enrolls the snapshot and materializes first steps. Stale previews are refused. Pause, resume and complete. - approvals.ts: action, batch and campaign-version scopes bound to content hashes. Batch approvals list exactly their actions and content. Decisions require the shown operation hash (two-phase), an optional separation of duties, and expiry; rejection cancels and stops the affected enrollments. Revocation holds sending at preflight until requestBatchApproval() gathers waiting actions into a new batch. - suppressions.ts: global, channel, provider-account and domain scopes; an atomic stopEnrollment() that cancels every cancellable action and expires open tasks; self-verifying HMAC unsubscribe tokens. - wiring.ts: domain preflight checks (enrollment live, campaign live, not suppressed, approval valid for this exact content, inside the send window); a per-campaign rate policy (account/day, domain/day, campaign/day, recipient gap for new threads); and effects (advance on success, error on failure, stop on suppression or expiry, suppress plus bounce on bad recipients, pause the campaign on account failures). - operations.ts: principals, provider account registration (records the adapter capability snapshot; secrets are env:/keychain: references only), manual contacts, suppressions, enrollment pause/resume/stop, manual task listing, and recordManualOutcome() (only a human completes a social task; completion advances the sequence). - engine.ts: createEngine() wires execution and domain; the default send gate is an EMPTY allowlist, so nothing is emailed until an admin opens it. Also bootstrapWorkspace, kill switches as a service (operators may stop, only approvers resume, global needs admin), review listing and resolution, and campaign status. Design notes - A contact can be live in only one campaign at a time (a safety default stricter than the per-campaign unique index). - Reply templates may omit a subject and inherit "Re: <original>". Tests (28 new, 99 total) - Calendar: inside/early/late/weekend, holidays, DST spring-forward gap and fall-back overlap, zone fallback, a 300-run property test (slot is never earlier, always inside the window, idempotent) across six zones including half-hour and 45-minute offsets, business-day durations. - Playbook: defaults, path-bearing structural errors, no non-manual social steps, no unknown fields, semantic compile issues, role checks. Templates: fail-closed rendering, HTML escaping, header injection stripped, immutable versions, unknown tokens rejected. - Lifecycle: import to completion with audience exclusions, first-batch approval, List-Unsubscribe, a 3-business-day threaded follow-up, a manual social task and completion, audit chain valid; the recipient's window across time zones; stale preview and unapproved activation refused; separation of duties; batch rejection; every_action approvals; revoke then re-batch; mid-sequence suppression; campaign and enrollment pause; hard bounce suppression; missing template value; the default send gate parks sends for review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Implements milestone M5 of outreach-engine/BUILD_PLAN.md (§10, §14):
detect, parse inertly, stage, map, normalize and validate, resolve
duplicates, preview, then commit exactly the preview.
@splitin/outreach-import (new; depends only on contracts)
- detect.ts: format detection by magic bytes (ZIP means XLSX), then
extension, then content sniffing. Size and row limits (25 MB / 100k
by default). Decoding: BOM (UTF-8/16), strict UTF-8, otherwise a
Windows-1252 fallback reported as a preview warning.
- parsers.ts:
* CSV/TSV via csv-parse 7 in strict array mode. Rows are never objects
keyed by untrusted headers, and records are null-prototype, so
"__proto__"/"constructor" headers are plain keys. csv-parse is
pinned at >=7.0.2 to avoid the columns-path prototype replacement
advisory.
* JSON arrays and JSON Lines, with nested objects flattened to dotted
keys (depth 3) and "__proto__" keys dropped.
* XLSX via exceljs, reading cached values only: formulas are never
evaluated, a formula without a cached result reads as empty, and
hyperlinks/rich text are reduced to text. An npm override moves
exceljs to uuid 11 (clears the uuid advisory).
- html.ts: parse5 builds a tree only; no scripts, resources or styles.
Text inside script/style/template/noscript/iframe/object is ignored
and traversal is depth-limited. Table mode (header row, cells, plus
mailto/http links as "<column> link") and card mode (repeated records
with a small selector language: tag, .class, [attr], descendant
chains, and "selector@attr" for attributes). Anything richer is
rejected, never guessed.
- profile.ts: versioned, immutable mapping profiles (zod-validated):
canonical field to source column(s), attributes for {{attr.*}},
required fields, consent basis and evidence, jurisdiction, name
splitting.
- normalize.ts: deterministic normalization and validation. Emails are
lowercased and validated (mailto: stripped). Profile URLs are
canonical https, no www, no query or fragment. Organization domains
are validated or derived from a non-freemail email. IANA time zones
are validated. Names are whitespace-collapsed. Errors are specific
("invalid email ...", "unknown time zone ...") and never duplicated
by a "missing required" on the same field.
- resolve.ts: explainable duplicate rules. Rows with errors are
rejected; a repeat email/profile in the file merges into its first
row; a known email/profile updates that contact; the same name at the
same organization under a different email is ambiguous and never
auto-merged; everything else creates.
- importer.ts: previewImport() stages rows with locators (csv:row=N,
xlsx:Sheet!N, html:table[i]/tr[j], jsonl:line=N), outcomes, errors,
counts and up to 20 samples per outcome, plus a preview hash over
source bytes, profile and resolved rows. It creates no contacts.
commitImport() requires the preview hash and an idempotency key,
re-resolves every row against the current workspace, and refuses
with ImportStaleError if anything changed. It then creates or updates
contacts, organizations and contact points (with source provenance,
consent basis/evidence, jurisdiction, permitted channels), links
import rows to contacts (used by playbook audiences with
source: { importBatch }), and audits. Updates fill empty fields only.
Re-committing with the same key is a no-op; a different key is
refused. Commit never enrolls anyone or creates actions.
- export.ts: CSV export neutralizing formula injection (cells starting
with = + - @ tab CR are prefixed with a quote; all cells quoted).
Tests (17 new, 116 total): mixed CSV with a merge, invalid email,
missing email and an unknown zone; formula-looking cells kept inert;
__proto__/constructor headers; Windows-1252 fallback and UTF-8 BOM;
size and row limits; profile validation; commit provenance, consent,
merge linking, idempotency, no enrollments or actions, valid audit
chain; update-without-overwrite plus an ambiguous look-alike; a stale
preview refused; HTML tables and cards with scripts, templates and
onerror ignored; 3,000-deep nesting rejected, not crashed; unsupported
selectors rejected; JSON/JSONL flattening and prototype safety; XLSX
cached formula results, an unevaluated WEBSERVICE() formula, rich
text, hyperlinks, a corrupt file; CSV export escaping. Performance: a
100,000-row CSV preview runs in about 3.4 s locally against the plan's
10 s budget (memory not yet measured).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Implements milestone M6 of outreach-engine/BUILD_PLAN.md (§8, §13, §14):
replies, bounces, complaints and opt-outs stop sequences atomically,
and the full import-to-audit loop runs end to end on fake providers.
@splitin/outreach-core, src/inbound/ (new)
- ingest.ts:
* ingestWebhook(): 1 MB cap, then the adapter verifies the RAW body
(signature and timestamp) with the account's webhook secret
reference before anything is parsed. Rejections are audited and
store nothing. Accepted events are stored durably and processed
later, so the endpoint can answer fast.
* pollMailbox() / pollDueMailboxes(): cursor-based polling as the
webhook fallback, with the cursor persisted in the same transaction
as the events. Per-account failures never block sends or other
accounts.
* storeInboundEvents(): (account, provider event id) is unique, so
webhook retries and poll overlap collapse. Payloads are minimized to
identifiers, headers and a snippet of at most 500 characters; full
bodies are never kept.
- classify.ts: deterministic rules. Complaint and delivery events;
DSNs (5.x.x hard, 4.x.x soft, otherwise unknown); auto-replies
(Auto-Submitted unless "no", X-Autoreply/X-Autorespond,
Precedence auto_reply/bulk/junk, multilingual out-of-office
subjects); opt-out phrases or a bare "STOP"; everything else is a
human reply. No LLM applies a class.
- correlate.ts: in order of trust: provider thread id, then
In-Reply-To / References / DSN original Message-ID against our
generated Message-IDs (strong), then the sender or bounced recipient
matching an outbound recipient within 30 days (weak).
- process.ts: one transaction per event, oldest first: classify,
correlate, record the inbound message, apply, audit, notify.
* human reply: stop the enrollment as `replied`, cancelling pending
actions (including one another worker has claimed; its preflight
then finds the lease lost). Weak matches still stop, the safe
direction, but go to review for a human.
* opt-out: global suppression plus every live enrollment of the
contact stopped as `opted_out`.
* hard bounce: email-channel suppression plus `bounced`; soft bounces
stop only on the third.
* complaint: global suppression, stop, and the sending account's kill
switch engaged (plan §18 circuit breaker).
* auto-reply: recorded, no stop. Uncorrelated mail: review.
Deliveries: ignored.
* notifications: when a workspace has a notification account
(configureNotifications), replies, opt-outs, bounces, complaints and
review items enqueue durable notify.publish actions with ids and
routing only, never message bodies.
- unsubscribe.ts: handleUnsubscribe() behind POST /u/:token (RFC 8058
one-click). The token is verified by HMAC alone, so forged tokens
cannot probe the database. Global opt-out suppression, all live
enrollments of the contact stopped, audited, idempotent.
- engine.runOnce() now polls due mailboxes and processes inbound
events BEFORE the execution pass, so a reply received since the last
pass stops a follow-up in the same pass it would have been sent.
pollIntervalMs is configurable (default 5 minutes).
@splitin/outreach-e2e (new, private, tests only)
- Plan §13 end to end on fake providers: bootstrap a workspace with
Slack-style principals; register email and notification accounts;
import a CSV (4 create, 1 merge, 1 reject) through preview and
commit; templates; a playbook whose audience is that import batch;
campaign-version approval plus first-batch approval (only the first
two wait); sends inside each recipient's window, including Helsinki;
List-Unsubscribe on every message. Then: one recipient replies by
signed webhook (stopped, notified), one clicks unsubscribe (globally
suppressed), one hard-bounces (suppressed, notified), one stays
silent, gets the manual social task, and after a human completes it,
a threaded bump. Final state: replied 1, opted_out 1, bounced 1,
completed 1; every message delivered exactly once; the audit chain
verifies.
Tests (30 new, 146 total): 18 classification cases; threaded webhook
reply stops before the follow-up and notifies without the body;
webhook plus poll of the same event processed once; Message-ID and
weak-address correlation, with weak matches to review; a reply racing
a claimed follow-up; auto-replies do not stop; opt-out suppression and
notification; hard vs soft bounces; complaint engages the account kill
switch; uncorrelated mail and delivery receipts; forged, stale and
oversized webhooks rejected with nothing stored; unsubscribe link
suppression, idempotency and forged-token rejection; the e2e suite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
splitintech
changed the title
First part of milestone M7 of outreach-engine/BUILD_PLAN.md (§11.2,
§12): the HTTP surface. It includes the M7.0 Papr spike finding that
shapes the rest of M7 (full write-up lands with the Papr app in part 3).
@splitin/outreach-store-sqlite
- Migration 0002_api_tokens: bearer tokens bound to a principal and
workspace, storing only a SHA-256 of the secret, a role ceiling,
expiry, revocation and last use.
@splitin/outreach-core
- domain/access.ts:
* createApiToken() (admin only, 1 to 366 days): token format
oet_<ulid>.<43-char secret>, shown once.
* authenticateToken(): every failure raises the same
ForbiddenError, so a caller cannot tell a bad token from an expired
one; constant-time hash comparison. The effective role is the lower
of the principal's role and the token's ceiling, so dashboards can
get read-only tokens. last_used_at is recorded. revokeApiToken().
* readSendGate()/setSendGate(): the live-send gate is now workspace
state, changed only by an admin through an audited action. Opening
it fully requires a written reason. The default is still an empty
allowlist.
- ExecutionDeps.sendGate may be a resolver; the engine defaults to
readSendGate, evaluated per action inside preflight, so a gate change
takes effect on the next send without restarting workers.
- listCampaigns(). campaignStatus() and resolveReview() now raise
NotFoundError instead of a plain Error (they surfaced as HTTP 500
before).
@splitin/outreach-server (new; hono + @hono/node-server)
- createApp(): thin routes. Each one authenticates, validates with
strict zod schemas and calls one application service; business rules
stay in core.
* /v1/me; campaigns: list, create, status, activation prepare/commit,
pause/resume, request a batch approval; approvals: list, decide
(operation hash required), revoke; imports: raw-body preview, then
commit with preview hash and idempotency key; tasks: list, record
outcome; review: list (payloads omitted), resolve; enrollments
pause/resume/stop; kill switches.
* POST /v1/webhooks/:accountId authenticates by provider signature,
not bearer token (401 forged, 404 unknown, 413 too large).
* GET /u/:token only renders a confirmation form, because mail
clients and scanners prefetch links; POST /u/:token performs the RFC
8058 one-click unsubscribe. Pages are noindex and escape everything.
* Errors map to 400 (zod, bad JSON), 403, 404, 409 (conflict, stale
import or preview, review state), 413, 422 (invalid playbook or
profile, rejected import, template), and 500 without leaking
messages or stacks. Every response has x-trace-id, no-store and
nosniff. Body limits: 256 KB JSON, 25 MB imports.
- startServer(): binds loopback by default and refuses any other
address unless allowPublic is passed (TLS belongs in front of it).
Tests (8 new, 154 total): identical rejection of missing, malformed,
tampered, revoked and expired tokens; role ceiling enforced; trace and
safety headers; a full HTTP flow (import preview/commit, create,
prepare, stale-hash 409, commit, worker send, status, list); strict
body validation, 422 with compile issues, 404, 413, unknown routes;
signed webhook accepted, forged 401, unknown account 404; GET
unsubscribe does not suppress, POST does, forged token 400; public bind
refused, loopback server answers /healthz over real HTTP.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Second part of milestone M7 of outreach-engine/BUILD_PLAN.md (§11.1,
§6.6). Adds the command line and closes the 4-process x 10k soak test
deferred from M3.
@splitin/outreach-cli (new; bin: outreach)
- Commands, each a thin call into core/import/server:
* setup: init (creates database and workspace; the caller becomes
admin; the gate starts closed), principal add, token create/revoke
(printed once), account add (secrets are env: references only),
gate show/allowlist/open (open needs a written reason), notify set,
audit verify (exit 3 if the chain is broken).
* content: profile add, import preview/commit (prints the exact commit
command), template add, playbook compile (exit 1 with issues),
suppress.
* campaigns: campaign create/list/status/prepare/activate/pause/
resume, approvals list/approve/reject/batch (approve and reject
need the operation hash), tasks list/done/skip, review list/resolve,
kill engage/release.
* runtime: worker --once | --loop --interval (SIGINT/SIGTERM finish the
current pass before exiting), and serve (loopback unless --public).
- Global options: --db / OUTREACH_DB, --workspace / OUTREACH_WORKSPACE,
--config (default ./outreach.config.mjs, which default-exports
{ adapters, manual? } so hosts plug in providers), --fake (in-memory
fake providers that never send anything), --json.
- Identity: OUTREACH_PRINCIPAL or cli:<os user>, resolved against
registered principals. There is no implicit admin; unknown callers
get "unknown principal".
- Unsubscribe links are enabled when OUTREACH_PUBLIC_URL and
OUTREACH_UNSUBSCRIBE_SECRET are set.
- Strict argument parsing (node:util parseArgs): unknown commands and
flags are usage errors with the command's usage line (exit 2);
failures exit 1 with the message and any validation issues; per-command
--help. Commands return their exit code, never setting
process.exitCode, and read the invocation's environment, never
process.env directly. This fixed `init` recording the wrong admin
when run in-process.
@splitin/outreach-fakes
- Fake message, thread and event ids include a per-instance prefix, so
several worker processes using fakes never collide on
(account, provider_message_id).
scripts/soak.mjs (npm run soak; now a CI step)
- Builds a real database through the built CLI (init, account add, gate
open), seeds 10,000 due actions, then runs rounds of 4 concurrent
`outreach worker --once --fake` processes until drained, plus a settle
pass. It asserts all 10,000 succeeded, no action has more than one
attempt, attempts equal actions, one outbound message per action, and
that work really was shared across processes. Locally: 6 rounds,
2,500 actions per process, about 13 s. The first run showed
`worker --once` caps a pass at 500 actions (by design), so the script
runs rounds rather than a single pass.
Tests (3 new, 157 total): a full in-process CLI run (init idempotent,
account, gate, profile, import preview/commit, template, playbook
compile, campaign create/prepare/activate, worker, status, audit
verify); help, unknown command and flag, unknown principal and invalid
playbook exit codes; gate-open reason required, token format, role
validation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
Completes milestone M7 of outreach-engine/BUILD_PLAN.md (§11.5): the
M7.0 Papr spike is recorded as a decision, and the engine ships as a
Papr Work bundle.
docs/adr/0004-papr-host-integration.md (M7.0 spike, Papr v2.6.18)
- Findings, from the Papr source:
* jobs get JOB_DIR/JOB_DB and PAPR_DB_* for registry databases;
replica (Turso-synced) registry databases are read-only to jobs, and
raw SQLite writes are blocked by guards, with writes going through a
one-statement-per-request proxy;
* node jobs with a package.json get `npm install --production`;
* keys arrive as env vars only when listed in requiredKeys or used as
${KEY};
* placement is local-only/local-preferred/cloud-preferred and cannot
be set in the bundle JobSpec;
* mini-apps reach server code through POST /api/app/backend/:action
handlers that receive PAPR_ACTION_PARAMS plus declared keys and
answer on stdout.
- Decisions:
* the engine keeps its own SQLite file (~/Papr/outreach/outreach.db),
because it relies on multi-statement BEGIN IMMEDIATE transactions
that the replica proxy cannot provide;
* the Papr app is a client of the loopback HTTP API through a
server-side handler, so the token never reaches the browser;
* a fixed action allowlist;
* Papr may host the worker for pilots, while production runs
`outreach worker --loop` and `outreach serve` as services (D2).
BUILD_PLAN.md §11.5 updated accordingly.
apps/papr (@splitin/outreach-app-papr, private)
- bundle/manifest.json: Papr portable bundle (schema 1.0.0) declaring
the app, the optional worker job and two server-only keys
(OUTREACH_API_URL, OUTREACH_API_TOKEN).
- Outreach Console mini-app (vanilla JS/CSS, light and dark,
reduced-motion aware, all engine data rendered with textContent only):
* "Next 24 hours": a timeline of everything that will leave, with a
chip saying why each item is held (approval, send window, daily
limit, kill switch, paused, account attention) and a per-item
"Pause campaign";
* "Waiting for your approval": exact recipients and subjects and the
content hash, with "Approve exactly this" or reject;
* "Needs a human": review items with "It went out" (asks for the
provider message id), "Not sent — send it", or drop;
* "Your manual touches": open profile, copy draft, "I did it", skip;
* campaign health bars;
* a sticky "Stop all sending" (workspace kill switch; resuming needs
an approver);
* auto-refresh every 30 s and a clear empty state when the engine is
unreachable.
- backend/outreach.mjs: a dependency-free Node handler mapping 7 fixed
actions (overview, approval-decide, task-outcome, review-resolve,
campaign-pause/resume, kill-switch) to fixed API routes. It validates
ids (ULID), hashes and enums before calling, uses a 10 s timeout,
passes the engine's authorization errors through, and speaks the
Papr contract (PAPR_ACTION + PAPR_ACTION_PARAMS in, one JSON line
out).
- jobs/outreach-worker: an optional node job running one
`outreach worker --once` pass against the engine's own database, with
providers from ~/Papr/outreach/outreach.config.mjs.
- skills/outreach.md: agent rules. The agent may draft, validate and
report; it must never activate, approve, commit imports, open the
gate, mark sends as sent or release kill switches, and never automate
social networks.
@splitin/outreach-core / -server
- listUpcoming() + GET /v1/actions/upcoming?hours=: what leaves in the
next N hours (max 168), including held items and why, with no bodies.
This backs the console's timeline.
Tooling: ESLint declares browser globals for the mini-app and Node
globals for bundle .mjs files.
Tests (7 new, 164 total; soak still green): the manifest validates
against a copy of Papr's BundleManifestSchema subset and every path
exists; the token is server-only and every backend action is wired to a
declared key; the browser code never fetches credentials or injects
HTML; all bundle JS parses. Against a live API server: the overview
returns everything the console renders (a pending campaign approval);
ids are validated before any request (a path-traversal id is refused);
unknown actions are refused; missing configuration is reported; a
viewer token cannot stop sending (403 passed through); and the
subprocess contract is verified.
Not verified here: bundle import, backend invocation and job npm install
on a live Papr install. That checklist is in apps/papr/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
CI failed on the M7 commits at `npm run build`. npm runs
`--workspaces` scripts alphabetically, so @splitin/outreach-cli
("cli" < "contracts" < "core") built its type declarations before the
packages it imports had any. Local runs passed only because stale
dist/ folders existed; a clean checkout reproduces it ("Cannot find
module '@splitin/outreach-contracts'").
scripts/build.mjs reads every workspace's dependencies (including
dev), orders the packages topologically (a cycle is an error, not a
silent misorder) and builds them one by one: contracts, fakes,
store-sqlite, core, import, server, cli. `npm run build` now uses it.
Verified from a clean state (all dist/ removed): build, lint,
typecheck, 164 tests, boundaries, secret scan, line limit and the
10k x 4-process soak all pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
splitintech
changed the title
…inst Papr's primitives Decision D5 (where email may legally go) is now enforced by the engine, and BUILD_PLAN.md §19 records D1–D5 after a second read of Papr Work (Papr-ai/paprwork@faf6de5) with one principle: build on what Papr already provides wherever its guarantees are enough, own only the rest. Jurisdiction policy (core, new domain/jurisdictions.ts) - A workspace records a rule per ISO 3166 country: allow, consent_required (only `consent` or `existing_relationship` pass; legitimate interest does not) or block, plus rules for unlisted and unknown countries. - Every policy carries its sign-off: who approved it and where the decision is recorded. It is admin-only and audited as `jurisdiction_policy_changed`; invalid country codes are a ZodError (HTTP 400). - Enforced twice. At activation the audience snapshot excludes recipients it does not permit (new `excluded.jurisdiction` count in the preview). Before every send a preflight check re-reads the policy and cancels the action (`jurisdiction_blocked:XX` or `jurisdiction_needs_consent:XX`), stopping the enrollment, so a policy tightened after approval still wins. - The live-send gate can no longer open without a signed-off policy. Allowlist mode (testing with your own addresses) does not need one. - `addContact` takes an optional jurisdiction. Importer - New `country` column per row: ISO codes or English country names, resolved from the runtime's CLDR region names (no table to maintain), plus a few aliases (UK, USA, England...). Pseudo-regions (EU, UN...) and unresolvable values reject the row rather than guess. - The profile-wide `jurisdiction` stays the fallback. A re-import fills in an unknown country but never overwrites a known one. CLI - `outreach jurisdiction show` - `outreach jurisdiction set --allow US,GB --consent DE,CA --block FR --default block --unknown allow --signed-off-by <name> --reference <memo>`; a country listed under two rules is an error. - The soak script records a policy before opening the gate. The Papr agent skill forbids `jurisdiction set` (a human decision). Decisions recorded in BUILD_PLAN.md §19 (evidence cited per row) - D1: mailbox adapters (Gmail, then Microsoft Graph, then Zoho Mail) behind a TokenSource port, so tokens can later come from Papr's planned server-side connectors; Zoho Campaigns rejected for cold outreach (permission-based lists only). - D2: the worker is a Papr job, local-only. Papr's cloud scheduler is live, but its synced databases' atomic write-batch reports `changes: 1` without executing guards, which breaks the claim step that at-most-once depends on. Cloud placement is pending answers from Papr. - D3: no tunnel. Polling for inbound, reply-STOP, and a footer unsubscribe served by a small public Papr app that writes one INSERT the engine verifies by HMAC. Slack via Socket Mode. - D4: Papr's Auth0 tenant for remote MCP (Papr has no MCP server or client in-repo; the M7 skill + CLI + backend actions are the Papr agent path). - D5: mechanism built; which countries and message class stay Legal's call. - Appendix A (the U2 issue draft) is now four concrete questions to the Papr maintainers: connector tokens, Auth0 audience, cloud-job single-flight and read-after-write, and query params for public backend actions. 181 tests (was 164), lint, typecheck, boundaries, secret scan, line limit and the 4-process x 10k soak all pass from a clean build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
…wait instead of failing The CI soak (Node 24, a1c1c21) failed with a worker exiting on "database is locked" about 3 s in, i.e. as the first round of worker processes opened the database, too early for the 5 s busy timeout to have expired. Cause: configure() ran `PRAGMA journal_mode = WAL` before `PRAGMA busy_timeout`. journal_mode takes a lock. When another process closes the last connection, SQLite checkpoints and removes the WAL under an exclusive lock; the soak's main process does exactly that right before spawning workers. With no timeout set yet, a concurrent open failed immediately instead of waiting. Reproduced outside the soak with one process repeatedly writing and closing as the last connection while three others open: 42 of 1,200 opens failed before this change, 0 of 3,600 after (three runs). No data was at risk: a worker that fails to open never claims or sends anything. The failure only aborted a pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
splitintech
changed the title
…: secrets First real provider per decision D1: @splitin/outreach-provider-email-gmail sends as the connected mailbox user through the Gmail API. It passes the email conformance kit over real HTTP, and the engine drives it end to end. Contracts - AccessTokenSource port + TokenError. Adapters take OAuth access tokens from a source, so the grant can be held by the engine (a refresh token behind secretRef) or later by a host such as Papr's planned server-side connectors (§19 D1) without adapter changes. Adapter (packages/outreach-provider-email-gmail) - MIME builder: RFC 2047 encoded words (<=45 bytes each), base64 UTF-8 bodies, multipart/alternative for HTML, CRLF. Refuses header injection, reserved headers and odd addresses before sending. Every message carries X-Outreach-Key: <idempotency key>. - Send outcome follows ADR 0002. Connection refused or DNS failure is a transient rejection (nothing reached Google). Timeouts, resets and 5xx are "unknown", never retried blindly. 429 and quota reasons map to rate_limited with Retry-After (1 h for daily limits); 401 maps to auth_expired and refreshes the token; domainPolicy maps to policy_blocked. - Reconcile: rfc822msgid: search first, then an authoritative walk of the SENT label (spam/trash included) matching X-Outreach-Key, which does not depend on search indexing. "absent" only after that walk and after settleMs (default 3 min) since the attempt. - Threading: reports the Message-ID Gmail actually stored (Gmail may replace ours), so replies still correlate. A deleted thread (404 on a threaded send, i.e. refused, not sent) is retried unthreaded. - Inbound via the History API with a cursor that resumes after the last processed record within a per-poll budget. Our own SENT/DRAFT mail is skipped. An expired cursor falls back to recent inbox mail (deduplicated by event id). - Bounces: status, recipient and original Message-ID are parsed from the message/delivery-status and rfc822-headers parts. Without a DSN part the status stays empty, so the classifier routes the bounce to review instead of suppressing anyone on a guess. - Health: ok only if the connected mailbox equals the account's sender address; a revoked grant reports reauth_required. - Purposes default to manual_correspondence only. automated_outreach is declared explicitly in outreach.config.mjs after a terms review (§19 D1). - authorizeGoogle: installed-app loopback flow (RFC 8252). PKCE S256, random state, 127.0.0.1-only listener, one callback. It refuses grants missing a required scope and reads the mailbox address from Gmail itself. CLI - outreach account connect gmail --client-id ... --client-secret-env NAME [--login-hint] [--out]: runs the flow and writes the grant to an owner-only (0600, dir 0700) file. The client secret is read from the environment, never argv. A login-hint mismatch saves nothing. - file:PATH secret references alongside env:NAME; a file readable by group or others is refused. - Commands can now be three words; stderr notices are shown even with --json. Fakes and tests - FakeGmailServer (outreach-fakes): the Gmail API subset plus Google's authorize/token endpoints over real HTTP. It can drop the connection before or after accepting, rewrite Message-IDs, lag search, page and expire history, and verify PKCE. - 20 new tests: the full conformance kit over HTTP, MIME, error mapping, token refresh/revocation, reconcile (found via SENT despite search lag; absent only after settle), inbound paging, resume, cursor recovery, DSN parsing, health, OAuth (happy path, missing scope, forged state) and CLI connect with 0600 checks. - Mutation checks: disabling the settle guard or the SENT walk each fails a test. - gmail.e2e: the engine over HTTP. A send whose response is lost after Gmail stored it (and search lags) goes executing -> uncertain -> reconciling -> succeeded with exactly one attempt. Ada's threaded reply (to Gmail's rewritten Message-ID) stops her follow-up. Grace gets hers. The audit chain verifies. Docs: package README (setup: Internal consent screen, Desktop client, connect, account add, volume guidance), README table, BUILD_PLAN M8 row; the stale "nothing implemented yet" status line is fixed. 202 tests (was 181), lint, typecheck, boundaries (10 packages), secret scan, line limit and the 4-process x 10k soak pass from a clean build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
…ith no public URL
Decision D3 settled on "no tunnel or VM", but with `unsubscribe: reply`
(the policy that implies) the engine sent no List-Unsubscribe header at
all. Gmail and Outlook therefore showed no Unsubscribe button, and
recipients had to find the footer.
Change (core, domain/materialize.ts)
- Every message on an adapter with custom headers now carries
List-Unsubscribe: <https-link>, <mailto:SENDER?subject=unsubscribe>
(the https link only when the playbook uses `link` and a URL is
configured).
- The mailto target is the sending mailbox, which the engine already
polls. A click on the provider's Unsubscribe button becomes a
message with subject "unsubscribe"; the deterministic classifier
marks it opt_out; the processor suppresses the address globally and
stops the enrollment. Suppression is by address, so it holds even
though the message is not in the thread (weak correlation, flagged
for a person to confirm).
- List-Unsubscribe-Post (RFC 8058 one-click) is only emitted when the
link is the engine's own POST /u/:token endpoint. New
UnsubscribeConfig.oneClick (default true) lets a host with a page
URL (e.g. a hosted confirmation page) turn it off; a provider's
one-click POST to a static page would silently do nothing, which
is a compliance failure.
Tests
- unsubscribe-headers.test: reply policy gives mailto only and no
one-click; a page link with oneClick:false keeps the mailto fallback
and no Post header; the link policy still advertises one-click.
- gmail.e2e now covers three recipients over the Gmail adapter:
- Ada replies in the thread (stopped as replied).
- Grace presses Gmail's Unsubscribe, i.e. a mailto message outside
any thread (opted_out and suppressed).
- Linus is the only one who gets the follow-up.
- The lost send is still reconciled with exactly one attempt per
action.
- Existing header assertions were updated for the added mailto target.
BUILD_PLAN.md §19 D3 records the built mechanism; the Papr-hosted
footer page is now optional rather than required.
204 tests, lint, typecheck, boundaries, secret scan, line limit and the
soak pass from a clean build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
…r; file: refs accepted
Two real gaps closed, plus a bug in the previous commit's workflow.
1. Accounts never recovered. A send failing with auth_expired,
auth_revoked or forbidden marks the account reauth_required or
unhealthy, and preflight then defers every send on it. Nothing ever
set it back, so after reconnecting Gmail that account stayed dead.
2. Nothing called AccountPort.health() (BUILD_PLAN.md §6.5), and
mailbox polling swallows errors, so a revoked grant failed silently.
Health checker (core, domain/health.ts)
- checkAccountHealth asks the adapter and records the answer;
checkDueAccountHealth re-checks every account whose last check is
older than healthIntervalMs (default 15 min). It now runs first in
every worker pass, so a recovered or broken account is treated
accordingly in the same pass. Accounts whose adapter is not
registered in this process are skipped (preflight already holds
their sends for review).
- Recovery rule (nextHealth): an unhealthy or reauth_required account
returns only on an affirmative "ok" from the provider. A check that
throws or times out, or a "degraded" answer, never re-enables
sending on a revoked account. A healthy account whose check fails
becomes degraded (sending continues).
- Changes are audited (health_changed {from, to, detail}). When a
notification account is configured, a bad transition or a recovery
enqueues a notice through that other account (never through the
broken one), e.g. "Reconnect sending account sam@...".
- registerProviderAccount now checks health immediately, so a wrong
mailbox (the Gmail adapter compares the connected address with the
sender) or a broken grant shows at registration, not at the first
send.
Bug fix
- registerProviderAccount rejected file: secret references, although
`outreach account connect gmail` (previous commit) tells the operator
to register with --secret file:PATH. It now accepts env:NAME,
keychain:NAME and file:PATH.
CLI
- outreach account check <id>: ask the provider now and record the
result.
Tests (health.test.ts, 10)
- The nextHealth table.
- An auth_expired send marks the account reauth_required and holds the
send; an inconclusive check keeps it held; an affirmative ok
releases it and it is delivered once.
- The interval inside runOnce (report.health) and the notification
routed via the separate notify account.
- The health check at registration.
- file: refs accepted and plain values rejected.
214 tests, lint, typecheck, boundaries, secret scan, line limit and the
soak pass from a clean build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
…ok adapter
Pure refactor with no behaviour change. The Outlook (Microsoft Graph)
adapter needs exactly what the Gmail adapter already built, so those
parts move to shared places instead of being copied.
- MIME builder -> @splitin/outreach-contracts (mime.ts). It is pure,
depends on nothing but providers.ts, and both adapters must refuse
header injection the same way. Graph can only carry List-Unsubscribe
if we send raw MIME, since internetMessageHeaders accepts X- headers
only.
- New @splitin/outreach-provider-kit (depends on contracts only):
- jsonRequest + ProviderNetworkError: one HTTP call, where a failure
carries whether the server was reached. A connection that was
refused or failed DNS is the only proof that nothing was
processed. Also parseRetryAfter (seconds or an HTTP date).
- loopbackAuthorize: the RFC 8252 installed-app code flow. It uses
PKCE S256 and a random state, accepts exactly one callback, and
binds its listeners to loopback only. A `localhost` redirect
(Microsoft's registration) listens on both 127.0.0.1 and ::1.
Callers supply the token exchange.
- DSN helpers: parseDeliveryStatus (RFC 3464 fields), parseRawDsn
(multipart/report in raw MIME, base64/QP parts), looksLikeBounce
and decodeHtmlEntities.
- Boundary rule: provider-kit depends on contracts only, and provider
adapters may depend on contracts + provider-kit.
- The Gmail adapter now uses the kit for HTTP, OAuth and DSN parsing.
Its public API is unchanged apart from GmailNetworkError, which is
now the kit's ProviderNetworkError (unreleased, no users).
Tests: 5 new kit tests (refused connection, Retry-After, raw DSN with
a base64 delivery-status part, DSN without one, a loopback callback on
`localhost` with PKCE). All existing Gmail, CLI and e2e tests pass
unchanged. 219 tests; lint, typecheck, boundaries (11 packages), secret
scan and line limit pass from a clean build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
…-token write-back
Second mailbox adapter per decision D1:
@splitin/outreach-provider-email-outlook sends as the connected
Exchange Online mailbox through Microsoft Graph. It passes the email
conformance kit over HTTP, and the engine drives it end to end.
Design (packages/outreach-provider-email-outlook)
- Draft, then send. The message is created from the shared MIME
builder (POST /me/messages with base64 MIME), since Graph's
internetMessageHeaders only takes X- headers and List-Unsubscribe
would be lost. Then POST /me/messages/{id}/send.
- Creating a draft never sends anything, so every failure before the
send call (unreachable, 5xx, refused) is a safe transient
rejection. Only the send call can be "unknown". A refused send
discards its draft; an unknown one keeps it until reconciliation
decides.
- Reconcile: Sent Items filtered by our Message-ID, then a newest-first
walk matching X-Outreach-Key, because Exchange may replace the
Message-ID. "absent" only after the walk and after settleMs, and then
the orphaned draft is deleted so nobody sends it by hand later.
- The provider message id is internetMessageId. The conformance kit
caught that Exchange gives the Sent Items copy a new item id, so
the Graph item id is not stable across the send.
- Inbound: the Inbox polled by receivedDateTime with a 2-minute overlap
(deduplicated by event id); the first poll starts from now. Exchange
NDRs are read as raw MIME (/$value) and parsed with the kit's DSN
parser; without a delivery-status part a bounce goes to review.
- Errors: 429, ApplicationThrottled and quota codes map to rate_limited
(Retry-After, or 1 h for quotas); ErrorMessageSubmissionBlocked maps
to policy_blocked; other 403s to forbidden; ErrorInvalidRecipients to
invalid_recipient.
- Health: ok only if mail or userPrincipalName equals the sender
address; a revoked grant reports reauth_required.
- authorizeMicrosoft: the desktop-registration code flow with PKCE on
http://localhost (the kit listens on 127.0.0.1 and ::1). It checks
that Mail.Send, Mail.ReadWrite and User.Read were granted and reads
the mailbox from Graph.
Rotating refresh tokens
- Microsoft issues a new refresh token on every refresh. Without
persisting it the grant lapses (~90 days).
- SecretResolver gains an optional put(ref, value). The token source
writes the rotated token back through it, best effort. The CLI
implements put for file: secrets as an atomic, owner-only (0600)
replace; env: secrets refuse writes with a clear message.
CLI
- outreach account connect outlook --tenant ... --client-id ...
[--client-secret-env] [--login-hint] [--out]. The grant-file writing
is shared with the Gmail command.
Fakes and tests
- FakeGraphServer: identity authorize and token (PKCE, rotation),
/me, MIME drafts, send moving the message to Sent Items under a new
id (optionally without our Message-ID), OData filters and paging,
/$value, forced outcomes.
- outlook.test (9): conformance kit, MIME headers intact and stored
Message-ID reported, a lost send found by the idempotency header,
absent only after settling with the orphan draft deleted, error
mapping with draft cleanup, rotation persisted (the old token then
revoked), a revoked grant, inbox replies and parsed NDRs, health and
purposes, the OAuth flow.
- Mutation checks: disabling the settle guard or header matching each
fails a test.
- outlook.e2e: a lost send is reconciled with exactly one attempt per
action; rotation is persisted; Ada's reply stops her follow-up;
Grace's NDR (correlated via the original Message-ID) marks her
bounced and suppressed as hard_bounce; Linus alone gets the bump.
The audit chain verifies.
- CLI connect outlook plus file: put (atomic, 0600, env: refused).
Docs: Outlook package README (Entra app registration, permissions,
connect, add, volume guidance); README and BUILD_PLAN M8 row.
230 tests, lint, typecheck, boundaries (12 packages), secret scan,
line limit and the soak pass from a clean build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
@splitin/outreach-notify-slack implements the notify port only. The engine routes its notices through it: replies, opt-outs, bounces, complaints and account-health changes. Per the M6 design they carry identifiers and routing only, never message bodies. - Mode by secret. An incoming-webhook URL (https://hooks.slack.com/services/...) posts to that webhook's channel. A bot token (xoxb-..., chat:write) posts with chat.postMessage to the channel in the account's externalAccountId. Anything else is refused as auth_revoked. - Content is data, never markup: Block Kit with every value escaped (&, <, >), so an engine string cannot ping <!channel> or render a link. Link buttons appear only for https URLs. There is a plain-text fallback. - Outcomes follow ADR 0002. A refused connection is a transient rejection and is retried. A dropped connection or a 5xx is unknown, and the engine never retries an unknown notification. HTTP 429 and ratelimited map to rate_limited with Retry-After. invalid_auth, token_revoked and similar map to auth_revoked. channel_not_found, not_in_channel, is_archived and similar map to forbidden. Other codes map to content_rejected. - Health: bot tokens via auth.test (a revoked token reports reauth_required). Incoming webhooks report ok with a note that they cannot be verified without posting. - Provider kit: textRequest can now send a JSON body and returns Retry-After, since Slack webhooks answer plain "ok". Notify adapters may depend on the kit (boundary rule). Tests (4, against a local fake Slack over HTTP): webhook posting with escaping; bot posting to the account channel plus error-code mapping and auth.test; unknown only when the post may have landed (dropped connection) vs a transient rejection on a refused connection; bad secrets refused and non-https links dropped. The secret scanner caught a realistic-looking fake bot token in the first draft of the test; the fixture was shortened rather than the scanner weakened. Docs: package README (modes, setup, behaviour), README table, BUILD_PLAN M8 row (remaining: Zoho Mail adapter, Slack control app in slack-agent-hq). 234 tests, lint, typecheck, boundaries (13 packages), secret scan, line limit and the soak pass from a clean build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv
splitintech
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new product folder,
outreach-engine/: a provider-neutral outreach orchestration engine (@splitin/outreach-*, MIT), built milestone by milestone fromoutreach-engine/BUILD_PLAN.md. Each milestone is its own commit with a detailed message.What's in this PR
df178bbBUILD_PLAN.md: architecture, schema, state machine, surfaces, security, tests, M0–M108122da8cd931419e0b865fd8a75eexecutingbefore any provider call, uncertain-outcome reconciliation, rate budgets, kill switches, review queue0724a4678af15b22240b8e106011a3e086aoutreachCLI + 4-process × 10k soak test in CI62f96d32d51f39a1c1c219970502busy_timeoutbeforejournal_mode: concurrent opens wait instead of failing with "database is locked"3b5c49foutreach account connect gmail(loopback OAuth + PKCE),file:secrets6abd4f0List-Unsubscribeon every message: provider Unsubscribe buttons work with no public URL (D3); one-click only for the engine's own endpointf19e3b1ok; health checked at registration;file:refs accepted8c6853f@splitin/outreach-provider-kit(HTTP outcome classification, loopback OAuth, DSN parsing); MIME builder moved to contracts6d0e3d7outreach account connect outlook2c18ab7chat.postMessage, escaped Block Kit)Guarantees under test
234 tests plus the soak test. Gmail and Outlook are tested against local fakes of their APIs over real HTTP, not against live Google or Microsoft accounts.
Not in this PR
Zoho Mail adapter and the Slack control app (M8 remainder, the latter a separate PR in
slack-agent-hq), M9 (MCP), M10 (release). First live sends to the allowlist should confirm provider details the fakes model from documentation. The Papr bundle still needs a first run on a live Papr install. Which countries the first campaign may target is Legal's call (D5).Test plan
outreach-engineCI (build, lint, typecheck, test, boundaries, secret scan, line limit, soak) green on Node 22 and 24 at2c18ab7dependency-reviewis red on every PR in this repo until Dependency graph is enabled in settings (see the comment below).🤖 Generated with Claude Code
https://claude.ai/code/session_01MbDW8kfMcxUuZ3WBtaDKBv