test: add an opt-in emulate.dev end-to-end tier for the Slack adapter by ThomasK33 · Pull Request #149 · coder/chat

ThomasK33 · 2026-10-30T10:23:25Z

Closes #147

What this adds

An opt-in end-to-end test tier that runs the Slack adapter against emulate.dev (vercel-labs/emulate, Apache-2.0), pinned to emulate@0.12.3.

mise run emulate starts Slack on :4710 and Linear on :4711 with testdata/emulate/seed.yaml (a workspace, a user, a bot from a seeded OAuth app, a channel, and a Linear team). mise run test:emulate runs the tests against it.
adapters/slack/emulate_test.go has two tests. They skip unless CHAT_EMULATE_SLACK_URL is set, like TestLive.
- TestEmulateSlackEventReply: a user message is posted to emulate, then a signed Events API message event goes to the adapter's HTTP handler. The mention handler replies in the thread, and conversations.replies on emulate must show the bot reply.
- TestEmulateSlackBlockAction: the bot posts a button. emulate's block-action simulator clicks it and sends the signed interaction to the seeded interactivity URL (127.0.0.1:4799). The interaction handler must get the action ID, the button value, and the clicking user.
.github/workflows/emulate.yaml runs the tier in its own Emulate workflow, so a failure there does not turn CI red. Every action is pinned by full SHA. ci.yaml is unchanged, so Node stays out of the test jobs.
"Emulator Tests" in CONTRIBUTING.md explains how to run it and lists the known gaps. The Slack and Linear package READMEs point to it.

No new Go module dependencies. No ADR: this is test tooling, and the owner decision (D3, Option A) is recorded in #147.

Scope changes from the request

No Linear tests. emulate's Linear schema differs from Linear's in four ways that stop every adapter reply (missing User.organization, no CommentCreateInput.parentId, a different AgentActivityCreateInput, and different webhook shapes). emulate 0.13.0 has the same code. The owner chose to ship without Linear tests. test: add Linear to the emulate.dev tier once emulate's Linear schema matches Linear #148 tracks adding them when emulate matches.
The event test sends the event_callback itself. emulate 0.12.3 dispatches Events API callbacks only to subscriptions registered through its internal JavaScript dispatcher. Neither the CLI nor the seed can register one. So the test posts the user message to emulate, then signs and sends the event for that message. The reply and conversations.replies still go through emulate. This gap is in the docs.

Local evidence

Unset variable: the new tests skip.

$ go test -run TestEmulate -count=1 -v ./adapters/slack/
    emulate_test.go:144: set CHAT_EMULATE_SLACK_URL and run `mise run emulate` to run the emulate.dev Slack tests
--- SKIP: TestEmulateSlackEventReply (0.00s)
    emulate_test.go:222: set CHAT_EMULATE_SLACK_URL and run `mise run emulate` to run the emulate.dev Slack tests
--- SKIP: TestEmulateSlackBlockAction (0.00s)
PASS

Every module, variable unset: mise run vet and mise run test pass (exit 0), the same as on main.

With mise run emulate running in another terminal:

$ GOFLAGS=-race mise run test:emulate
[test:emulate] $ go test -run TestEmulate -count=1 -v ./adapters/slack/
--- PASS: TestEmulateSlackBlockAction (0.01s)
--- PASS: TestEmulateSlackEventReply (0.01s)
PASS
ok  	github.com/coder/chat/adapters/slack	1.032s

Red checks

I dropped the reply: the mention handler returned nil without ev.Thread.Post. TestEmulateSlackEventReply failed:

emulate_test.go:209: conversations.replies has no bot reply "pong: <@UAF4CA824C> ping 1793355633532971594" in thread 1793355633.000016: [{Type:message ... User:U6B1A81290 BotID: ...}]
--- FAIL: TestEmulateSlackEventReply (0.01s)

I gave the adapter a wrong signing secret. emulate's signed click got a 401, and TestEmulateSlackBlockAction failed:

emulate_test.go:270: simulate block action: slack: _slack/simulate/block-actions failed: interactivity_delivery_failed
--- FAIL: TestEmulateSlackBlockAction (0.01s)

I restored the file after each check (byte-identical copy), and both tests passed again.

Known gaps (also in CONTRIBUTING.md)

The fakes (slackapitest, linearAPIServer) and the live tests (TestLive*) stay the source of truth for Slack's exact rules.

No Socket Mode: apps.connections.open returns 404 (checked).
No slash commands.
No Events API request URL from the CLI or seed (see above).
emulate accepts a JSON body for conversations.history and conversations.replies (checked: ok:true). Real Slack returns invalid_arguments, so this tier would have missed bug(adapters/slack): HistoryReader fails on live Slack — conversations.history/replies sent as JSON #98.
Scope checks are relaxed unless strict_scopes: true. The seed leaves them relaxed.
The Linear schema differences listed in test: add Linear to the emulate.dev tier once emulate's Linear schema matches Linear #148.

npx emulate@0.12.3 pins emulate itself, but npm resolves its three runtime dependencies (commander, picocolors, yaml) by semver range at install time.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Add `mise run emulate` (emulate@0.12.3) with a seed in testdata/emulate/seed.yaml, and TestEmulate* tests in adapters/slack that skip unless CHAT_EMULATE_SLACK_URL is set. They cover a signed Events API message event through the adapter's HTTP handler to a thread reply that conversations.replies shows, and a simulated button click through the adapter's HTTP handler to the interaction handler. A separate Emulate workflow runs the tier, so it cannot turn CI red. CONTRIBUTING.md explains how to run it and lists emulate's known gaps, including the Linear schema differences that block Linear tests (#148). Closes #147 _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

ThomasK33 · 2026-10-30T10:23:37Z

@codex review

ThomasK33 · 2026-10-30T10:23:43Z

@codex security review

chatgpt-codex-connector · 2026-10-30T10:23:52Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T10:25:18.948544Z	`283fb70`	Manual request
🔒 Security Review	✅ Completed 2026-10-30T10:25:43.607274Z	`283fb70`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-10-30T10:25:17Z

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 283fb708a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-30T10:25:42Z

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 283fb708a4

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

ThomasK33 · 2026-10-30T10:31:15Z

Readiness record for head 283fb708a4c4b1aec20d55c00db271f60222fd70:

CI on this head: test (pinned), test (minimum), and emulate (slack) passed.
Codex code review: "Didn't find any major issues" on 283fb708a4. Codex security review: "No security issues were found" on 283fb708a4.
Review rounds: 1 of 6. Review threads: 0.
Independent advisor (clean context): ready with tracked follow-ups. The Linear tests are tracked in test: add Linear to the emulate.dev tier once emulate's Linear schema matches Linear #148. The other notes are documented in the PR body (unlocked npm transitive dependencies, fixed port 4799).
Decision: ready for review and merge. I stop here and do not merge.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

ThomasK33 merged commit 902f4ee into main Oct 30, 2026
3 checks passed

Navigation Menu

Repository navigation

test: add an opt-in emulate.dev end-to-end tier for the Slack adapter - #149

test: add an opt-in emulate.dev end-to-end tier for the Slack adapter#149
ThomasK33 merged 1 commit into
mainfrom
test/emulate-tier

ThomasK33 commented Oct 30, 2026

Uh oh!

ThomasK33 commented Oct 30, 2026

Uh oh!

ThomasK33 commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026

Uh oh!

ThomasK33 commented Oct 30, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ThomasK33 commented Oct 30, 2026

What this adds

Scope changes from the request

Local evidence

Red checks

Known gaps (also in CONTRIBUTING.md)

Uh oh!

ThomasK33 commented Oct 30, 2026

Uh oh!

ThomasK33 commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026

🛡️ Codex Security Review

Uh oh!

ThomasK33 commented Oct 30, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

chatgpt-codex-connector Bot commented Oct 30, 2026 •

edited

Loading