Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/emulate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# The emulate.dev test tier: the Slack adapter against a local emulator. It
# lives in its own workflow so that a failure here does not mark CI red. See
# "Emulator Tests" in CONTRIBUTING.md.
name: Emulate

on:
push:
branches: [main]
pull_request:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
emulate:
name: emulate (slack)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install Node
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 22

- name: Install mise-pinned toolchain
uses: jdx/mise-action@8418fedb1110381a0d3ba51a7a415d7c1b884550 # v5.2.2

- name: Start emulate
run: |
nohup mise run emulate > "$RUNNER_TEMP/emulate.log" 2>&1 &
for _ in $(seq 90); do
if curl -fsS -X POST -H 'Authorization: Bearer xoxb-emulate-bot' \
http://127.0.0.1:4710/api/auth.test > /dev/null 2>&1; then
exit 0
fi
sleep 1
done
echo "emulate did not start within 90 seconds"
cat "$RUNNER_TEMP/emulate.log"
exit 1

- name: Test
run: mise run test:emulate
env:
GOFLAGS: -race

- name: Show emulate log
if: always()
run: cat "$RUNNER_TEMP/emulate.log"
46 changes: 46 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,52 @@ Some docs are checked by `documentation_test.go`:
- A few key phrases must stay in the README, reference, explanation, and
package docs. If you reword one, update the test in the same change.

## Emulator Tests

An opt-in tier runs the Slack adapter end to end against
[emulate.dev](https://emulate.dev) (`vercel-labs/emulate`, pinned to 0.12.3),
a local, stateful emulator of the Slack and Linear APIs. It covers flows that
the fakes do not cover end to end:

- a signed Events API message event, through the adapter's HTTP handler and
the mention handler, to a thread reply that `conversations.replies` shows;
- a button click from emulate's block-action simulator, through the adapter's
HTTP handler, to the interaction handler.

You need Node.js 22 or later for `npx`. Run from the repository root:

```sh
mise run emulate # terminal 1: Slack on :4710, Linear on :4711
mise run test:emulate # terminal 2: sets CHAT_EMULATE_SLACK_URL, runs TestEmulate*
```

The seed is `testdata/emulate/seed.yaml`. The block-action test listens on
`127.0.0.1:4799`, the seeded interactivity URL. Without
`CHAT_EMULATE_SLACK_URL`, the tests skip and `go test ./...` is unchanged. The
`Emulate` workflow runs the tier on each pull request, separately from `CI`.

emulate is not the source of truth for Slack's rules. The fakes
(`adapters/slack/slackapi/slackapitest` and `linearAPIServer` in
`adapters/linear/linear_test.go`) and the live tests (`TestLive*` in
`adapters/slack/slackapi/live_test.go`) are. Known gaps in emulate 0.12.3:

- No Socket Mode: `apps.connections.open` returns 404.
- No slash commands.
- No Events API request URL in the CLI or the seed. The event test posts the
user message to emulate, then signs and sends its `event_callback` itself.
- emulate accepts a JSON body for `conversations.history` and
`conversations.replies`. Real Slack returns `invalid_arguments`, so this tier
would have missed #98.
- Scope checks are relaxed unless the seed sets `strict_scopes: true`. This
seed leaves them relaxed.
- No Linear tests. emulate's Linear schema differs from Linear's, so the
adapter cannot reply: `User` has no `organization` (single-install `Init`
fails), `CommentCreateInput` has no `parentId`, `AgentActivityCreateInput` is
`{sessionId, type, body}` instead of `{agentSessionId, content}`, Comment
webhooks have no `data.user`, and `AgentSessionEvent` webhooks carry the
session in `data` instead of `agentSession`. The seed keeps a Linear team for
manual checks. Issue #148 tracks Linear tests.

## Validating Adapter Changes Live

CI has no live Slack or Linear tests. Before you claim that a change works
Expand Down
1 change: 1 addition & 0 deletions adapters/linear/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,4 @@ The adapter owns its own rate-limit retry (`RetryPolicy`) and Linear-specific ex
## Verification

- Run: `go test ./adapters/linear/...`
- The emulate.dev tier has no Linear tests yet, because emulate's Linear schema differs from Linear's. See "Emulator Tests" in `CONTRIBUTING.md` and issue #148.
1 change: 1 addition & 0 deletions adapters/slack/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,3 +109,4 @@ The code has compile-time assertions for `chat.Listener`, `chat.HistoryReader`,
## Verification

- Run: `go test ./adapters/slack/...`
- Emulator tier (opt-in): `mise run emulate`, then `mise run test:emulate`. It runs `TestEmulate*` in `emulate_test.go`, which skip without `CHAT_EMULATE_SLACK_URL`. See "Emulator Tests" in `CONTRIBUTING.md`. [emulate_test.go:newEmulateSlack]
Loading
Loading