Skip to content

feat: classify onboarding requests mentioned to the Slack bot (CM-1841) - #4884

Open
ulemons wants to merge 7 commits into
mainfrom
feat/CM-1841-slack-thread-bot
Open

ulemons wants to merge 7 commits into
mainfrom
feat/CM-1841-slack-thread-bot

Conversation

@ulemons

@ulemons ulemons commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • New POST /v1/slack/events endpoint (Slack Events API): signature check, url_verification challenge, immediate ack, bot messages ignored. Events are deduplicated by event_id on Redis (5 minutes), so a Slack retry is handled only if the first delivery was not; if Redis is down the event is handled anyway. Only mounted when CROWD_SLACK_SIGNING_SECRET is set, same as the interactivity route.
  • On an app_mention, the bot runs the same classifier as the discovery worker (LLM parser, PCC, CDP) and replies in the thread with the outcome and the step of the flow reached. Slack links (<url|label>) and the mention are normalised before parsing, and the permalink points to the message with the mention. Untrusted values in the reply are escaped so they cannot trigger <!channel>-style mentions, and a reply Slack does not deliver is logged.
  • Read-only for now: every reply is labelled [DRY RUN] and nothing is written or onboarded. Interactive steps and actions come in the next PRs.
  • The Slack alert builder moved from the worker to @crowd/project-onboarding (new dependency on @crowd/slack, no cycle) so the worker alerts and the bot replies use the same text.

Slack app setup needed before this works

  • Event Subscriptions on, Request URL <api>/v1/slack/events, bot event app_mention.
  • Bot scope app_mentions:read (and chat:write, already used), then reinstall the app and invite the bot to the channel.
  • Env on the backend: CROWD_SLACK_BOT_TOKEN, CROWD_SLACK_SIGNING_SECRET, plus Bedrock and Snowflake credentials.

pnpm-lock.yaml matches pnpm install --lockfile-only.

Test plan

  • vitest (backend slack, worker, lib), tsc (backend and worker), pnpm lint --deny-warnings, pnpm format
  • Configure the Slack app, mention the bot in a test channel and check the thread reply

Copilot AI balanced review requested due to automatic review settings October 2, 2026 11:05
@cursor

cursor Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Introduces a new unauthenticated Slack ingress path and runs the full classifier (Snowflake/Bedrock) on mentions; mitigations include signature checks, deduplication, and dry-run-only replies, but PCC SQL changes can alter match ranking.

Overview
Adds Slack Events API support via POST /v1/slack/events, mounted next to interactivity (before DB/tenant middleware) when a signing secret is configured. The handler verifies signatures, answers url_verification, acks within Slack’s window, deduplicates deliveries with Redis (SET NX, 15‑minute TTL), and on user app_mention asynchronously runs the same onboarding request classifier as the discovery worker, replying in the thread.

Bot replies are read-only: always [DRY RUN], with mention/link/HTML normalization on input, sanitized failure text, escaped untrusted fields in alert copy, and shared buildRequestClassificationAlert logic moved into @crowd/project-onboarding (worker updated to import it). pccLookup rewrites the Snowflake leaf-project check from a NOT IN subquery to a LEFT JOIN on parent IDs.

Reviewed by Cursor Bugbot for commit 8ff6cb6. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Retry handling, permalink selection, unescaped mrkdwn, and silent reply failures need correction.

Review effort: Balanced
Findings: 4 Medium severity

Open (4)
What changed in this PR

Adds a signed Slack Events endpoint that classifies onboarding requests mentioned to the bot and returns dry-run results in-thread.

Changes:

  • Handles and validates Slack app_mention events.
  • Reuses shared onboarding classification and alert formatting.
  • Adds focused endpoint and response-building tests.
File Description
services/​libs/​project-onboarding/​tsconfig.json References the Slack library.
services/​libs/​project-onboarding/​src/​requestClassificationAlert.ts Exposes the shared alert contract.
services/​libs/​project-onboarding/​src/​requestClassificationAlert.test.ts Updates alert test imports.
services/​libs/​project-onboarding/​src/​index.ts Exports alert utilities.
services/​libs/​project-onboarding/​package.json Adds the Slack dependency.
services/​apps/​automatic_projects_discovery_worker/​src/​activities/​requestClassification.ts Uses the shared alert type.
services/​apps/​automatic_projects_discovery_worker/​src/​activities/​activities.ts Uses shared alert builders.
pnpm-lock.yaml Records the workspace dependency.
backend/​src/​services/​slack/​requestClassificationBot.ts Classifies mentions and builds replies.
backend/​src/​services/​slack/​requestClassificationBot.test.ts Tests normalization and replies.
backend/​src/​api/​slack/​index.ts Mounts the signed Events route.
backend/​src/​api/​slack/​events.ts Validates, acknowledges, and dispatches events.
backend/​src/​api/​slack/​events.test.ts Tests event handling behavior.
backend/​src/​api/​index.ts Registers the Events route early.
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread backend/src/api/slack/events.ts
Comment thread backend/src/api/slack/events.ts Outdated
return Message()
.blocks(
Section({ text: `*${buildRequestClassificationAlertTitle(alert)}*` }),
...sections.map(({ title, text }) => Section({ text: title ? `*${title}*\n${text}` : text })),
Comment thread backend/src/services/slack/requestClassificationBot.ts Outdated
@ulemons ulemons self-assigned this Oct 2, 2026
@ulemons
ulemons force-pushed the feat/CM-1841-extract-request-classifier branch from 388dca8 to a97359b Compare October 2, 2026 12:51
Base automatically changed from feat/CM-1841-extract-request-classifier to main October 2, 2026 13:27
Copilot AI balanced review requested due to automatic review settings October 2, 2026 13:31
@ulemons
ulemons force-pushed the feat/CM-1841-slack-thread-bot branch from ac8951e to d6305b4 Compare October 2, 2026 13:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Slack retry deduplication and oversized Block Kit replies can currently cause duplicate processing or failed responses.

Review effort: Balanced
Findings: 3 Medium severity

Open (3)
Resolved since last review (3)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

Comment thread backend/src/api/slack/eventDeduplication.ts Outdated
Comment thread backend/src/services/slack/requestClassificationBot.ts Outdated
Copilot AI balanced review requested due to automatic review settings October 2, 2026 13:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Slack entity normalization and the deduplication TTL can respectively misclassify names and allow duplicate replies.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file
Previously missed (1)

In code that hasn't changed since last review

Medium severity Decode Slack HTML entities before project name matching

backend/​src/​services/​slack/​requestClassificationBot.ts:28

Slack escapes literal &, <, and > in incoming event text as &amp;, &lt;, and &gt;. Because this normalization only removes Slack's structural mention/link markup, a project such as R&D reaches the classifier as R&amp;D and can fail PCC/CDP name matching. Decode Slack's three entities after unwrapping links, and add a regression case for an ampersand-containing project name.

Copilot AI balanced review requested due to automatic review settings October 2, 2026 13:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The Redis deduplication TTL is 15 minutes, contradicting the documented 5-minute behavior.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

import type { RedisClient } from '@crowd/redis'

const EVENT_KEY_PREFIX = 'slack_event'
const EVENT_TTL_SECONDS = 15 * 60
Copilot AI balanced review requested due to automatic review settings October 2, 2026 14:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

User-facing exception leakage and the lack of a per-user or daily LLM quota should be addressed.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file
Previously missed (2)

In code that hasn't changed since last review

Medium severity Avoid exposing internal exception details in Slack responses

backend/​src/​services/​slack/​requestClassificationBot.ts:49

classifyOnboardingRequest puts arbitrary caught exception text into an ambiguous resolution (services/libs/project-onboarding/src/classifyRequest.ts:46-49), and this user-facing path renders that reason verbatim. Snowflake, database, or Bedrock failures can therefore expose internal service details in the Slack channel. Keep the detailed failure in server logs and substitute a generic message for resolve failures before building the reply.

Medium severity Add per-user daily LLM reservation for webhook mentions

backend/​src/​services/​slack/​requestClassificationBot.ts:96

Every mention reaches Bedrock without the daily LLM reservation used by the existing Slack onboarding path (backend/src/services/slack/onboardProjectCommand.ts:228-244). The webhook's 200-per-minute IP limiter is not a per-user or daily cost guard, so sustained mentions can continuously consume model quota. Carry the Slack event's user ID through and reserve an appropriate daily allowance before classification.

Copilot AI balanced review requested due to automatic review settings October 2, 2026 15:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Slack entity normalization, diagnostic exposure, accessibility fallback, and deduplication documentation need correction.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity

Open (3)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file
Previously missed (2)

In code that hasn't changed since last review

Medium severity Decode Slack HTML entities before project parsing

backend/​src/​services/​slack/​requestClassificationBot.ts:40

Slack Events API text encodes literal &, <, and > as &amp;, &lt;, and &gt;. This normalization removes Slack control markup but never decodes those entities, so a project such as R&D reaches the parser/PCC lookup as R&amp;D and can be misclassified. Decode the three Slack entities after stripping mentions and links.

Medium severity Add top-level Slack text fallback for screen readers

backend/​src/​services/​slack/​requestClassificationBot.ts:58

This Block Kit reply has no top-level text fallback before it is passed to chat.postMessage. Slack screen readers default to the top-level text and do not read interior blocks, so users of assistive technology can miss the classification. Include a concise fallback containing the title, outcome, and step reached.

Comment thread backend/src/services/slack/requestClassificationBot.ts
Copilot AI balanced review requested due to automatic review settings October 2, 2026 15:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Detached processing can lose acknowledged events, and Slack mentions bypass the existing per-user LLM cost controls.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity

Open (3)
Resolved since last review (1)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file
Previously missed (1)

In code that hasn't changed since last review

Medium severity Enforce per-user daily LLM budget for Slack mention classification

backend/​src/​services/​slack/​requestClassificationBot.ts:114

Every mention reaches the Bedrock-backed classifier without the per-actor daily LLM cap used by the existing Slack onboarding flow (backend/src/services/slack/onboardProjectCommand.ts:228-244). The HTTP limiter allows 200 requests/minute per source IP and does not isolate Slack users, so one member can generate an unbounded daily LLM bill. Pass event.user through and reserve a per-user classification/LLM budget before invoking the classifier.

return
}

dispatchAppMention(payload, req)
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Snowflake rejects a NOT IN subquery in the select list. Fixes the query added in #4877.

Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
…(CM-1841)

Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 15:52
@ulemons
ulemons force-pushed the feat/CM-1841-slack-thread-bot branch from 341f008 to 8ff6cb6 Compare October 2, 2026 15:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The classifier lacks a daily usage cap for costly Bedrock and Snowflake calls, and its deduplication TTL contradicts the documented behavior.

Review effort: Balanced
Findings: 1 High severity · 3 Medium severity

Open (4)
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

Comment on lines +112 to +114
const classification = await withRequestClassifierDeps(qx, (deps) =>
classifyOnboardingRequest(requestText, deps),
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants