Skip to content

Phase 6 summary health - #6

Merged
ShehanSuraweera merged 2 commits into
mainfrom
phase-6-summary-health
Oct 1, 2026
Merged

ShehanSuraweera merged 2 commits into
mainfrom
phase-6-summary-health

Conversation

@ShehanSuraweera

Copy link
Copy Markdown
Owner

This pull request introduces a new AI-powered thread summarization feature to the service, allowing admins to quickly understand the state of an issue by generating concise summaries of comment threads. It adds a new /v1/summarize-thread API endpoint, supporting models and logic for summarizing threads, and updates documentation and tests accordingly. The implementation ensures that every key point in the summary cites only comments actually present in the thread, with robust safety and validation.

New thread summarization feature:

  • Added the /v1/summarize-thread endpoint, with request and response schemas, to summarize an issue's thread with key points and open questions, each citing the relevant comments or the issue description (ai-service/README.md, ai-service/src/ai_service/api/routes.py, ai-service/src/ai_service/schemas.py). [1] [2] [3]
  • Implemented the thread summarization logic, including prompt construction, comment trimming (to the 40 most recent, each capped at 1,500 characters), output validation, and citation checking (ai-service/src/ai_service/services/summary.py, ai-service/src/ai_service/prompts/summary.py). [1] [2]
  • Added the wrap_thread_comment function and updated safety delimiters to securely wrap and identify thread comments, ensuring that only trusted metadata is used for citations (ai-service/src/ai_service/safety.py). [1] [2]

Documentation and prompt updates:

  • Updated the documentation to describe the new endpoint, its behavior, and security constraints, and documented the new prompt version (summary-v1) (ai-service/README.md). [1] [2]
  • Added the new prompt template for thread summaries, with detailed instructions and rules for the model (ai-service/src/ai_service/prompts/summary.py).

Testing:

  • Added and documented new tests to ensure that key points only cite valid comments, that trimming and ordering are correct, and that forged comment tags are neutralized (ai-service/README.md, ai-service/tests/test_api.py). [1] [2]

These changes together provide a robust, safe, and easily auditable way to generate AI-powered summaries of support threads, improving admin efficiency and oversight.

- POST /v1/summarize-thread returns a summary, up to 6 key points and
  open questions. Every key point cites the comments it comes from, or 0
  for the issue description; a citation outside the thread is rejected.
- Each comment is wrapped in a thread_comment delimiter whose id, author
  role, internal flag and time come from the backend's data, so client
  text can't pose as staff. Forged tags are neutralised.
- Long threads keep their 40 most recent comments (1,500 characters
  each), and the prompt states how many older ones were left out.
- 19 new tests.
- GET/POST /issues/:id/ai/summary (admin): summarises the whole thread,
  including internal notes, for threads of 3 or more comments. Citations
  outside the issue are dropped; the description is stored as
  "description". A summary is marked stale once newer comments exist.
  The cost of every call is logged, including failures.
- GET /ai/escalations (engineer, admin): open issues whose latest
  sentiment reading is high escalation risk, within the viewer's
  tenancy. Priority is shown alongside and never changed.
- GET /ai/client-health (admin): weekly average frustration, negative
  share and high-risk readings per company, and the last 30 days against
  the previous 30. Aggregates stored readings; no LLM calls.
- 15 new tests.
@ShehanSuraweera
ShehanSuraweera merged commit a216ea0 into main Oct 1, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant