Repository navigation
feat(feedback): thumbs up/down on answers (MCP + chat) with an answer_log table - #1162
Open
northdpole wants to merge 36 commits into
Open
northdpole wants to merge 36 commits into
northdpole wants to merge 36 commits into
Conversation
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review: 1
Please check the settings in the CodeRabbit UI or the
 |
northdpole
force-pushed
the
feat/answer-feedback
branch
2 times, most recently
from
October 20, 2026 08:51
b40939d to
957841c
Compare
northdpole
force-pushed
the
feat/ingest-comprehension
branch
from
October 20, 2026 09:21
c306180 to
6b2c573
Compare
northdpole
force-pushed
the
feat/answer-feedback
branch
14 times, most recently
from
October 21, 2026 20:05
76307f2 to
dc9eb78
Compare
northdpole
force-pushed
the
feat/ingest-comprehension
branch
from
October 21, 2026 21:09
b6ded96 to
0a34ae0
Compare
northdpole
force-pushed
the
feat/answer-feedback
branch
from
October 21, 2026 21:09
dc9eb78 to
c672c15
Compare
northdpole
force-pushed
the
feat/ingest-comprehension
branch
from
October 21, 2026 23:09
0a34ae0 to
a6573ed
Compare
northdpole
force-pushed
the
feat/answer-feedback
branch
2 times, most recently
from
October 22, 2026 01:29
2556722 to
6acc47d
Compare
northdpole
force-pushed
the
feat/ingest-comprehension
branch
from
October 22, 2026 07:59
c4b23d4 to
594e1d0
Compare
northdpole
force-pushed
the
feat/answer-feedback
branch
2 times, most recently
from
October 22, 2026 11:09
640e7c4 to
f082dd3
Compare
northdpole
marked this pull request as ready for review
October 22, 2026 11:17
northdpole
requested review from
Pa04rth,
paoga87 and
robvanderveer
as code owners
October 22, 2026 11:17
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…relying on a committed or hand-copied one Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…when the question names none (or says unsure) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rts; PERSTD rule also in the retrieval path (off by default) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…uestions; COUNT enumerate-then-count also applies to whole-project reading (off by default) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e/length per OWASP YouTube talk)" This reverts commit 1a708f0.
…essage instead of a false 'not found' Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e the question so the provider can reuse the cached prefix (off by default) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…P_CX_WIDEMATCH lets whole-project reading match any indexed repo (off by default) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…; personal API keys; MCP tools gated by role Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… kept for chat; ask accepts it too); per-account rate limits Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…82.8); interactive calls retry provider outages (503/500/timeouts) quickly, fall back to a cheaper model and trip a circuit breaker Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…hotpatch image upgrades it Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…y is background, not a second question); CRE handler built only when needed; opt-in web warm-up Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…turn probe Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…messages (background, not a question); rewrite stays for retrieval Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…list; ungrounded free-form answers are replaced by an honest refusal (manual-test items 11 and 15) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…box (off by default) replaces New chat Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…PROJECTS (3 best projects each contribute sources) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…x capped results at 40) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…t chunks to the candidate pool Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…h in the references instead of reordering sources Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
northdpole
force-pushed
the
feat/ingest-comprehension
branch
from
October 25, 2026 10:35
5406406 to
4c92743
Compare
northdpole
force-pushed
the
feat/answer-feedback
branch
from
October 25, 2026 10:35
7c49a5d to
7335acc
Compare
… default) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ast question and answer Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…t references) The OWASP agent answers bypassed the CRE lookup, so control questions no longer showed their closest CRE. route_question now adds it to the references and citations. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… so [n] stays the only link form Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…es in citation URLs Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds thumbs up/down on answers so staging can collect labelled data.
answer_logtable (+ migrationd1e2f3a4b5c6, which joins the two existing heads) and recording of ratings.askreturns ananswer_id; newrate_answertool.answer_id;POST /rest/v1/answer_feedback;GET /admin/feedback. Chat answers are logged only when a deployment opts in (ANSWER_LOG_CHAT=1).alembic_single_head_testfails when the migration chain has more than one head.Stacked on #1158 (order: #1142, #1155, #1159, #1158, this). Needs the migration run and
ANSWER_LOG_CHAT=1set on staging.asknow writes one row per question, so this should be deployed after measurement runs.Tested:
answer_feedback_test,alembic_single_head_test,mcp_ask_testpass; Black clean. Frontend tests run in CI.🤖 Generated with Claude Code