feat(mcp): enable model and conversation capture by default - #278
Open
lucasheriques wants to merge 1 commit into
Open
lucasheriques wants to merge 1 commit into
lucasheriques wants to merge 1 commit into
Conversation
Match the current JavaScript and Python MCP SDK defaults for both instrumented servers and custom dispatcher tool schemas. Keep explicit false values as opt-outs and isolate transport-session tests from conversation anchoring. Verified with: - bundle exec rspec (1202 examples, 0 failures) - bundle exec rubocop (127 files, 0 offenses) - bundle exec rake public_api:check
posthog-ruby-async Compliance ReportDate: 2026-09-23T12:53:33.643579+00:00
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 109ms |
| Format Validation.Event Has Uuid | ✅ | 106ms |
| Format Validation.Event Has Lib Properties | ✅ | 108ms |
| Format Validation.Distinct Id Is String | ✅ | 106ms |
| Format Validation.Token Is Present | ✅ | 106ms |
| Format Validation.Custom Properties Preserved | ✅ | 106ms |
| Format Validation.Event Has Timestamp | ✅ | 106ms |
| Format Validation.Non Utc Event Timestamp Is Converted To Utc | ✅ | 8ms |
| Retry Behavior.Retries On 503 | ✅ | 5311ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 2108ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 2108ms |
| Retry Behavior.Respects Retry After Header | ✅ | 8115ms |
| Retry Behavior.Implements Backoff | ✅ | 15722ms |
| Retry Behavior.Retries On 500 | ✅ | 5212ms |
| Retry Behavior.Retries On 502 | ✅ | 5212ms |
| Retry Behavior.Retries On 504 | ✅ | 5212ms |
| Retry Behavior.Max Retries Respected | ✅ | 15722ms |
| Deduplication.Generates Unique Uuids | ✅ | 111ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 5211ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 10416ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 5215ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 112ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 107ms |
| Compression.Sends Gzip When Enabled | ✅ | 106ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 106ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 4ms |
| Batch Format.Multiple Events Batched Together | ✅ | 109ms |
| Error Handling.Does Not Retry On 403 | ✅ | 2108ms |
| Error Handling.Does Not Retry On 413 | ✅ | 2108ms |
| Error Handling.Retries On 408 | ✅ | 5212ms |
Feature_Flags Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Request Payload.Request With Person Properties Device Id | ✅ | 107ms |
| Request Payload.Flags Request Uses V2 Query Param | ✅ | 106ms |
| Request Payload.Flags Request Hits Flags Path Not Decide | ✅ | 106ms |
| Request Payload.Flags Request Omits Authorization Header | ✅ | 106ms |
| Request Payload.Token In Flags Body Matches Init | ✅ | 106ms |
| Request Payload.Groups Round Trip | ✅ | 106ms |
| Request Payload.Groups Default To Empty Object | ✅ | 106ms |
| Request Payload.Disable Geoip False Propagates As Geoip Disable False | ✅ | 106ms |
| Request Payload.Disable Geoip Omitted Defaults To False | ❌ | 106ms |
| Request Payload.Flag Keys To Evaluate Contains Only Requested Key | ✅ | 107ms |
| Request Lifecycle.No Flags Request On Init Alone | ✅ | 2ms |
| Request Lifecycle.No Flags Request On Normal Capture | ✅ | 106ms |
| Request Lifecycle.Two Flag Calls Produce Two Remote Requests | ✅ | 111ms |
| Request Lifecycle.Mock Response Value Is Returned To Caller | ✅ | 106ms |
| Retry Behavior.Retries Flags On 502 | ✅ | 239ms |
| Retry Behavior.Retries Flags On 504 | ✅ | 254ms |
| Side Effect Events.Get Feature Flag Captures Feature Flag Called Event | ✅ | 107ms |
Failures
request_payload.disable_geoip_omitted_defaults_to_false
Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']
posthog-ruby-sync Compliance ReportDate: 2026-09-23T12:53:38.839702+00:00
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 9ms |
| Format Validation.Event Has Uuid | ✅ | 6ms |
| Format Validation.Event Has Lib Properties | ✅ | 6ms |
| Format Validation.Distinct Id Is String | ✅ | 9ms |
| Format Validation.Token Is Present | ✅ | 7ms |
| Format Validation.Custom Properties Preserved | ✅ | 7ms |
| Format Validation.Event Has Timestamp | ✅ | 7ms |
| Format Validation.Non Utc Event Timestamp Is Converted To Utc | ✅ | 7ms |
| Retry Behavior.Retries On 503 | ✅ | 5241ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 2008ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 2011ms |
| Retry Behavior.Respects Retry After Header | ✅ | 8013ms |
| Retry Behavior.Implements Backoff | ✅ | 15232ms |
| Retry Behavior.Retries On 500 | ✅ | 5115ms |
| Retry Behavior.Retries On 502 | ✅ | 5150ms |
| Retry Behavior.Retries On 504 | ✅ | 5142ms |
| Retry Behavior.Max Retries Respected | ✅ | 15592ms |
| Deduplication.Generates Unique Uuids | ✅ | 20ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 5115ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 10255ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 5150ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 18ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 9ms |
| Compression.Sends Gzip When Enabled | ✅ | 6ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 7ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 3ms |
| Batch Format.Multiple Events Batched Together | ❌ | 17ms |
| Error Handling.Does Not Retry On 403 | ✅ | 2008ms |
| Error Handling.Does Not Retry On 413 | ✅ | 2010ms |
| Error Handling.Retries On 408 | ✅ | 5152ms |
Failures
batch_format.multiple_events_batched_together
Expected 1 requests, got 5
Feature_Flags Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Request Payload.Request With Person Properties Device Id | ✅ | 8ms |
| Request Payload.Flags Request Uses V2 Query Param | ✅ | 7ms |
| Request Payload.Flags Request Hits Flags Path Not Decide | ✅ | 6ms |
| Request Payload.Flags Request Omits Authorization Header | ✅ | 8ms |
| Request Payload.Token In Flags Body Matches Init | ✅ | 8ms |
| Request Payload.Groups Round Trip | ✅ | 7ms |
| Request Payload.Groups Default To Empty Object | ✅ | 8ms |
| Request Payload.Disable Geoip False Propagates As Geoip Disable False | ✅ | 7ms |
| Request Payload.Disable Geoip Omitted Defaults To False | ❌ | 7ms |
| Request Payload.Flag Keys To Evaluate Contains Only Requested Key | ✅ | 7ms |
| Request Lifecycle.No Flags Request On Init Alone | ✅ | 2ms |
| Request Lifecycle.No Flags Request On Normal Capture | ✅ | 6ms |
| Request Lifecycle.Two Flag Calls Produce Two Remote Requests | ✅ | 9ms |
| Request Lifecycle.Mock Response Value Is Returned To Caller | ✅ | 7ms |
| Retry Behavior.Retries Flags On 502 | ✅ | 108ms |
| Retry Behavior.Retries Flags On 504 | ✅ | 134ms |
| Side Effect Events.Get Feature Flag Captures Feature Flag Called Event | ✅ | 8ms |
Failures
request_payload.disable_geoip_omitted_defaults_to_false
Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']
Contributor
|
The PR should not merge until the documented custom-dispatcher model-capture opt-out also prevents call-time extraction and telemetry capture. Reviews (1) · Last reviewed commit: "feat(mcp): enable model and conversation..." |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # @param tools [Array<Hash>] `tools/list` entries | ||
| # @return [Array<Hash>] | ||
| def prepare_tool_list(tools, context: true, report_missing: false, capture_model: false) | ||
| def prepare_tool_list(tools, context: true, report_missing: false, capture_model: true) |
Contributor
There was a problem hiding this comment.
Passing capture_model: false only removes llm_model from the advertised schema. prepare_tool_call does not receive or retain that setting, so if an agent still sends llm_model, the method strips the value from the tool arguments and returns it for analytics capture. This makes the documented opt-out ineffective and differs from the wrapped-server path, which gates extraction on capture_model_enabled?. Preserve the setting through call preparation so disabling model capture also prevents extracting and recording the value.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/posthog/mcp/client.rb
Line: 124
Comment:
**Model opt-out is bypassed**
Passing `capture_model: false` only removes `llm_model` from the advertised schema. `prepare_tool_call` does not receive or retain that setting, so if an agent still sends `llm_model`, the method strips the value from the tool arguments and returns it for analytics capture. This makes the documented opt-out ineffective and differs from the wrapped-server path, which gates extraction on `capture_model_enabled?`. Preserve the setting through call preparation so disabling model capture also prevents extracting and recording the value.
**Knowledge Base Used:**
- [MCP analytics integration](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/posthog-ruby/-/docs/mcp-analytics-integration.md)
- [MCP server lifecycle instrumentation](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/posthog-ruby/-/docs/mcp-server-lifecycle-instrumentation.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
capture_modelby default for instrumented MCP servers.enable_conversation_idby default.prepare_tool_listmodel default.falsevalues as opt-outs.report_missingoff, exception capture on, and context capture on.Why
The latest JavaScript and Python MCP SDKs enable model capture and conversation correlation by default. Ruby should produce the same useful analytics from the one-line setup.
A valid echoed conversation handle takes precedence over transport sessions. Tests that specifically verify transport-based grouping now disable conversation correlation explicitly.
Verification
bundle exec rspec- 1,202 examples, 0 failuresbundle exec rubocop- 127 files, 0 offensesbundle exec rake public_api:check- passedIncludes a minor changeset.