MCP response trimming replaces the largest values whole (often the top result) and ignores topN; clickstack_table has no limit, having or default order #3542

Activity

1. github-actions

   added

   external

   on Nov 4, 2026
2. brandon-pereira commented on Nov 4, 2026

   

   brandon-pereira

   on Nov 4, 2026

   Member

   More actions

   Thanks for the feedback. There are some good ideas here.

   Scope

   Let's please focus on trimToolResponse in this issue. New table tool features like having and limit should move to a separate issue and PR so we can keep the scope smaller.

   trimToolResponse

   I agree. Let's do the following:

   1. If trimming must occur, shorten strings with a suffix. This can be done across all tools.
   2. Send notes in the trimToolResponse output about stats on the trimming that occurred.
   3. Dropping samples before patterns is a good idea. However, please consider how we can do this from inside trimToolResponse itself.

   An idea for 3 (open to alternates)

   type Reduction =
     | { path: string; keep: number }     // shrink array elements to max of keep
     | { path: string; maxChars: number } // shorten matching strings, with "…(+6,900 chars)"
     | { path: string; drop: true };      // remove the key
   
   trimToolResponse(data, maxSize, { reductions?: Reduction[] })

   For event patterns:

   trimToolResponse(output, undefined, {
     reductions: [
       { path: 'patterns[].samples', keep: 3 },
       { path: 'patterns[].samples', keep: 1 },
       { path: 'patterns[].samples[]', maxChars: 2000 },
       { path: 'patterns[].pattern', maxChars: 2000 },
       { path: 'patterns[].trend', drop: true },
     ],
   });

   How it would run:

   1. If the response fits, return it as is (unchanged).
   2. Apply the reductions in order. Each one applies to every match (all patterns go to 3 samples, then all to 1), so the ranks stay comparable. After each step, measure the whole response and stop as soon as it fits.
   3. Only if it's still over budget, fall back to today's per-key split and array cuts.

   Measuring the whole response against the whole budget also fixes the case where a response slightly over budget loses most of its patterns, because the patterns key no longer gets capped at a third of the budget.
3. kodiakhq

   added a commit that references this issue on Nov 7, 2026

   fix(mcp): shorten instead of replacing when trimming tool responses, …

   ...

   0262822
4. bcanvural commented on Nov 13, 2026

   

   bcanvural

   on Nov 13, 2026

   ContributorAuthor

   More actions

   Thanks for #3554, the pattern reductions work well for us. One more case for trimToolResponse, from PromQL dashboard tiles (clickstack_query_tile):

   Trimming drops whole series. PromQL tile rows are series-major (all points of series 1, then series 2, …; toPromqlChartRows in common-utils/src/core/promqlChart.ts). When the result is over budget, the array cut keeps the first rows of result.data, so the series at the end disappear completely. seriesLabels is not trimmed and still lists every series, and the note only says result.data: cut from 702 to 343 items. An agent then reads the remaining series as the whole answer.

   reproTileTrimming.int.test.ts (below; copy to packages/api/src/mcp/__tests__/dashboards/, main at c220e16) writes one counter for 6 series (env dev / prod, 3 pods each), one sample per minute for 2 hours:

   query_tile "sum by (env, pod) (rate(...[5m]))":
     rows returned=343 of 702; points per series: dev-pod-0=117, dev-pod-1=117, dev-pod-2=109
     seriesLabels lists 6; missing from data: prod-pod-0, prod-pod-1, prod-pod-2
     trimNotes=["Response trimmed from 77,217 to 37,734 chars (limit 50,000).","result.data: cut from 702 to 343 items"]
   

   In agent investigations on real dashboards this hid every production series of an unfiltered "requests per workload" tile (143 of 600 rows kept), dropped one of two production pods from a "job failures" tile, and kept the full timelines of 3 of ~261 series in a per-customer tile.

   Proposal, in the reductions framework: for tile results, a reduction that thins points per series before any array cut, e.g. { path: 'result.data', perSeries: 'series_name', keepEvery: k } (every k-th point plus the last one), so every series in seriesLabels keeps data. If series still have to go, drop the smallest and name them in the trim note (3 series dropped: …).

   Two related tile issues are outside trimToolResponse; happy to open them separately if you prefer:

   • clickstack_query_tiles counts rowCount after trimming and drops the trim note, so a trimmed tile reports status: "ok" with a plausible count (9 of 30 tiles in one run). A trimmed flag and the pre-trim row and series counts would show it (per-tile timing is in MCP: PromQL, dashboard-tile, pattern and describe_source tools return no cost statistics #3649).
   • A single-series PromQL tile without a legend uses its whole expression as series_name on every row (promqlSeriesNameInputs); with a 544-char expression that is 75% of the response and the reason it gets trimmed at all.

   reproTileTrimming.int.test.ts

   // Repro: clickstack_query_tile / clickstack_query_tiles on PromQL tiles whose result is over the
   // 50,000-char response budget. Copy to packages/api/src/mcp/__tests__/dashboards/ and run:
   //   make dev-int FILE=reproTileTrimming
   import { DisplayType, SourceKind } from '@hyperdx/common-utils/dist/types';
   
   import {
     DEFAULT_DATABASE,
     dropTimeSeriesTable,
     executeTimeSeriesSqlCommand,
     seedTimeSeriesTagsTable,
   } from '@/fixtures';
   import { callTool, getFirstText } from '@/mcp/__tests__/mcpTestUtils';
   import Dashboard from '@/models/dashboard';
   import { Source } from '@/models/source';
   
   import { setupDashboardTests } from './setup';
   
   const mockFetch = jest.mocked(global.fetch);
   const realFetch: typeof fetch = globalThis.realFetch;
   
   const TABLE = 'repro_tile_trimming';
   const START = 1700000000;
   const MINUTES = 120;
   // 6 series: 3 pods in env "dev", 3 in env "prod", one sample per minute for 2 h.
   const SERIES = ['dev', 'prod'].flatMap(env =>
     [0, 1, 2].map(i => ({ env, pod: `${env}-pod-${i}` })),
   );
   // A single-series KPI expression of ~600 chars, as dashboards converted from other tools often have.
   const LONG_EXPR =
     'sum(rate(repro_requests_total{env=~"dev|prod", pod=~".+-pod-[0-9]+", job!="", instance!~"canary-.*", route!~"/(health|ready|live|metrics)z?", method=~"GET|POST|PUT|DELETE"}[5m])) ' +
     '/ ignoring() group_left() ' +
     'clamp_min(sum(rate(repro_requests_total{env=~"dev|prod", pod=~".+-pod-[0-9]+", job!="", instance!~"canary-.*", route!~"/(health|ready|live|metrics)z?"}[5m])), 1) ' +
     '* 100 + 0 * sum(rate(repro_requests_total{env=~"dev|prod", pod=~".+-pod-[0-9]+", job!="", instance!~"canary-.*", route!~"/(health|ready|live|metrics)z?", status=~"[25].."}[5m]))';
   
   describe('repro: tile results over the response budget', () => {
     const ctx = setupDashboardTests();
   
     beforeAll(async () => {
       await seedTimeSeriesTagsTable({
         table: TABLE,
         series: SERIES.map(tags => ({
           metricName: 'repro_requests_total',
           tags: { ...tags, job: 'api', route: '/v1/items', method: 'GET', status: '200' },
           startSec: START,
           endSec: START + MINUTES * 60,
         })),
         withSamples: true,
       });
       // One sample per minute per series (a counter going up by 60/min).
       await executeTimeSeriesSqlCommand(
         `INSERT INTO TABLE FUNCTION timeSeriesData('${DEFAULT_DATABASE}', '${TABLE}')
          SELECT id, toDateTime64(${START} + n * 60, 3), n * 60
          FROM timeSeriesTags('${DEFAULT_DATABASE}', '${TABLE}') ARRAY JOIN range(0, ${MINUTES + 1}) AS n`,
       );
     });
   
     afterAll(async () => {
       await dropTimeSeriesTable({ table: TABLE });
     });
   
     beforeEach(async () => {
       await ctx.reconnect({ promqlEnabled: true });
       mockFetch.mockImplementation(realFetch);
     });
     afterEach(() => mockFetch.mockReset());
   
     const createDashboard = async (tiles: { id: string; name: string; expression: string }[]) => {
       const source = await Source.create({
         kind: SourceKind.Promql,
         team: ctx.team._id,
         connection: ctx.connection._id,
         name: 'Metrics',
         from: { databaseName: DEFAULT_DATABASE, tableName: TABLE },
         timestampValueExpression: 'timestamp',
       });
       return new Dashboard({
         name: 'Repro',
         tiles: tiles.map(t => ({
           id: t.id,
           x: 0,
           y: 0,
           w: 6,
           h: 3,
           config: {
             configType: 'promql',
             name: t.name,
             displayType: DisplayType.Line,
             source: source._id.toString(),
             granularity: '1 minute',
             promqlExpression: [{ expression: t.expression }],
           },
         })),
         filters: [],
         team: ctx.team._id,
       }).save();
     };
   
     const window = {
       startTime: new Date((START + 300) * 1000).toISOString(),
       endTime: new Date((START + MINUTES * 60) * 1000).toISOString(),
     };
   
     it('query_tile drops whole series when it trims, and seriesLabels still lists them', async () => {
       const dashboard = await createDashboard([
         { id: 'per-pod', name: 'Requests per pod', expression: 'sum by (env, pod) (rate(repro_requests_total[5m]))' },
       ]);
       const result = await callTool(ctx.client!, 'clickstack_query_tile', {
         dashboardId: dashboard._id.toString(),
         tileId: 'per-pod',
         ...window,
       });
       expect(result.isError).toBeFalsy();
       const out = JSON.parse(getFirstText(result));
       const rows: { series_name: string }[] = out.result.data;
       const perSeries = new Map<string, number>();
       for (const r of rows) perSeries.set(r.series_name, (perSeries.get(r.series_name) ?? 0) + 1);
       const listed = Object.keys(out.result.seriesLabels);
       const missing = listed.filter(s => !perSeries.has(s));
       console.log(
         `REPRO query_tile: rows returned=${rows.length} of ${out.result.rows}; ` +
           `points per series=${JSON.stringify(Object.fromEntries(perSeries))}; ` +
           `seriesLabels lists ${listed.length}; missing from data: ${JSON.stringify(missing)}; ` +
           `note=${JSON.stringify(out.note)} trimNotes=${JSON.stringify(out.trimNotes)}`,
       );
       expect(listed).toHaveLength(6);
       expect(missing.length).toBeGreaterThan(0); // whole series gone
       expect(missing.every(s => s.includes('prod'))).toBe(true); // all from the later env
       expect(JSON.stringify(out.trimNotes)).not.toContain('prod'); // nothing names them
     });
   
     it('a single-series tile without a legend repeats the whole expression as series_name on every row', async () => {
       const dashboard = await createDashboard([{ id: 'kpi', name: 'Error ratio', expression: LONG_EXPR }]);
       const result = await callTool(ctx.client!, 'clickstack_query_tile', {
         dashboardId: dashboard._id.toString(),
         tileId: 'kpi',
         ...window,
       });
       expect(result.isError).toBeFalsy();
       const text = getFirstText(result);
       const out = JSON.parse(text);
       const rows: { series_name: string }[] = out.result.data;
       const nameChars = rows.reduce((n, r) => n + r.series_name.length, 0);
       console.log(
         `REPRO single-series tile: expression=${LONG_EXPR.length} chars, rows=${rows.length}, ` +
           `series_name === expression on every row: ${rows.every(r => r.series_name === LONG_EXPR)}, ` +
           `series_name chars=${nameChars} of ${text.length} response chars; trimNotes=${JSON.stringify(out.trimNotes)}; ` +
           `last row kept=${(rows.at(-1) as any)?.__hdx_time_bucket} (window ends ${window.endTime})`,
       );
       expect(rows.every(r => r.series_name === LONG_EXPR)).toBe(true);
     });
   
     it('query_tiles reports ok and the post-trim rowCount, with no trimmed flag or timing', async () => {
       const dashboard = await createDashboard([
         { id: 'per-pod', name: 'Requests per pod', expression: 'sum by (env, pod) (rate(repro_requests_total[5m]))' },
         { id: 'per-env', name: 'Requests per env', expression: 'sum by (env) (rate(repro_requests_total[5m]))' },
       ]);
       const result = await callTool(ctx.client!, 'clickstack_query_tiles', {
         dashboardId: dashboard._id.toString(),
         ...window,
       });
       const out = JSON.parse(getFirstText(result));
       console.log(`REPRO query_tiles: ${JSON.stringify(out.tiles)}`);
       const perPod = out.tiles.find((t: { tileId: string }) => t.tileId === 'per-pod');
       expect(perPod).toMatchObject({ status: 'ok', hasData: true });
       expect(perPod.rowCount).toBeLessThan(6 * (MINUTES - 4)); // the trimmed count, not the real one
       expect(Object.keys(perPod).sort()).toEqual(['displayType', 'hasData', 'name', 'rowCount', 'status', 'tileId']);
     });
   });