Skip to content

Add to Filters on a value inside an array in a native JSON column silently returns no rows #3640

Open

Description

@ralaruri

What happened?

"Add to Filters" on a value inside an array in a native JSON column returns no rows and shows no error. I expected the filter to match the rows that contain that value.

The row panel writes the array positions as quoted JSON keys, so the generated filter is:

toString(message.`events`.`0`.`detail`.`parameters`.`1`.`value`) IN ('EMAIL_SERVICE')

ClickHouse finds no key named 0 or 1 and returns nothing. Array elements are addressed with 1-based subscripts:

toString(message.`events`[1].`detail`.`parameters`[2].`value`) IN ('EMAIL_SERVICE')

On a table of 1,500 rows, the first form matched 0 rows and the second matched all 1,500.

Steps to reproduce

Reproduces on play.hyperdx.io? didn't try (needs a source whose attributes column is a native JSON type)

Table (fake data):

CREATE TABLE demo.events_json (
  ingested_at DateTime64(3) DEFAULT now64(3), organization_id UUID,
  occurred_at DateTime64(3), source_name LowCardinality(String), message JSON
) ENGINE = MergeTree ORDER BY (organization_id, occurred_at);

INSERT INTO demo.events_json (organization_id, occurred_at, source_name, message)
SELECT generateUUIDv4(), now64(3) - toIntervalSecond(number), 'EMAIL_SERVICE',
  '{"email":"user@example.test","events":[{"name":"event_a","detail":{"parameters":[{"name":"param_1","value":"x"},{"name":"param_2","value":"EMAIL_SERVICE"}]}}]}'::JSON
FROM numbers(200);

Logs source: table demo.events_json, Timestamp Column occurred_at, Log Attributes Expression message, Default Select ingested_at, organization_id, message.

  1. Open the source, set the range to the last hour, click Run.
  2. Expand a row and open Column Values. Expand message → events → 0 → detail → parameters → 1.
  3. Hover value and click the filter icon (Add to Filters).
  4. The table shows no rows and no error. In </>, the filter uses `events`.`0` and `parameters`.`1`.

How are you running HyperDX?

The all-in-one image docker.clickhouse.com/clickhouse/clickstack-local, version 2.44.0 (from /health).

Where does it show up?

The UI: the row side panel, Column Values tab, "Add to Filters" icon.

Debug info

Not included. Version above.

Logs

None. There is no error; the query succeeds and returns zero rows.

Notes

mergePath in packages/app/src/utils.ts quotes every segment under a JSON column, so a numeric segment that is an array index comes out as a key. Numeric segments can also be real object keys, and facet keys built from the JSON schema rely on that, so the row viewer (which has the row data) is the place that can tell the two apart.

I have a fix with tests and a changeset and plan to open a PR from ralaruri:fix/json-array-paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions