Live bucket query tests and docs (#16 PR B)

[craigmcchesney]

PR B of #16 (plan/tickets/16/plan.md, "PR B"). It follows #72, which added the client and bucket_conversions.

Live tests

• New tests/integration/test_query_buckets_integration.py. It ingests its own data, one request per bucket, and covers:
  • a SamplingClock bucket returned verbatim, including axis, provider, and column type;
  • a sub-window query returning the whole boundary bucket, then trim_bucket() and query_buckets_to_dataframes(trim=True) cutting it exactly;
  • limit counting buckets, with two PVs reassembling across pages;
  • max_buckets;
  • the stream returning the same buckets as the unary pages;
  • an empty result reported as success;
  • ColumnMetadata provenance read back live, then absent under exclude_column_metadata (reported as None in attrs["buckets"]).
• test_non_scalar_columns_reach_success is now test_non_scalar_columns_read_back_exactly. It reads back values, dims, the image descriptor, schemaId, and the serialized payload and encoding.

Docs

• query.md: a new "Whole buckets" section covering:
  • stored form and the structural accessors;
  • whole buckets, opt-in trimming, and the configuration-gap limit;
  • per-PV DataFrames;
  • limit counting buckets, and how the stream behaves;
  • the status-filter refusal and serialized pass-through.
• ingestion.md:
  • The arrays/images/structs section now reads the data back.
  • Fix: its imports block lacked QueryParams, PV, and bc. The snippet checker can't catch this because its preamble defines those names; running the recipe as a script did.
• conventions.md, sample-status.md, README.md (feature list, TODO, and the recipe table), and NEXT.md: dropped the "not yet read back / not yet wrapped" wording.
• CLAUDE.md:
  • a new "Bucket Query API" section recording T4–T6 and D2/D5–D8 as invariants;
  • the _build_query_spec() seam note and the "until interface to v2 bucket-oriented query API (queryBuckets / queryBucketsStream) #16" line updated;
  • Key Files entries added.

Verification

• Checks: ruff, ruff format, mypy src/, the cookbook checker (122 snippets), and the release-notes checker all pass. 968 unit tests pass.
• Live integration suite: all 72 tests pass. The stack runs from a local dp-service development checkout, not a release tag.
• Recipes run as one script: the ingestion recipe's frames, the ingestion read-back, and every new query.md bucket snippet. The output matched the docs: the array/image/struct values shown, and the quarter-second trim cutting 10,000 samples to 2,500.

Closes #16

🤖 Generated with Claude Code

https://claude.ai/code/session_01CajTMkjkkzeoWXLSgMpn5k

[Copilot]

Copilot review overview

🟡 Changes recommended

The query cookbook uses dfc without importing it, causing its standalone recipe to fail.

Review effort: Balanced
Findings: 1

Open (1)

• Import dfc before using it in query.md · New

What changed in this PR

Adds live verification and documentation for bucket-oriented queries.

Changes:

• Adds integration coverage for bucket querying and non-scalar round trips.
• Documents bucket paging, trimming, metadata, and conversion behavior.
• Updates release notes, feature listings, and snippet-checker context.

File	Description
tests/​integration/​test_query_buckets_integration.py	Adds live bucket-query tests.
tests/​integration/​test_ingestion_client_integration.py	Verifies non-scalar read-back.
README.md	Advertises bucket-query support.
doc/​release-notes/​NEXT.md	Records user-visible changes.
doc/​cookbook/​sample-status.md	Documents bucket filter refusal.
doc/​cookbook/​README.md	Updates recipe index.
doc/​cookbook/​query.md	Adds bucket-query guidance.
doc/​cookbook/​ingestion.md	Adds non-scalar read-back example.
doc/​cookbook/​conventions.md	Clarifies whole-bucket semantics.
CLAUDE.md	Records implementation invariants.
.dev/​tools/​check-cookbook-snippets.py	Adds bucket conversion imports.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.