Skip to content

Docs search: name the ideas readers search for, and pin a set of test queries #574

Closed

Description

@aicayzer

Summary

Searching the docs for an idea by the name an outside reader would use returns nothing, even when a page explains the idea. Name each such idea in the page that explains it, and pin a set of test queries with the page each should find, so the search cannot quietly regress.

Context

At docs commit aeb5162, marfa docs search returns no hits for optimistic concurrency, etag and timestamps, though Versions and Write safely explain the idea. delete vs purge returns /guides/connector-pull-and-push first. Exact terms and error codes work well (version_conflict, type_not_permitted and MARFA_API_KEY each find the right page in the first two). Search is Orama BM25 over page rows, in src/lib/docs-tools.ts, shared by the MCP server, /api/docs/search, the site search box and marfa docs search. These came from a spot check of 40 queries an agent might type.

Acceptance criteria

  • Each of optimistic concurrency, etag and timestamps returns the page that explains it in the first three hits, because that page names the idea in plain words (STYLE.md applies).
  • A test holds a table of queries and the page each must return in the first three hits, seeded with at least the 40 queries from the spot check and the three above, and fails when a page drops out.
  • No new term is added to a page that the contract or the glossary does not support.

Out of scope

  • Changing the search engine, its tokenizer or its ranking (a separate sub-issue).
  • Embeddings.

Notes

Queries from the spot check: If-Match header, 412 precondition, version_conflict, 409, idempotency key, retry a write safely, pagination cursor, next_cursor, list items by tag, filter items by property, webhook signature, hmac, rate limit 429, retry-after, CORS, oauth device flow, marfa login, create type scaffold, environment variables, MARFA_API_KEY, docker compose, litestream, s3 bucket, upgrade server, change instance name, passkey, json output, exit code 5, unauthorized, type_not_permitted, blob upload, attach a file to a note, rotate a key, how do I back up my data, install marfa on linux, restore.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions