Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/api-compatibility.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# REST API contract compatibility

Tapper and Tapper Hub select a REST contract independently of their release
numbers. The initial and currently sole supported revision is `2026-09-11`.
numbers. The sole supported revision is `2026-09-28`, which replaced `2026-09-11` when grep, listing, and relationship responses gained server-side projection.
Change the revision for incompatible API behavior. Compatible additions and
fixes retain the revision; release numbers remain diagnostic metadata.

`GET /api/version` is public, requires no database access or credentials, and
returns `Cache-Control: no-store` with the actual server build version:

```json
{"server_version":"0.24.0","api_versions":["2026-09-11"]}
{"server_version":"0.24.0","api_versions":["2026-09-28"]}
```

Every request under `/api/v1`, including event-stream handshakes and telemetry,
must send exactly one `Tapper-API-Version: 2026-09-11` header. Accepted requests
must send exactly one `Tapper-API-Version: 2026-09-28` header. Accepted requests
echo it in the response. Clients send their release in `Tapper-Client-Version`.
The Hub validates the contract before authentication, flight authority, or
application handlers. Discovery, health, OAuth, browser routes, and hosted
Expand All @@ -26,7 +26,7 @@ duplicate, comma-joined, or unsupported values return HTTP 400 with
`operationPerformed: false`, for example:

```json
{"error":"unsupported REST contract; upgrade Tapper and Hub together","code":"API_VERSION_UNSUPPORTED","requested_api_versions":["2099-01-01"],"supported_api_versions":["2026-09-11"],"server_version":"0.24.0","operationPerformed":false}
{"error":"unsupported REST contract; upgrade Tapper and Hub together","code":"API_VERSION_UNSUPPORTED","requested_api_versions":["2099-01-01"],"supported_api_versions":["2026-09-28"],"server_version":"0.24.0","operationPerformed":false}
```

CLI invocations and MCP connections discover each selected Hub once, before
Expand Down Expand Up @@ -57,7 +57,7 @@ working against the new Hub, and new clients reject legacy Hubs. Update custom
scripts explicitly, for example:

```sh
curl -H 'Tapper-API-Version: 2026-09-11' \
curl -H 'Tapper-API-Version: 2026-09-28' \
-H "Authorization: Bearer ${TAPPER_TOKEN}" \
https://hub.example.com/api/v1/whoami
```
Expand Down
2 changes: 2 additions & 0 deletions docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ touching). Two implementations exist:
and maintains derived state itself.
- `*keg.RemoteKeg` (`keg_remote.go`) speaks the hub's operation-level HTTP
API — one request per operation; all orchestration happens server-side.
Complete TAP operations (links, backlinks, tags) forward to the Hub's
`/api/v1/tap/*` routes through `Tap.Operations`.

`pkg/tapper` resolves a keg via `keg.NewKegFromTarget`, which returns the
interface; `pkg/tapper` never touches `Repository` directly. All node
Expand Down
6 changes: 6 additions & 0 deletions docs/architecture/runtime-and-concurrency.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,12 @@ use of `time.Now()` that cannot be driven by a fake clock:
- **Remote operations are single-request**: each `RemoteKeg` method is one
HTTP round trip, and the hub serializes per-node writes server-side. There
is no client-side lock lease or dex write over HTTP.
- **Orientation reuse in `tap mcp`**: each tool call needs the session's
resolved orientation, which costs several Hub requests to build. `tap mcp`
reuses it for up to ten seconds (`ServerOptions.OrientationCacheTTL`);
`orient`, `session_refresh`, flight mutations, and any failed call discard
it. Reuse cannot widen access, because the Hub rechecks the orientation
header and KEG authority on every request. Hosted servers leave it off.
- **Advisory locks are session primitives**: `Keg.Lock`/`Unlock`/
`LockStatus`/`ForceUnlock` (used by `tap lock` / `tap node edit`) are opt-in
advisory locks backed by the Hub's `/nodes/{id}/lock` endpoints. Leases carry a TTL
Expand Down
12 changes: 12 additions & 0 deletions docs/architecture/service-layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@ editor open/save, schema creation, dex reads, create, and
lock acquisition therefore have matching local and hosted semantics without
client fan-out.

Listings resolve the fields their format renders where the data lives:
`grep` sends its paging and field selectors with the scan, and any known
result set resolves fields with one `ListView` call restricted by `node_ids`.
No listing reads metadata one node at a time.

Complete TAP operations go through `Tap.Operations`. `links`, `backlinks`, and
`tags` are each one unit — the lookup, paging, and the fields the format
renders — so `DefaultOperations` forwards a Hub KEG's operation to the Hub's
`/api/v1/tap/*` route (`pkg/tapapi`) in one request and composes KEG
operations only for a KEG without a Hub. A Hub serving its own MCP endpoint
injects its in-process TAP service here.

The Tap layer groups mixed-keg read and validation arguments by resolved keg,
issues one batch per group, and restores caller order. Interactive editing and
watching remain separate phases. Cross-keg import uses one source export, one
Expand Down
15 changes: 9 additions & 6 deletions docs/output-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ that appears in both means the same thing in both.
| `%{id}` | the node id | free |
| `%{title}` | the node title | free |
| `%{.updated}`, `%{.created}`, `%{.accessed}` | index timestamps, RFC3339 | free |
| `%{.hash}`, `%{.lead}`, `%{.accessCount}`, `%{.omega}` | statistics fields | one read per node |
| `%{tags}` | the node's tags, comma separated | one read per node |
| `%{anything-else}` | that metadata key | one read per node |
| `%{.hash}`, `%{.lead}`, `%{.accessCount}`, `%{.omega}` | statistics fields | one batch per listing |
| `%{tags}` | the node's tags, comma separated | one batch per listing |
| `%{anything-else}` | that metadata key | one batch per listing |

`id`, `title`, and `tags` are reserved. A node carrying metadata under one of
those keys cannot address it in field position; the intrinsic wins.
Expand Down Expand Up @@ -127,9 +127,12 @@ Formats naming only `id`, `title`, the three dates, or the legacy verbs — whic
includes the default — read nothing beyond the index that a listing already
loads.

Any other selector reads one file per node, for the nodes in the result window
only. Combine with `--limit` on a large keg. The same is true of the equivalent
query predicates: filtering on `entity=plan` or `.hash=` also reads per node.
Any other selector is resolved by the Hub for the nodes in the result window
only, in one batch with the listing itself: `list` and `grep` stay a single
request, and `links`, `backlinks`, and a `tags` query add one listing request
however many rows they return. Paging (`--limit`, `--offset`) happens before
fields are resolved, so combine it with a large keg. Query predicates on the
same fields (`entity=plan`, `.hash=`) are evaluated by the Hub as well.

## Notes

Expand Down
2 changes: 1 addition & 1 deletion pkg/apicontract/contract.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import (
"time"
)

const Revision = "2026-09-11"
const Revision = "2026-09-28"
const VersionHeader = "Tapper-API-Version"
const ClientHeader = "Tapper-Client-Version"
const Required = "API_VERSION_REQUIRED"
Expand Down
4 changes: 2 additions & 2 deletions pkg/apicontract/contract_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ func TestClientDiscoveryMatrix(t *testing.T) {
name, body, code string
status int
}{
{"matching", `{"server_version":"0.24.0","api_versions":["2026-09-11"]}`, "", 200},
{"newer compatible", `{"server_version":"99.0.0","api_versions":["2026-09-11","2099-01-01"]}`, "", 200},
{"matching", `{"server_version":"0.24.0","api_versions":["2026-09-28"]}`, "", 200},
{"newer compatible", `{"server_version":"99.0.0","api_versions":["2026-09-28","2099-01-01"]}`, "", 200},
{"newer incompatible", `{"server_version":"99.0.0","api_versions":["2099-01-01"]}`, Unsupported, 200},
{"legacy", "", DiscoveryUnavailable, 404}, {"redirect", "", DiscoveryUnavailable, 302},
{"malformed", "oops", DiscoveryInvalid, 200}, {"missing versions", `{"server_version":"v1"}`, DiscoveryInvalid, 200},
Expand Down
8 changes: 8 additions & 0 deletions pkg/cli/cmd_mcp.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"io"
"log/slog"
"strings"
"time"

"github.com/jlrickert/cli-toolkit/mylog"
"github.com/jlrickert/cli-toolkit/toolkit"
Expand Down Expand Up @@ -89,9 +90,16 @@ func mcpServerOptions(logger *slog.Logger, reporter tapper.InvocationReporter) m
// stdio puts the server on the same machine as its agent host, so
// attachment paths in tool arguments name the same files for both sides.
SharedFilesystem: true,
// Each orientation resolution costs several Hub requests; an agent's
// burst of tool calls reuses one for a few seconds (#123).
OrientationCacheTTL: mcpOrientationCacheTTL,
}
}

// mcpOrientationCacheTTL bounds how long `tap mcp` reuses a resolved
// orientation between tool calls.
const mcpOrientationCacheTTL = 10 * time.Second

// buildMCPLogger constructs the structured logger for the MCP server.
// Unlike CLI commands, MCP always logs to stderr because stdout is reserved
// for JSON-RPC. When a log file is also configured, entries fan out to both.
Expand Down
2 changes: 1 addition & 1 deletion pkg/keg/format_fields.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ var LegacyFormatVerbs = map[byte]string{
// FormatVocabularyDescription is the one-line summary of the listing field
// vocabulary. It is duplicated as a literal in the MCP tool schemas, which
// require a constant struct tag; a test holds the two in agreement.
const FormatVocabularyDescription = "output format template. Legacy verbs %i (id), %t (title), %d (updated), %c (created), %a (accessed); %% is a literal percent. Named selectors use %{...}: a bare word names a metadata key such as %{type} or %{status}, a leading dot names a statistics field such as %{.accessCount} or %{.omega}, and %{tags} is the node's tag list. Backslash escapes are interpreted: \\t tab, \\n newline, \\r return, \\\\ backslash. Selectors other than id, title, and the three dates read one file per node."
const FormatVocabularyDescription = "output format template. Legacy verbs %i (id), %t (title), %d (updated), %c (created), %a (accessed); %% is a literal percent. Named selectors use %{...}: a bare word names a metadata key such as %{type} or %{status}, a leading dot names a statistics field such as %{.accessCount} or %{.omega}, and %{tags} is the node's tag list. Backslash escapes are interpreted: \\t tab, \\n newline, \\r return, \\\\ backslash. Selectors other than id, title, and the three dates are resolved by the Hub for the returned page in one batch."

func isIndexTimeField(name string) bool {
return slices.Contains(IndexTimeFieldNames, name)
Expand Down
64 changes: 59 additions & 5 deletions pkg/keg/keg_aggregate.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ type ListViewOptions struct {
// it here rather than in the caller keeps paging and TotalMatches correct.
TitleContains string `json:"title_contains,omitempty"`

// NodeIDs, when set, restricts the listing to these nodes. It is how a
// caller that already knows its result set (grep, links, tags) resolves
// the fields for all of it in one call instead of one read per node.
NodeIDs []NodeId `json:"node_ids,omitempty"`

// Fields are field selectors to resolve per row, in the vocabulary of
// ParseFieldSelector ("type", ".omega", "tags"). Intrinsics and index
// timestamps cost nothing; other selectors are read per returned row.
Expand Down Expand Up @@ -82,6 +87,12 @@ type ReadNodesOptions struct {
NodeIDs []NodeId `json:"node_ids,omitempty"`
Query string `json:"query,omitempty"`
Touch bool `json:"touch,omitempty"`
// MetaOnly reads each node's metadata and stats without its content or
// asset lists, in the same consistent read, so a listing that renders
// metadata fields never loads bodies. Content is still read for a node
// whose stats predate the state hash, to derive it. It cannot be combined
// with Touch: a metadata read is not someone opening the node.
MetaOnly bool `json:"meta_only,omitempty"`
}

type RelatedDirection string
Expand All @@ -96,6 +107,22 @@ type RelatedNodesOptions struct {
Direction RelatedDirection `json:"direction"`
}

// RelatedPair is one relationship found by RelatedNodes: From is the input
// node's path and To is the related entry's ID exactly as it appears in
// Entries, which may be a qualified reference into another keg.
type RelatedPair struct {
From string `json:"from"`
To string `json:"to"`
}

// RelatedNodesResult is the merged set of related entries plus the pairs that
// produced them, so callers can attribute each relationship to its input
// without asking once per input.
type RelatedNodesResult struct {
Entries []NodeIndexEntry `json:"entries"`
Pairs []RelatedPair `json:"pairs"`
}

type KegInfo struct {
Settings *Settings `json:"settings"`
Summary *KegSummary `json:"summary"`
Expand Down Expand Up @@ -271,6 +298,19 @@ func (k *LocalKeg) listView(ctx context.Context, opts ListViewOptions) (*ListVie
}
entries = kept
}
if len(opts.NodeIDs) > 0 {
wanted := make(map[string]struct{}, len(opts.NodeIDs))
for _, id := range opts.NodeIDs {
wanted[id.Path()] = struct{}{}
}
kept := make([]NodeIndexEntry, 0, len(opts.NodeIDs))
for _, entry := range entries {
if _, ok := wanted[nodeKey(entry)]; ok {
kept = append(kept, entry)
}
}
entries = kept
}
total := len(entries)

// Sorting by metadata or a non-timestamp stat needs a value for every
Expand Down Expand Up @@ -486,6 +526,9 @@ func (k *LocalKeg) ReadNodes(ctx context.Context, opts ReadNodesOptions) ([]Node
}

func (k *LocalKeg) readNodes(ctx context.Context, opts ReadNodesOptions) ([]NodeView, error) {
if opts.MetaOnly && opts.Touch {
return nil, fmt.Errorf("meta_only and touch are mutually exclusive: %w", ErrInvalid)
}
ids := slices.Clone(opts.NodeIDs)
if q := strings.TrimSpace(opts.Query); q != "" {
if len(ids) != 0 {
Expand Down Expand Up @@ -526,9 +569,13 @@ func (k *LocalKeg) readNodes(ctx context.Context, opts ReadNodesOptions) ([]Node
}
}

read := k.ReadNode
if opts.MetaOnly {
read = k.readNodeMeta
}
views := make([]NodeView, 0, len(ids))
for _, id := range ids {
view, err := k.ReadNode(ctx, id)
view, err := read(ctx, id)
if err != nil {
if errors.Is(err, ErrNotExist) {
err = fmt.Errorf("node %s not found: %w", id.Path(), err)
Expand All @@ -543,11 +590,11 @@ func (k *LocalKeg) readNodes(ctx context.Context, opts ReadNodesOptions) ([]Node
return views, nil
}

func (k *LocalKeg) RelatedNodes(ctx context.Context, opts RelatedNodesOptions) ([]NodeIndexEntry, error) {
return withKegReadValue(ctx, k, func(ctx context.Context) ([]NodeIndexEntry, error) { return k.relatedNodes(ctx, opts) })
func (k *LocalKeg) RelatedNodes(ctx context.Context, opts RelatedNodesOptions) (*RelatedNodesResult, error) {
return withKegReadValue(ctx, k, func(ctx context.Context) (*RelatedNodesResult, error) { return k.relatedNodes(ctx, opts) })
}

func (k *LocalKeg) relatedNodes(ctx context.Context, opts RelatedNodesOptions) ([]NodeIndexEntry, error) {
func (k *LocalKeg) relatedNodes(ctx context.Context, opts RelatedNodesOptions) (*RelatedNodesResult, error) {
if len(opts.NodeIDs) == 0 {
return nil, fmt.Errorf("at least one node id is required: %w", ErrInvalid)
}
Expand All @@ -556,6 +603,8 @@ func (k *LocalKeg) relatedNodes(ctx context.Context, opts RelatedNodesOptions) (
return nil, err
}
seen := map[string]struct{}{}
pairs := []RelatedPair{}
pairSeen := map[RelatedPair]struct{}{}
for _, id := range opts.NodeIDs {
exists, err := k.NodeExists(ctx, id)
if err != nil {
Expand All @@ -575,6 +624,11 @@ func (k *LocalKeg) relatedNodes(ctx context.Context, opts RelatedNodesOptions) (
}
for _, rel := range related {
seen[rel.Path()] = struct{}{}
pair := RelatedPair{From: id.Path(), To: rel.Path()}
if _, dup := pairSeen[pair]; !dup {
pairSeen[pair] = struct{}{}
pairs = append(pairs, pair)
}
}
}
entriesByID := map[string]NodeIndexEntry{}
Expand All @@ -594,7 +648,7 @@ func (k *LocalKeg) relatedNodes(ctx context.Context, opts RelatedNodesOptions) (
out = append(out, NodeIndexEntry{ID: id})
}
}
return out, nil
return &RelatedNodesResult{Entries: out, Pairs: pairs}, nil
}

func (k *LocalKeg) Info(ctx context.Context) (*KegInfo, error) {
Expand Down
43 changes: 42 additions & 1 deletion pkg/keg/keg_aggregate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ func TestLocalKegAggregateOperations(t *testing.T) {
require.Equal(t, []string{"2", "1"}, []string{views[0].ID.Path(), views[1].ID.Path()})
related, err := k.RelatedNodes(ctx, keg.RelatedNodesOptions{NodeIDs: []keg.NodeId{two.ID}, Direction: keg.RelatedLinks})
require.NoError(t, err)
require.Equal(t, "1", related[0].ID)
require.Equal(t, "1", related.Entries[0].ID)
require.Equal(t, []keg.RelatedPair{{From: "2", To: "1"}}, related.Pairs)
info, err := k.Info(ctx)
require.NoError(t, err)
require.Equal(t, 3, info.Summary.NodeCount)
Expand Down Expand Up @@ -191,3 +192,43 @@ func TestRemoteAggregateMethodsUseOneRequest(t *testing.T) {
})
}
}

func TestReadNodesMetaOnly(t *testing.T) {
fx := NewSandbox(t)
ctx := fx.Context()
repo := newTestMemoryRepo(fx.Runtime())
k := keg.NewLocalKeg(repo, fx.Runtime())
initNonStrictTestKeg(t, k, ctx)
created, err := k.Create(ctx, &keg.CreateOptions{Body: []byte("# One\n\nbody\n"), Meta: []byte("kind: note\n")})
require.NoError(t, err)
id := created.ID

full, err := k.ReadNode(ctx, id)
require.NoError(t, err)
views, err := k.ReadNodes(ctx, keg.ReadNodesOptions{NodeIDs: []keg.NodeId{id}, MetaOnly: true})
require.NoError(t, err)
require.Len(t, views, 1)
require.Equal(t, full.Meta, views[0].Meta)
require.Equal(t, full.Hash(), views[0].Hash())
require.Empty(t, views[0].Content)

_, err = k.ReadNodes(ctx, keg.ReadNodesOptions{NodeIDs: []keg.NodeId{{ID: 999}}, MetaOnly: true})
require.ErrorIs(t, err, keg.ErrNotExist)

// A node that exists without metadata, and whose stats predate the state
// hash, is still a node: empty metadata and a hash derived from content.
bare, err := k.Create(ctx, &keg.CreateOptions{Body: []byte("# Bare\n\nbody\n")})
require.NoError(t, err)
require.NoError(t, repo.WriteMeta(ctx, bare.ID, nil))
require.NoError(t, repo.WriteStats(ctx, bare.ID, &keg.NodeStats{}))
bareFull, err := k.ReadNode(ctx, bare.ID)
require.NoError(t, err)
views, err = k.ReadNodes(ctx, keg.ReadNodesOptions{NodeIDs: []keg.NodeId{bare.ID}, MetaOnly: true})
require.NoError(t, err)
require.Len(t, views, 1)
require.Empty(t, views[0].Meta)
require.NotEmpty(t, views[0].Hash())
require.Equal(t, bareFull.Hash(), views[0].Hash())
_, err = k.ReadNodes(ctx, keg.ReadNodesOptions{NodeIDs: []keg.NodeId{id}, MetaOnly: true, Touch: true})
require.ErrorIs(t, err, keg.ErrInvalid)
}
Loading
Loading