Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ Implemented:
- `ti configure`
- `ti update --check`
- `ti update`
- `ti exec`
- `ti db create-db-cluster`
- `ti db list-db-clusters`
- `ti db describe-db-cluster`
Expand Down Expand Up @@ -240,6 +241,7 @@ make live-e2e-fs
make live-e2e-fs-git
make live-e2e-fs-journal
make live-e2e-fs-vault
make live-e2e-runtime
make live-e2e
make release-snapshot
make clean
Expand All @@ -263,7 +265,7 @@ delivery path before dropping only that temporary database. It must not run as
part of `make test`, `make e2e`, or any live-e2e target.
The `make live-e2e-<family>` targets build `bin/ti` and run only the selected
top-level command family against the `live-e2e` profile by default. Keep
configure, db, fs, fs-git, fs-journal, and fs-vault tests
configure, db, fs, fs-git, fs-journal, fs-vault, and exec tests
independently selectable. Do not make a focused family target run tests from a
different family, and do not add separate mutating/non-mutating variants.
`make live-e2e` runs every live family together in one test process and remains
Expand Down Expand Up @@ -368,6 +370,7 @@ internal/fs/tokenmgmt/ Filesystem token lifecycle and local rotation safety
internal/oplog/ local JSONL operation log writer
internal/output/ structured JSON/text/raw rendering
internal/query/ JMESPath query application
internal/runtimeexec/ one-shot Drive9 Runtime command adapter
internal/secretinput/ no-echo secret input helper
internal/settings/ global settings parsing and legacy logging migration
internal/telemetry/ CLI eligibility, identity, event, and delivery path
Expand All @@ -392,12 +395,14 @@ without underscores.
Follow these rules unless `docs/priciples.md` is updated:

- The command tree is at most two levels: `ti <command> [subcommand]`.
- `ti configure` and `ti update` are the only intentional top-level verb
exceptions. `ti configure` is the only interactive command.
- `ti configure`, `ti update`, and `ti exec` are the only intentional top-level
verb exceptions. `ti exec` stays top-level to match the public Drive9
Runtime interaction. `ti configure` is the only interactive command.
- Other top-level commands are nouns such as `db` and `fs`.
- Use long flags only, for example `--profile` and `--db-cluster-name`.
- Do not add short flags or one-letter aliases. The current CLI rejects short
flags before invoking Cobra.
- Use long flags except for the exact `ti exec` compatibility aliases `-w`,
`-C`, `-e`, `-t`, and `-v`.
- Do not add any other short flags or one-letter aliases. The CLI rejects short
flags outside `ti exec` before invoking Cobra.
- `ti fs` Unix-style aliases are command-name aliases only. They must keep the
same long flags, output modes, auth, permissions, dry-run behavior, and command
handlers as their canonical commands.
Expand Down Expand Up @@ -515,6 +520,8 @@ Follow these rules unless `docs/priciples.md` is updated:

Implemented command behavior:

- `ti exec [flags] [--] command [arg...]`: streamed remote stdout/stderr, exact exit status, explicit Filesystem selection, and no retries; see `docs/spec/0036-runtime-exec.md` for the pending release gate

- `ti` without a command returns `cli.missing_command` with exit code `2` and
an AWS-style compact two-level usage synopsis on stderr
- `ti configure`
Expand Down Expand Up @@ -669,6 +676,8 @@ Implemented command behavior:

Registered command surface:

- `ti exec [flags] [--] command [arg...]`

- `ti update --check`
- `ti update`
- `ti db create-db-cluster`
Expand Down Expand Up @@ -1061,11 +1070,12 @@ Generated `ti fs` resource API keys live only in the ID-keyed credentials
files under `~/.ti/fs_credentials/`. User-facing docs and commands must call
these `ti fs` API keys or resource credentials, never reference implementation
API keys. Filesystem data-plane
commands route through the installer-managed Drive9 companion binary named
commands and `ti exec` route through the installer-managed Drive9 companion binary named
`ti-drive9`. ti owns profile loading, region resolution, credential storage,
preflight errors, output/query handling, and command naming; Drive9 owns the
filesystem runtime semantics for data-plane file operations, FUSE/WebDAV mount,
FUSE mount drain, layer behavior, pack/unpack, Git workflows, journal, and vault.
FUSE mount drain, layer behavior, pack/unpack, Git workflows, journal, vault,
and Runtime execution/streaming/cancellation.
Do not reintroduce a runtime fallback to ti-native fs behavior. Public fs
service methods must route through the Drive9 companion path unconditionally;
do not add switches such as `UseDrive9Companion` or hidden environment flags
Expand Down
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ ifneq ($(strip $(TELEMETRY_ENDPOINT)),)
LDFLAGS += -X $(MODULE)/internal/version.telemetryEndpoint=$(TELEMETRY_ENDPOINT)
endif

.PHONY: all build build-telemetry-backend build-telemetry-migrator test e2e telemetry-e2e live-e2e live-e2e-configure live-e2e-db live-e2e-fs live-e2e-fs-git live-e2e-fs-journal live-e2e-fs-vault release-snapshot clean
.PHONY: all build build-telemetry-backend build-telemetry-migrator test e2e telemetry-e2e live-e2e live-e2e-configure live-e2e-db live-e2e-fs live-e2e-fs-git live-e2e-fs-journal live-e2e-fs-vault live-e2e-runtime release-snapshot clean

all: build

Expand Down Expand Up @@ -55,7 +55,7 @@ telemetry-e2e: build build-telemetry-backend build-telemetry-migrator
TI_TEST_TELEMETRY_TIDB_DSN="$$TI_TEST_TELEMETRY_TIDB_DSN" TI_E2E_BIN="$(abspath $(TI_BIN))" TI_TELEMETRY_BACKEND_E2E_BIN="$(abspath $(TELEMETRY_BACKEND_BIN))" TI_TELEMETRY_MIGRATOR_E2E_BIN="$(abspath $(TELEMETRY_MIGRATOR_BIN))" TI_TELEMETRY_E2E=1 $(GO) test ./e2e -count=1 -v -run '^TestTelemetryDeliveryToTiDB$$'

live-e2e: build
$(LIVE_E2E_RUN) -run '^TestLive'
TI_LIVE_RUNTIME=1 $(LIVE_E2E_RUN) -run '^TestLive'

live-e2e-configure: build
$(LIVE_E2E_RUN) -run '^TestLive(ProfileConfigured|CLICommandSurface)$$'
Expand All @@ -75,6 +75,9 @@ live-e2e-fs-journal: build
live-e2e-fs-vault: build
$(LIVE_E2E_RUN) -run '^TestLiveFSVault'

live-e2e-runtime: build
TI_LIVE_RUNTIME=1 $(LIVE_E2E_RUN) -run '^TestLiveRuntimeExec$$'

release-snapshot:
$(GORELEASER) release --snapshot --clean

Expand Down
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,25 @@ Automatic mounting uses FUSE on Linux and WebDAV on macOS and Windows. macOS use

Mount commands start the companion runtime in the background, wait until the mount is ready, and then return a structured result. Use `ti fs unmount-file-system` or `ti fs-vault unmount-vault` to end a mount. The public CLI does not expose a foreground mount mode.

Run a command in a remote sandbox:

```shell
ti exec --file-system-id <file-system-id> -w repo npm test
# With TI_FS_FILE_SYSTEM_ID or TI_FS_TOKEN already set:
ti exec -w repo 'npm ci && npm test'
ti exec -w repo -e CI=1 -e GITHUB_TOKEN -t 20m -- make -j8 test
```

The selected Filesystem is available at `/workspace`. Standard output and standard error stream live, and the remote command's exact exit status is preserved. Only files under `/workspace` persist; `--fresh` starts a new sandbox without erasing those files. Remote stdin and interactive terminals are not supported.

Select the Filesystem with `--file-system-id` / `TI_FS_FILE_SYSTEM_ID` or an explicit `--fs-token` / `TI_FS_TOKEN`, using the same profile and region resolution as other data-plane commands. `-w` selects a working directory within the Filesystem, not an isolation boundary. Inside a matching ti-created FUSE mount, omit `-w` to drain pending local writes and run in the corresponding directory. Explicit `-w` does not drain local writes. WebDAV mounts are not inferred; use `-w` there.

Execution flags are `-w/--workspace`, `-C/--cwd`, repeatable `-e/--env`, `-t/--timeout`, `--fresh`, `--layer`, `--read-only`, and `-v/--verbose`. Put them before the remote command. `ti exec help` and `ti exec --help` show local help; `ti exec -w repo help` and `ti exec -- help` run a remote command named `help`. Arguments after the remote command, including `--help`, are passed unchanged. `--output`, `--query`, and `--dry-run` are not supported.

The first Ctrl-C or SIGTERM requests cancellation and waits for workspace drain. A second interruption detaches; the remote command may still be running. Timeout uses status `124`, cancellation `130`, and an unknown outcome `255`. A remote command can also return any of these statuses, or ti's own error statuses such as `2`, `3`, `4`, and `5`; the exit code alone cannot identify the source. Executions are never retried. After an unknown outcome, inspect the workspace before deciding whether to run the command again.

Use `ti update` or reinstall ti if the local capability check reports an incompatible companion. Release validation is described in [Releasing](docs/releasing.md).

Mount readiness means the mounted path is readable: `ti fs mount-file-system` and `ti fs-vault mount-vault` keep polling the mount path until directory listing succeeds, bounded by `--ready-timeout` (default `30s`). If the mount path never becomes readable in time, the command fails with `fs.mount_ready_timeout`, the background mount is left running, and the mount locator is preserved so the matching unmount command can stop it.

Filesystem layers can fork copy-on-write child timelines without copying a workspace. Layer and checkpoint mounts require FUSE; checkpoint mounts are always read-only. Drive9 does not support combining recursive copy with a layer, so seed a directory tree through a writable layer mount, drain it, and then create the checkpoint:
Expand Down Expand Up @@ -310,6 +329,7 @@ Structured commands output JSON by default. Use `--output text` for command-spec
```text
ti configure
ti update
ti exec [flags] [--] command [arg...]

ti db create-db-cluster --db-cluster-type starter
ti db list-db-clusters --db-cluster-type starter
Expand Down Expand Up @@ -461,6 +481,7 @@ make live-e2e-fs
make live-e2e-fs-git
make live-e2e-fs-journal
make live-e2e-fs-vault
make live-e2e-runtime
```

Run the complete live suite in one test process:
Expand All @@ -469,4 +490,4 @@ Run the complete live suite in one test process:
make live-e2e
```

Set `LIVE_E2E_PROFILE=<profile>` to use a profile other than `live-e2e`. The DB and FS suites perform real cloud mutations and clean up only resources created by the test run.
Set `LIVE_E2E_PROFILE=<profile>` to use a profile other than `live-e2e`. The DB and FS suites perform real cloud mutations and clean up only resources created by the test run. `make live-e2e-runtime` and the complete `make live-e2e` suite require Runtime to be enabled for the selected Filesystem; they run harmless commands, verify separate stdout and stderr plus exact non-zero exit propagation, and fail instead of treating a disabled Runtime as proof of compatibility.
9 changes: 6 additions & 3 deletions cmd/ti/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@ import (
"encoding/json"
"fmt"
"os"
"os/signal"
"syscall"

"github.com/tidbcloud/ti-cli/internal/apperr"
"github.com/tidbcloud/ti-cli/internal/cli"
"github.com/tidbcloud/ti-cli/internal/config/homemigration"
"github.com/tidbcloud/ti-cli/internal/interrupt"
"github.com/tidbcloud/ti-cli/internal/version"
)

Expand All @@ -18,13 +19,15 @@ func main() {
runHomeMigration()
return
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
ctx, stop := interrupt.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()

info := version.Current()
root := cli.NewRootCommand(info)
if err := cli.Execute(ctx, root, info, os.Args[1:], os.Stdout, os.Stderr, cli.WithHomeMigration()); err != nil {
fmt.Fprintf(os.Stderr, "\nti [ERROR]: %s\n", apperr.MessageFor(err))
if !apperr.IsSilent(err) {
fmt.Fprintf(os.Stderr, "\nti [ERROR]: %s\n", apperr.MessageFor(err))
}
os.Exit(apperr.ExitCodeFor(err))
}
}
Expand Down
17 changes: 12 additions & 5 deletions docs/priciples.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ ti is currently in Preview. Its feature and command contracts can change before
- `ti db` manages TiDB Cloud Starter clusters and branches, prepares SQL users, formats connection strings, executes one SQL statement per invocation, lists export tasks, and downloads completed local-target export files.
- `ti fs` manages TiDB Cloud Filesystem resources, files, layers, packs, and mounts.
- `ti fs-git`, `ti fs-journal`, and `ti fs-vault` expose Filesystem-backed Git workspace, append-only journal, and secret-management workflows.
- `ti exec` runs one command in a remote sandbox with the selected Filesystem mounted at `/workspace`.
- `ti configure` initializes a local profile.
- `ti update` explicitly checks for or installs a release update.

Expand All @@ -22,14 +23,14 @@ The command tree has at most two command levels:
ti <command> [subcommand]
```

`configure` and `update` are intentional top-level verb exceptions. Other top-level commands identify product domains: `db`, `fs`, `fs-git`, `fs-journal`, and `fs-vault`.
`configure`, `update`, and `exec` are intentional top-level verb exceptions. Other top-level commands identify product domains: `db`, `fs`, `fs-git`, `fs-journal`, and `fs-vault`. `ti exec` intentionally matches the concise public `drive9 exec` interaction instead of adding a `runtime` parent.

- Commands and flags use complete, self-explanatory names.
- Flags are long-only. Do not add one-letter flags.
- Flags are long-only except for the exact `ti exec` compatibility aliases `-w`, `-C`, `-e`, `-t`, and `-v`.
- Required flags appear before optional flags in help output.
- `ti fs` Unix-style aliases shorten only the command name. Their flags stay long and match the canonical command.
- Only `ti configure` may prompt.
- Help works through `ti help`, `ti <command> help`, and `ti <command> <subcommand> help`.
- Help works through `ti help`, `ti <command> help`, and `ti <command> <subcommand> help`. For `ti exec`, only the exact bare `help` alias is local; `ti exec -w repo help` and `ti exec -- help` run a remote command named `help`.
- `--version` remains available at every command level.
- A command does not infer access mode, resource selection, or mutation intent from SQL or other user content.

Expand All @@ -39,6 +40,7 @@ ti <command> [subcommand]
- `--output json` and `--output text` are the supported structured output modes.
- `--query` applies a JMESPath expression after execution and before rendering.
- Commands that stream raw bytes reject `--query`.
- `ti exec` streams stdout and stderr separately, rejects `--output` and `--query`, and returns the remote command's exact exit status without adding a structured error line.
- Mutating control-plane commands support `--dry-run`; read-only commands reject it.
- Dry run validates local input, credentials, placement, and request construction without sending a mutation.
- Errors use `ti [ERROR]: <actionable message>` and stable exit categories.
Expand Down Expand Up @@ -166,10 +168,11 @@ SQL execution prefers the HTTPS SQL API and uses the selected SQL username/passw

## Filesystem Ownership Boundary

ti does not implement filesystem runtime semantics itself. Installed `ti fs`, `ti fs-git`, `ti fs-journal`, and `ti fs-vault` commands route through the bundled `ti-drive9` companion.
ti does not implement filesystem or remote-execution runtime semantics itself. Installed `ti fs`, `ti fs-git`, `ti fs-journal`, `ti fs-vault`, and `ti exec` commands route through the bundled `ti-drive9` companion.

- ti owns command naming, profile and region resolution, resource selection, credential storage, preflight validation, output/query behavior, errors, installation, and updates.
- Drive9 owns file data-plane semantics, layers, pack/unpack, FUSE and WebDAV mounts, drain, Git workspace behavior, journal behavior, and Vault behavior.
- Drive9 owns Runtime execution, live streams, local-mount drain and inference, sandbox reuse, cancellation, and unknown-outcome classification.
- There is no native ti filesystem fallback.
- `ref/drive9` is context only and is never imported, built, packaged, or used by tests.
- ti exposes only operations present in the Drive9 public CLI. It does not expose Drive9 internal APIs.
Expand All @@ -182,11 +185,15 @@ Each resource runs the companion with isolated state under:

ti supplies a sanitized companion environment containing the resolved server, canonical region, and resource owner token. Inherited `DRIVE9_*` values must not override ti's selection. Users do not edit `~/.drive9` for ti workflows.

`ti exec [flags] [--] command [arg...]` mirrors Drive9 Runtime flags exactly: `-w/--workspace`, `-C/--cwd`, repeatable `-e/--env`, `-t/--timeout`, `--fresh`, `--layer`, `--read-only`, and `-v/--verbose`. Flag parsing stops at the remote command boundary. Runtime stdout and stderr are never buffered or rendered as structured output, executions are never retried, and Runtime uses exit statuses `124`, `130`, and `255` for timeout, cancellation, and unknown outcomes. Remote command statuses are passed through exactly and can overlap these or ti errors; the status alone does not identify its source. Remote stdin and interactive terminals are not supported.

`ti exec` first runs a credential-free local `ti-drive9 exec --help` capability gate and accepts only exit status zero; human help text is not an API. The gate has a ten-second deadline, no stdin or visible output, and a disposable HOME. Cancellation is distinct from incompatibility. Only after that gate succeeds may ti load the selected profile, resolve a File System credential, or start a credential-bearing companion command. Installers and the updater enforce the same command gate before replacing installed binaries.

Filesystem resource selection is:

1. Explicit `--file-system-id` or `TI_FS_FILE_SYSTEM_ID`.
2. The file system ID embedded in an explicitly supplied FS token; any separate ID must match it.
3. Otherwise fail with `fs.missing_file_system_id` before endpoint resolution, companion startup, or a remote call.
3. Otherwise fail with `fs.missing_file_system_id` before endpoint resolution, credential-bearing companion startup, or a remote call. The credential-free local Runtime capability gate is the only `ti exec` exception.

ti never infers a target from profile state, the number of local credentials, creation order, or deletion side effects. Creating the first resource does not select it for later commands, and deleting a resource does not promote another resource. `TI_FS_FILE_SYSTEM_ID` is an explicit process-scoped selector, or a consistency assertion when a token is also supplied; it is not a persisted default.

Expand Down
Loading
Loading