diff --git a/.github/workflows/simulations.yml b/.github/workflows/simulations.yml index 03f03d6..bebb4f1 100644 --- a/.github/workflows/simulations.yml +++ b/.github/workflows/simulations.yml @@ -1,7 +1,7 @@ # Runs the scenarios in scenarios.yaml against your agent on LiveKit Cloud. # Simulations spend real inference, so this runs on merges to main and on demand # rather than on every pull request push. Run -# `lk agent simulate --scenarios scenarios.yaml` locally while iterating. +# `lk agent simulate text --scenarios scenarios.yaml` locally while iterating. name: Simulations on: @@ -43,4 +43,4 @@ jobs: run: curl -sSL https://get.livekit.io/cli | bash - name: Run simulations - run: lk agent simulate --scenarios scenarios.yaml + run: lk agent simulate text --scenarios scenarios.yaml diff --git a/AGENTS.md b/AGENTS.md index 8ddadd3..e93b752 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -42,7 +42,7 @@ Voice AI agents are highly sensitive to excessive latency. For this reason, it's ## Testing -When possible, add tests for agent behavior. Add a scenario to `scenarios.yaml` and run it with `lk agent simulate --scenarios scenarios.yaml`. The scenarios run in CI on every merge to main; read the [simulations documentation](https://docs.livekit.io/agents/start/testing/simulations/) before editing them. +When possible, add tests for agent behavior. Add a scenario to `scenarios.yaml` and run it with `lk agent simulate text --scenarios scenarios.yaml`. The scenarios run in CI on every merge to main; read the [simulations documentation](https://docs.livekit.io/agents/start/testing/simulations/) before editing them. For turn-level checks that don't need a live session, use the in-process [testing framework](https://docs.livekit.io/agents/start/testing/); `tests/test_agent.py` has a commented-out example. Run those with `uv run pytest`. diff --git a/README.md b/README.md index d55659a..e32a547 100644 --- a/README.md +++ b/README.md @@ -147,7 +147,7 @@ For advanced customization, see the [complete frontend guide](https://docs.livek Simulations run full multi-turn conversations between a simulated user and your agent on LiveKit Cloud, then judge each transcript. The scenarios live in [`scenarios.yaml`](scenarios.yaml). Run them locally with the [LiveKit CLI](https://docs.livekit.io/intro/basics/cli/): ```console -lk agent simulate --scenarios scenarios.yaml +lk agent simulate text --scenarios scenarios.yaml ``` The `Simulations` workflow in `.github/workflows/simulations.yml` runs the same file on every merge to `main` and on demand from the Actions tab. It runs there rather than on every pull request push because each run spends real inference. See the [simulations guide](https://docs.livekit.io/agents/start/testing/simulations/) for how to write scenarios and read results. diff --git a/scenarios.yaml b/scenarios.yaml index 812f50f..805c681 100644 --- a/scenarios.yaml +++ b/scenarios.yaml @@ -1,6 +1,6 @@ # Simulation scenarios for the starter assistant. Each one is a conversation a # simulated user has with your agent, judged against agent_expectations. Run them -# with `lk agent simulate --scenarios scenarios.yaml`; the Simulations workflow +# with `lk agent simulate text --scenarios scenarios.yaml`; the Simulations workflow # runs them on every merge to main. # # Add a scenario whenever you change instructions or add a tool, and pin any dates