Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/ai/L0_repo_card.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
| Language | Python 3.10+ (FastAPI + uvicorn) backend + Next.js 16 / React 19 web |
| Deploy Target | `web/` as Next.js app, `server/` as a reachable FastAPI service |
| Owner | Agora Conversational AI DevEx |
| Last Reviewed | 2026-08-21 |
| Last Reviewed | 2026-09-17 |
| Recipe Role | `base` |
| Recipe Version | `1.0.0` |
| Recipe Status | `experimental` |
Expand Down
4 changes: 2 additions & 2 deletions docs/ai/L1/01_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@ fastapi>=0.100.0
uvicorn>=0.20.0
requests>=2.31.0
python-dotenv>=1.0.0
agora-agents>=2.0.0
agora-agents>=2.9.0
```

The SDK is lower-bounded at v2 — add an upper bound or exact pin if you need reproducible SDK behavior.
The SDK requires version 2.9.0 or later.

## Quick Commands

Expand Down
2 changes: 1 addition & 1 deletion docs/ai/L1/02_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
3. `ConversationComponent` mounts inside a dynamic `AgoraRTCProvider` (RTC client in `useRef` for StrictMode safety) and:
- `useJoin` joins RTC.
- `useLocalMicrophoneTrack` + `usePublish` start mic publishing.
- `AgoraVoiceAI.init({ rtcEngine, rtmConfig: { rtmEngine: rtmClient } })` wires transcripts, state, metrics.
- `AgoraVoiceAI.init({ rtcEngine, rtmEngine: rtmClient })` wires transcripts, state, metrics.
- `subscribeMessage(channel_name)` opens the toolkit's RTM channel.
4. End: `stopAgent(agentId)` → `POST /api/stopAgent` → FastAPI stops the agent. `rtmClient.logout()` follows.
5. Renewal: on RTC `token-privilege-will-expire`, the client fetches `getConfig()` twice (once for RTC uid, once for the stored `agoraData.uid`) and renews RTC + RTM separately.
Expand Down
6 changes: 6 additions & 0 deletions docs/ai/L1/07_gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ RTM delivery, tool enablement, metrics, error messages, and data channel setting

The repo's git history is human-authored. Keep it that way — see `AGENTS.md` "Git Conventions."

## Toolkit 2.10.0 Migration

Pass `rtmEngine` directly to `AgoraVoiceAI.init`; the old nested `rtmConfig` is no longer supported. Install from the repository root with Bun and commit the root `bun.lock`. `@types/bun` supplies types for the existing `bun:test` files.

UIKit 1.1.0 declares an optional Toolkit ^1.2.0 peer. Bun resolves a separate Toolkit 1.2.0 for UIKit while the app uses 2.10.0. Passing type checks and a production build does not establish live session compatibility; verify transcript, agent state, metrics, and teardown in a real call before shipping.

## Related Deep Dives

- [Managed Agent Config](L2/managed_agent_config.md) — Backend defaults.
Expand Down
2 changes: 1 addition & 1 deletion docs/ai/L1/L2/from_scratch_bootstrap.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Why: provider schemas, SDK builder fields, token behavior, and RTM event details
Implement these pieces in order:

1. Create a bun workspace with `web` as a workspace member and root scripts that orchestrate backend, frontend, setup, doctor, verify, and clean tasks.
2. Create `server/` with FastAPI, uvicorn, python-dotenv, and `agora-agents>=2.0.0` in `server/requirements.txt`.
2. Create `server/` with FastAPI, uvicorn, python-dotenv, and `agora-agents>=2.9.0` in `server/requirements.txt`.
3. Add `server/.env.example` with `AGORA_APP_ID`, `AGORA_APP_CERTIFICATE`, and optional `PORT`.
4. Implement `server/src/agent.py` with an `Agent` class that reads env once, constructs `AsyncAgora`, builds `AgoraAgent` with managed `DeepgramSTT`, `OpenAI`, `MiniMaxTTS`, starts async sessions, stores sessions by `agent_id`, and stops by active session or `client.stop_agent`.
5. Implement `server/src/server.py` with `GET /get_config`, `POST /startAgent`, and `POST /stopAgent`; load `server/.env` relative to the source file.
Expand Down
9 changes: 5 additions & 4 deletions docs/ai/L1/L2/session_lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,11 @@ ConversationComponent.tsx
├─▶ useLocalMicrophoneTrack()
├─▶ usePublish([track])
│
└─▶ new AgoraVoiceAI({
rtcEngine,
rtmConfig: { rtmEngine: rtmClient },
}).subscribeMessage(channel_name)
└─▶ const ai = await AgoraVoiceAI.init({
rtcEngine: client,
rtmEngine: rtmClient,
})
ai.subscribeMessage(channel_name)
Listens: TRANSCRIPT_UPDATED, AGENT_STATE_CHANGED, AGENT_METRICS,
MESSAGE_ERROR, MESSAGE_SAL_STATUS, AGENT_ERROR
```
Expand Down
2 changes: 1 addition & 1 deletion server/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ fastapi>=0.100.0
uvicorn>=0.20.0
requests>=2.31.0
python-dotenv>=1.0.0
agora-agents>=2.3.0
agora-agents>=2.9.0
# Enables httpx to route through a SOCKS proxy (e.g. all_proxy=socks5://...)
socksio>=1.0.0
3 changes: 2 additions & 1 deletion web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"dependencies": {
"@radix-ui/react-dropdown-menu": "^2.1.16",
"@radix-ui/react-slot": "^1.2.4",
"agora-agent-client-toolkit": "1.2.0",
"agora-agent-client-toolkit": "2.10.0",
"agora-agent-uikit": "1.1.0",
"agora-rtc-react": "^2.5.1",
"agora-rtc-sdk-ng": "^4.24.3",
Expand All @@ -30,6 +30,7 @@
},
"devDependencies": {
"@biomejs/biome": "^1.9.0",
"@types/bun": "1.3.12",
"@types/node": "^25.3.1",
"@types/react": "^19.2.14",
"@types/react-dom": "^19.2.3",
Expand Down
3 changes: 2 additions & 1 deletion web/scripts/verify-local-proxy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ type Rewrite = {
}

type LocalServer = {
port: number
port: number | undefined
stop: (closeActiveConnections?: boolean) => void
}

Expand Down Expand Up @@ -112,6 +112,7 @@ async function withStubBackend<T>(run: (baseUrl: string) => Promise<T>) {
}

try {
assert(typeof server.port === 'number', 'Stub backend must listen on a TCP port')
return await run(`http://localhost:${server.port}`)
} finally {
server.stop(true)
Expand Down
2 changes: 1 addition & 1 deletion web/src/components/ConversationComponent.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ export default function ConversationComponent({
try {
const ai = await AgoraVoiceAI.init({
rtcEngine: client,
rtmConfig: { rtmEngine: rtmClient },
rtmEngine: rtmClient,
renderMode: TranscriptHelperMode.TEXT,
enableLog: true,
});
Expand Down
Loading