Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Each example only needs its own language toolchain. To work across the whole rep
./tests/smoke-test.sh
```

The smoke tests verify that every server starts and answers MCP requests (including validating structured tool results against the schemas each tool advertises), and that the Python, TypeScript, and Ruby clients can connect to a mock server and list tools. They run automatically on every pull request via GitHub Actions.
The smoke tests verify that every server starts and answers MCP requests (including validating structured tool results against the schemas each tool advertises), and that every client can connect to a mock server and list tools. They run automatically on every pull request via GitHub Actions.

## Contributing

Expand Down
4 changes: 3 additions & 1 deletion mcp-client-rust/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,9 @@ async fn main() -> Result<()> {
// does. Matching the Python and TypeScript clients, report and exit
// rather than failing, so the connection itself can be exercised
// without a key.
if std::env::var("ANTHROPIC_API_KEY").is_err() {
// Empty counts as unset, as it does in the other clients: `KEY= cmd`
// is how the smoke test forces this path.
if std::env::var("ANTHROPIC_API_KEY").map_or(true, |key| key.is_empty()) {
println!("\nNo ANTHROPIC_API_KEY found. To query these tools with Claude, set your API key:");
println!(" export ANTHROPIC_API_KEY=your-api-key-here");
return Ok(());
Expand Down
6 changes: 3 additions & 3 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,7 @@ This directory contains smoke tests for the MCP quickstart examples. These tests
The smoke tests verify:

- **Servers**: Each weather server (Python, TypeScript, Rust, Go, Ruby) can start, respond to MCP protocol requests, and honour the output schemas it advertises
- **Clients**: The Python, TypeScript and Ruby MCP clients can connect to a mock server and list tools

The Go and Rust clients are not covered here: on `main` both abort when no `.env` file is present, so they cannot be driven without credentials. Making them start credential-free is a change in their own directories, so their coverage lands with those changes rather than here.
- **Clients**: Each MCP client (Python, TypeScript, Ruby, Go, Rust) can connect to a mock server and list tools

## Structured content

Expand Down Expand Up @@ -64,6 +62,8 @@ Each client test:

**Note**: Client tests run the actual CLI programs without an Anthropic API key. The clients are designed to handle missing API keys gracefully by listing available tools and exiting, which is perfect for smoke testing the MCP connectivity without requiring external API calls.

The key is set to the empty string rather than unset. Every client loads `.env` without overriding variables already in the environment, so an empty value keeps a developer's local `.env` from starting the chat loop, and every client treats an empty key the same as a missing one.

## Test Helpers

### mcp-test-client.ts
Expand Down
46 changes: 37 additions & 9 deletions tests/smoke-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,15 @@ test_weather_server_typescript() {
check_dependency node || return 1
check_dependency npm || return 1
local server_dir="${PROJECT_ROOT}/weather-server-typescript"
ensure_built "${server_dir}" || return 1
ensure_built "${server_dir}" "" || return 1
node "${TEST_CLIENT}" node "${server_dir}/build/index.js"
}

# Test: Rust weather server
test_weather_server_rust() {
check_dependency cargo || return 1
local server_dir="${PROJECT_ROOT}/weather-server-rust"
ensure_built "${server_dir}" || return 1
ensure_built "${server_dir}" weather || return 1

# Determine which binary to use
local server_bin
Expand All @@ -73,7 +73,7 @@ test_weather_server_rust() {
test_weather_server_go() {
check_dependency go || return 1
local server_dir="${PROJECT_ROOT}/weather-server-go"
ensure_built "${server_dir}" || return 1
ensure_built "${server_dir}" server || return 1

local server_bin
server_bin=$(resolve_binary "${server_dir}/server") || {
Expand Down Expand Up @@ -109,7 +109,7 @@ test_mcp_client_typescript() {
check_dependency node || return 1
check_dependency npm || return 1
local client_dir="${PROJECT_ROOT}/mcp-client-typescript"
ensure_built "${client_dir}" || return 1
ensure_built "${client_dir}" "" || return 1
ANTHROPIC_API_KEY= node "${client_dir}/build/index.js" "${MOCK_SERVER}" >/dev/null 2>&1
}

Expand All @@ -122,12 +122,38 @@ test_mcp_client_ruby() {
(cd "${client_dir}" && ANTHROPIC_API_KEY= bundle exec ruby client.rb "${MOCK_SERVER}") >/dev/null 2>&1
}

# Test: Go MCP client
test_mcp_client_go() {
check_dependency go || return 1
local client_dir="${PROJECT_ROOT}/mcp-client-go"
ensure_built "${client_dir}" mcp-client-go || return 1

local client_bin
client_bin=$(resolve_binary "${client_dir}/mcp-client-go") || {
print_error "no mcp-client-go binary found in ${client_dir}"
return 1
}

ANTHROPIC_API_KEY= "${client_bin}" node "${MOCK_SERVER}" >/dev/null 2>&1
}

# Test: Rust MCP client
test_mcp_client_rust() {
check_dependency cargo || return 1
local client_dir="${PROJECT_ROOT}/mcp-client-rust"
ensure_built "${client_dir}" mcp-client-rust || return 1

local client_bin
client_bin=$(resolve_binary "${client_dir}/target/release/mcp-client-rust" \
|| resolve_binary "${client_dir}/target/debug/mcp-client-rust") || {
print_error "no mcp-client-rust binary found in ${client_dir}/target"
return 1
}

ANTHROPIC_API_KEY= "${client_bin}" node "${MOCK_SERVER}" >/dev/null 2>&1
}

# Run all tests
#
# The Go and Rust clients are not covered: on main both abort when no .env file
# is present, so they cannot be driven without credentials. Making them start
# credential-free is a change in their own directories, so their coverage lands
# with those changes rather than here.

print_header "Running smoke tests"
run_test "weather-server-python" test_weather_server_python
Expand All @@ -138,6 +164,8 @@ run_test "weather-server-ruby" test_weather_server_ruby
run_test "mcp-client-python" test_mcp_client_python
run_test "mcp-client-typescript" test_mcp_client_typescript
run_test "mcp-client-ruby" test_mcp_client_ruby
run_test "mcp-client-go" test_mcp_client_go
run_test "mcp-client-rust" test_mcp_client_rust

# Print summary
echo ""
Expand Down
13 changes: 9 additions & 4 deletions tests/utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,16 @@ run_build() {

# Ensure a project directory is built (TypeScript/Rust/Go)
#
# `bin` is the name of the binary a Rust or Go project produces (unused for
# TypeScript). Rust names it after the package, so it is looked for under
# target/; Go is told to write it to `bin` in the project directory.
#
# Returns rather than exits, for the same reason check_dependency does: one
# project failing to build should fail its own test, not abort the suite
# before the summary is printed.
ensure_built() {
local dir=$1
local bin=$2
cd "${dir}" || return 1

# Install npm dependencies if needed
Expand All @@ -128,14 +133,14 @@ ensure_built() {

# Build Rust if needed
if [ -f "Cargo.toml" ] \
&& ! resolve_binary "target/release/weather" >/dev/null \
&& ! resolve_binary "target/debug/weather" >/dev/null; then
&& ! resolve_binary "target/release/${bin}" >/dev/null \
&& ! resolve_binary "target/debug/${bin}" >/dev/null; then
run_build "cargo build in ${dir}" cargo build --release || return 1
fi

# Build Go if needed
if [ -f "go.mod" ] && ! resolve_binary "server" >/dev/null; then
run_build "go build in ${dir}" go build -o "server$(exe_suffix)" . || return 1
if [ -f "go.mod" ] && ! resolve_binary "${bin}" >/dev/null; then
run_build "go build in ${dir}" go build -o "${bin}$(exe_suffix)" . || return 1
fi
}

Expand Down
Loading