Minecraft AI Companion

A Minecraft agent that connects natural-language requests to game actions.

A two-process prototype built with Python, LangGraph, Node.js and Mineflayer. A task agent plans and executes game skills; a separate interaction agent handles player chat and expressive actions. Both consume the same live world state.

中文：一个通过游戏内聊天交流的 Minecraft AI 搭子，重点探索任务规划、工具执行与实时互动之间的协作。

What this project explores

Crafting an item involves a sequence of dependent actions: inspect a recipe, gather resources, reach a workbench, then craft. This project gives an LLM explicit game tools and feeds their results back into a ReAct loop. Separating task planning from conversation lets each use its own prompt and sampling settings.

The implementation includes resource collection, crafting, navigation, equipment, combat, block placement and chat interactions. These are available skills, not a claim that every natural-language request succeeds.

Architecture

flowchart LR
    World["Minecraft server"] <--> Bot["Mineflayer skills + HTTP API"]
    Bot -->|"WebSocket state and events"| State["Shared state"]
    State --> Task["LangGraph task agent"]
    State --> Chat["Interaction agent"]
    Chat -->|"pending goal"| State
    Task -->|"HTTP skill calls"| Bot
    Chat -->|"HTTP skill calls"| Bot

Loading

Component	Implementation	Responsibility
Game bridge	bot/bot_server.js	Minecraft connection, /skill, /recipe/:item, /ping, and /state WebSocket
Game skills	bot/skills/index.js	Concrete Mineflayer actions and their results
Task agent	brain/game_graph.py	LangGraph tool loop, live context and bounded graph execution
Interaction agent	brain/interaction_brain.py	Chat interpretation, replies, gestures and task routing
State and transport	shared_state.py · bot_client.py	World snapshots, event queue and Python–Node communication

World state is broadcast every 500 ms. The task agent disables parallel tool calls because actions depend on changing inventory and world state. This is per-agent sequencing; the two agents do not yet share a global action scheduler. Both use DeepSeek's deepseek-chat API.

Run locally

Use a local Minecraft Java test server compatible with Mineflayer, Node.js, Python 3.10+ and a DeepSeek API key. The bot currently uses offline authentication; the server must be configured accordingly and kept on a trusted test network. The HTTP control API has no authentication.

From the repository root:

npm --prefix bot ci
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r brain/requirements.txt

Set the Minecraft host, port and bot name in the configuration section of bot_server.js. Defaults are 127.0.0.1:25565 and AIBot. The Python bridge expects the bot API at 127.0.0.1:3000.

With the Minecraft server running, open two terminals in the repository root:

# Terminal A: game bridge
npm --prefix bot start

# Terminal B: AI agents
source .venv/bin/activate
export DEEPSEEK_API_KEY="YOUR_API_KEY"
python brain/main.py

Supply your own key through the environment or an untracked brain/.env file. Do not put credentials in Python source. Starting the agents begins the default goal, survive and build a shelter, and makes paid API calls.

Try it and inspect the result

1. Confirm the bot has spawned in your test world. /ping only checks the HTTP process; it does not establish that the bot is ready in-game.
2. Enter a small request in Minecraft chat, such as 采集 3 个橡木原木.
3. Compare tool logs with the bot's actual inventory and movement. A model's completion message is not an independent success check.
4. Try a conversational message such as 你好 to exercise the interaction path.

Current limits and next steps

• This is a prototype without a published task-success benchmark or automated end-to-end game test suite.
• Navigation and resource availability depend on the world. API failures, unreachable blocks and interrupted actions can prevent completion.
• Interaction and task actions can compete for control. Shared scheduling and cancellation are the next coordination improvements.
• The Node process exits after a game disconnect; automatic restart requires an external process supervisor.
• A useful next evaluation is a fixed set of seeded worlds and tasks, with inventory-based completion checks and saved action traces.

License and acknowledgements

MIT. Built on Mineflayer, its pathfinding/collection/combat plugins, and LangGraph. Minecraft is a Mojang/Microsoft product; this is an independent project.