Duke Kunshan University
Gihun Lee
Sandro Korkotashvili
This project implements a socket-based multi-user chatroom and extends it with coordinated large language model participants. The system combines a shared TCP chat server, human-operated clients, and an orchestration layer that manages multiple AI users within the same room. The primary objective is to examine how multiple participants can interact through a common chat protocol while preserving coherent turn-taking, explicit event handling, and stable runtime behavior.
The repository is organized around two stages. Stage 1 provides manual multi-user chatroom validation through separate terminal sessions. Stage 2 introduces coordinated LLM participation under two scenarios: a mixed room containing one human user and two AI users, and an autonomous room containing three AI users controlled by a single orchestration process.
Ensure that Assignment2/.env contains valid OpenAI-compatible runtime settings before executing any Stage 2 scenario.
- Python 3
- A working virtual environment
- Installed dependencies from
requirements.txt - Valid values for
OPENAI_API_KEY,OPENAI_BASE_URL, andOPENAI_MODELin.env
On macOS:
cd Assignment2
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtOn Windows PowerShell:
cd Assignment2
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txtStage 1 launches one server terminal and three human client terminals. Each client joins automatically with a distinct username, allowing direct manual interaction across separate windows.
On macOS:
./scripts/mac/run_stage1.shOn Windows PowerShell:
.\scripts\windows\run_stage1.ps1This scenario starts the chat server, waits for a human user to join through src/client_human.py, and then launches two AI users into the same room.
On macOS:
./scripts/mac/run_stage2_1.shOn Windows PowerShell:
.\scripts\windows\run_stage2_1.ps1This scenario starts the chat server and launches three coordinated AI users controlled by a single orchestrator process.
On macOS:
./scripts/mac/run_stage2_2.shOn Windows PowerShell:
.\scripts\windows\run_stage2_2.ps1If a different local port is required, prepend TEST_PORT=... on macOS:
TEST_PORT=8891 ./scripts/mac/run_stage1.sh
TEST_PORT=8891 ./scripts/mac/run_stage2_1.sh
TEST_PORT=8891 ./scripts/mac/run_stage2_2.shOn Windows PowerShell, pass a different Port value:
.\scripts\windows\run_stage1.ps1 -Port 8891
.\scripts\windows\run_stage2_1.ps1 -Port 8891
.\scripts\windows\run_stage2_2.ps1 -Port 8891Runtime logs are written to the following standardized files:
logs/stage1.loglogs/stage2_1.loglogs/stage2_2.log
Debug tracing in the orchestrator can be controlled through CHATBOT_DEBUG_FLOW in .env.
CHATBOT_DEBUG_FLOW=0Set it to 1 to re-enable detailed debug output.
The system is composed of three principal runtime components.
src/server.pyimplements the shared chatroom server. It accepts TCP connections, registers usernames, and broadcasts room events such as welcome, join, leave, and chat messages.src/client_human.pyimplements an interactive human client. It connects to the server, submits a username, receives room messages, and forwards terminal input to the shared chatroom.src/client_llm.pyimplements the orchestration layer for AI users. A single process manages multiple bot identities, maintains shared room state, selects an appropriate responder, constructs prompts, and submits completion requests to the configured language model backend.
In Stage 2, each AI participant owns an independent socket connection and username, allowing the server to treat each AI user as a distinct chatroom member. Coordination is therefore performed at the client side rather than at the server layer.
The orchestration layer does not operate directly on unstructured incoming text. Each raw line received from the server is first mapped into a structured room event so that later logic can reason over explicit message types and speaker roles.
The event model distinguishes the following categories:
welcome: a directed admission message such asWelcome X to the chatroom!join: a room-wide notification such asX has joined the chatroom!leave: a room-wide notification such asX has left the chatroom!chat: a standard user message in the formName: messageother: any line that does not match the expected room protocol
Each stored RoomEvent records an event identifier, the original raw line, parsed event type, speaker, message text, optional target user, and a monotonic timestamp. This structure supports deterministic ordering, explicit welcome handling, and timing-based coordination.
Per-bot runtime information is stored separately in BotState. This state includes admission status, generation status, cooldown timing, most recent speaking turn, response targeting metadata, and connection references required for send and receive operations.
The end-to-end behavior of the coordinated chatroom proceeds as follows.
- A client connects to the server and sends a username as the first protocol line.
- The server registers the client and emits room events for admission and later chat activity.
- In
Stage 2, each AI connection runs a receiver loop that forwards incoming lines to the orchestrator. - The orchestrator parses each line into a structured
RoomEventand stores it in the shared transcript. - Meaningful room activity triggers the scheduling process.
- After a short debounce interval, the scheduler selects a target event and chooses the most suitable responder.
- The selected bot constructs a prompt from recent room state and issues a completion request to the model backend.
- If the primary model request fails, the orchestrator retries the same prompt against a small fallback model list before treating the generation attempt as failed.
- The generated reply is sanitized, delayed with response timing controls, and transmitted back to the chat server.
- Each AI participant tracks both its own session duration and total number of sent replies. When a configured limit is reached, the bot sends a short natural goodbye and is excluded from future responder selection.
- The server broadcasts the resulting message to all connected participants, and the coordination cycle repeats.
This execution model preserves a standard chatroom protocol while allowing the AI layer to apply centralized response management.
The runtime design relies on a multi-threaded coordination model.
- Each AI user owns a dedicated receiver thread responsible for socket input.
- The orchestrator maintains one scheduler thread that waits for meaningful room activity and evaluates response opportunities.
- Actual model generation occurs in a worker thread spawned after responder selection.
Transcriptis protected by a lock to allow safe concurrent insertion and snapshot access.- Bot runtime metadata is guarded by a separate state lock because cooldowns, welcome flags, and generation state may be read or written from multiple threads.
- A generation lock ensures that only one AI generation task is active at a given time within the orchestrator.
- A shared
stop_eventprovides a common shutdown signal across receiver loops, scheduler logic, and cleanup paths.
This arrangement separates network I/O, scheduling, and generation while preserving a single coordination policy for all AI users.
The project includes explicit response-control mechanisms intended to reduce unstable or visibly artificial interaction patterns.
- Duplicate suppression prevents near-identical raw lines from entering the transcript repeatedly within a short interval.
- Scheduler debounce delays responder selection slightly so bursty room activity can be grouped before a decision is made.
- Reply jitter and additional delay reduce unnaturally immediate AI responses.
- Post-send cooldown prevents a recently active AI user from monopolizing the conversation.
MAX_AI_REPLIES_PER_HUMAN_TURNlimits how many AI turns may follow a single human turn before the system waits for new human input.- Repeated opener detection discourages shallow loops such as repeated greetings or repeated conversational resets.
- Candidate scoring penalizes over-active bots and the most recent AI speaker in order to distribute turns more evenly.
- Prompt instructions constrain tone, sentence count, repetition, and non-chat artifacts such as emojis or decorative formatting.
- Output sanitization removes unwanted formatting and applies message-length controls before transmission.
- Model fallback handling allows the orchestrator to retry failed completion requests with alternate compatible models instead of failing immediately on the first provider error.
- Per-bot session limits cap both the number of generated replies and total runtime duration; when a limit is reached, the bot exits naturally with a short goodbye rather than disappearing abruptly.
These mechanisms do not guarantee perfect realism. They do, however, provide explicit safeguards against the most common failure modes observed in naive multi-agent chat generation.
| Challenge | Cause | Resolution | Result |
|---|---|---|---|
| AI participants could respond independently to the same conversational moment, producing overlapping turns and unstable interaction order. | A naive multi-bot design allows each bot to react only from its own local perspective, without a shared decision policy. | The system introduced a centralized orchestrator with shared transcript state, responder selection, and guarded generation scheduling. | Turn-taking became more controlled, and simultaneous reply collisions were substantially reduced. |
| Raw chatroom lines did not provide sufficient structure for reliable downstream coordination. | Welcome, join, leave, and chat messages were all initially received as plain text, which made event-specific handling error-prone. | The orchestration layer introduced explicit event parsing through RoomEvent and stored room activity in a structured Transcript. |
Response logic could reason over event type, speaker, target, and timing directly rather than relying on repeated string-level heuristics. |
| The conversation could become trapped in repetitive greeting cycles or shallow response loops. | Multi-agent language generation tends to reinforce recent phrasing unless repetition controls are enforced. | The system added duplicate suppression, debounce, cooldowns, repeated opener detection, anchor selection, and per-turn reply limits. Prompt instructions were also revised to discourage repeated conversational patterns. | The interaction became more varied and more stable over longer exchanges. |
| AI generation could fail abruptly when a single configured model was unavailable, and unattended runs could continue longer than desired. | A single-model dependency is brittle under rate limits or provider-side failures, while autonomous multi-bot rooms need explicit stopping conditions to remain predictable. | The orchestrator now retries generation with fallback models and enforces per-bot message and duration limits followed by a short goodbye before retirement. | Stage 2 scenarios became more robust during transient backend failures and more predictable during demos and automated runs. |
| Runtime execution could fail when the selected Python environment or inference backend was not operationally aligned with the project configuration. | Multiple local virtual environments may not contain the same installed packages, and model backends may reject requests under resource pressure. | The execution scripts now prefer environments that satisfy required dependencies, and logging was standardized to make runtime failures easier to diagnose. | Operational issues became easier to isolate, and scenario scripts became more predictable across repeated runs. |