RedTeamAI

An autonomous AI red team platform that attacks LLM applications to find security vulnerabilities. It uses LangGraph to run a 5-step attack pipeline -- reconnaissance, payload generation, mutation, execution, and LLM-as-judge evaluation -- and learns from past successes to generate smarter attacks over time.

Live at: https://redteam.prismbrain.co Dashboard: http://redteam.prismbrain.co:8514

What It Does

RedTeamAI does not use hardcoded attack vectors. Every attack is generated fresh by an LLM based on the target's behavior, past successful techniques, and the attack category. It gets smarter with every attack it runs.

• Reconnaissance - Before attacking, probes the target with a normal message and a mildly harmful message to detect if guardrails are active. This intelligence is used to craft a harder-to-detect attack.
• LLM-driven payload generation - GPT-4o-mini generates a brand new attack payload every time based on the category, recon findings, and past successes.
• Intelligent mutation - A second LLM call picks the best mutation technique (social engineering, encoding, authority escalation, hypothetical framing) to make the payload harder to detect.
• LLM-as-judge evaluation - A separate conservative LLM judges whether the attack succeeded. Never trusts the target's own response.
• Learning - Successful payloads are stored in PostgreSQL and fed as examples to future attacks in the same category. The platform gets smarter over time.

Attack Categories

• jailbreak - Trick the AI into producing harmful content it should refuse
• prompt_injection - Override the AI's system instructions
• data_exfiltration - Trick the AI into leaking sensitive information
• system_prompt_leakage - Extract the AI's hidden system instructions
• encoding_obfuscation - Hide malicious instructions inside encoded or translated text

Compatible Targets

RedTeamAI works with any LLM API that accepts HTTP requests in OpenAI's format. Pass the model name when adding a target.

Provider	API URL	Model Name Example
OpenAI	https://api.openai.com/v1/chat/completions	gpt-4o-mini
Groq (free)	https://api.groq.com/openai/v1/chat/completions	llama-3.1-8b-instant
Together AI	https://api.together.xyz/v1/chat/completions	meta-llama/Llama-3-8b-chat-hf
Your own API	https://your-api.com/v1/chat/completions	whatever your API uses

---

Using the Hosted Version (redteam.prismbrain.co)

No setup required. The hosted version is live and ready to use.

Step 1 - Open the dashboard:

Go to https://redteam.prismbrain.co:8514

Step 2 - Add a target:

Go to the Targets page and fill in:

• Target Name: Any name (e.g. "My GPT Chatbot")
• API URL: Your LLM API endpoint
• API Key: Your API key for that provider
• Model Type: openai (for OpenAI-compatible APIs)
• Model Name: The exact model name (e.g. llama-3.1-8b-instant for Groq)

Step 3 - Run an attack:

Go to Run Attack, select your target, pick a category, set max attempts (1-5), and click Launch Attack. Each attack takes 15-45 seconds.

Step 4 - View results:

The Results page shows every attack with:

• Whether guardrails were detected during recon
• The original payload and mutated payload
• The target's full response
• The evaluator's reasoning

Step 5 - Check statistics:

The Statistics page shows success rates by category, severity distribution, guardrail detection rates, and the top successful techniques.

---

Self-Hosting Setup

Follow these steps to run your own instance.

Requirements:

• Python 3.11+
• PostgreSQL
• An OpenAI API key (used by the attack and evaluator LLMs)

1. Clone the repo:

git clone https://github.com/MSaiRam10/RedTeamAI.git
cd RedTeamAI

2. Create a virtual environment and install dependencies:

python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt

3. Create the PostgreSQL database:

psql -U postgres -c "CREATE USER redteam WITH PASSWORD 'redteam123';"
psql -U postgres -c "CREATE DATABASE redteamai OWNER redteam;"

4. Create .env in the root folder:

OPENAI_API_KEY=your-openai-api-key
DATABASE_URL=postgresql://redteam:redteam123@localhost:5432/redteamai

Note: OPENAI_API_KEY is used internally by the attack and evaluator LLMs -- not the target. The target uses whatever API key you provide when adding it as a target.

5. Start the backend:

uvicorn main:app --reload --port 8000

6. Start the dashboard in a separate terminal:

streamlit run dashboard.py --server.port 8501

7. Open the dashboard:

Go to http://localhost:8501

8. Add a target and run attacks:

Add any LLM API as a target. For a free option, sign up at console.groq.com and use:

• API URL: https://api.groq.com/openai/v1/chat/completions
• Model Name: llama-3.1-8b-instant

---

Docker Setup

1. Create .env in the root folder:

OPENAI_API_KEY=your-openai-api-key
DATABASE_URL=postgresql://redteam:redteam123@db:5432/redteamai

2. Run with Docker Compose:

docker-compose up --build -d

This starts:

• PostgreSQL on port 5435
• FastAPI backend on port 8000 (http://localhost:8000)
• Streamlit dashboard on port 8501 (http://localhost:8501)

---

API Endpoints

All endpoints available at http://localhost:8000. Full docs at http://localhost:8000/docs.

• POST /targets - Add a new target
• GET /targets - List all targets
• DELETE /targets/{id} - Delete a target
• POST /attacks - Run an attack
• GET /attacks - List all attacks
• GET /stats - Get statistics
• GET / - Health check

Example -- add a target:

curl -X POST http://localhost:8000/targets \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Llama 3.1 via Groq",
    "api_url": "https://api.groq.com/openai/v1/chat/completions",
    "api_key": "your-groq-key",
    "model_type": "openai",
    "model_name": "llama-3.1-8b-instant"
  }'

Example -- run an attack:

curl -X POST http://localhost:8000/attacks \
  -H "Content-Type: application/json" \
  -d '{
    "target_id": 1,
    "category": "jailbreak",
    "max_attempts": 3
  }'

Stack

• FastAPI - REST API backend
• LangGraph - 5-node attack pipeline orchestration
• LangChain + OpenAI - LLM-driven payload generation and evaluation
• PostgreSQL - Attack history and learning storage
• Streamlit - Web dashboard
• httpx - Async HTTP client for attacking targets
• Docker + Docker Compose - Containerized deployment

GitHub

github.com/MSaiRam10/RedTeamAI