Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# Quick Start:
# 1. cp .env.example .env
# 2. Add your ANTHROPIC_API_KEY
# 3. Generate DATADR_ENCRYPTION_KEY (see below)
# 3. Generate DATADR_ENCRYPTION_KEY and JWT_SECRET_KEY (see below)
# 4. docker compose up -d
#
# =============================================================================
Expand All @@ -23,6 +23,12 @@ ANTHROPIC_API_KEY=
# Generate with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
DATADR_ENCRYPTION_KEY=

# Key that signs and verifies login tokens (required, at least 32 bytes).
# The API refuses to start without it. Anyone who knows it can mint tokens for any
# user and role, so keep it secret, and rotate it to sign everyone out.
# Generate with: openssl rand -hex 32
JWT_SECRET_KEY=

# -----------------------------------------------------------------------------
# Database Configuration
# -----------------------------------------------------------------------------
Expand Down
29 changes: 29 additions & 0 deletions .flow/tasks/fn-68.7.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"assignee": "bordumbb@gmail.com",
"claim_note": "",
"claimed_at": "2026-09-28T18:42:30.667003Z",
"created_at": "2026-09-28T18:42:30.394050Z",
"depends_on": [],
"epic": "fn-68",
"evidence": {
"commits": [
"c063e9b3"
],
"prs": [
"https://github.com/bordumb/dataing/pull/230"
],
"tests": [
"uv run pytest python-packages/dataing/tests --no-cov",
"uv run pytest python-packages/dataing-ee/tests --no-cov",
"DATABASE_URL=postgresql://test:test@localhost:55473/test uv run pytest python-packages/dataing/tests/integration -m integration --no-cov",
"DATABASE_URL=postgresql://test:test@localhost:55473/test uv run pytest python-packages/dataing-ee/tests/integration -m integration --no-cov",
"uv run mypy python-packages/dataing/src python-packages/dataing-ee/src"
]
},
"id": "fn-68.7",
"priority": 7,
"spec_path": ".flow/tasks/fn-68.7.md",
"status": "done",
"title": "Refuse to run with the default JWT secret",
"updated_at": "2026-09-28T19:16:48.877864Z"
}
23 changes: 23 additions & 0 deletions .flow/tasks/fn-68.7.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# fn-68.7 Refuse to run with the default JWT secret

## Description
core/auth/jwt.py fell back to SECRET_KEY "dev-secret-change-in-production" when JWT_SECRET_KEY was unset, so any deployment without the variable accepted HS256 tokens that anyone who has read the source could forge for any user, org and role. Fail closed instead, and wire a real key into the local dev, demo and compose flows.

## Acceptance
- No token is signed or verified unless JWT_SECRET_KEY is set to at least 32 bytes; no built-in fallback
- The CE and EE APIs refuse to start without it
- just dev/dev-backend/dev-backend-ce/demo, docker-compose api, .env.example, check-env.sh, quickstart docs provide or document the key
- Tests run with a 32+ byte key (no pyjwt InsecureKeyLengthWarning); new tests cover refusal and forged old-default tokens
- CE/EE unit and integration tests pass


## Done summary
- core/auth/jwt.py: SECRET_KEY constant and fallback removed. New jwt_secret_key() returns JWT_SECRET_KEY or raises JWTSecretKeyError (a RuntimeError, exported from core.auth) when it is unset or under 32 bytes; the message says to run `openssl rand -hex 32`. Issuing, verifying, refresh and the SSE ?token= path all call it.
- create_app() calls jwt_secret_key() first, so the CE and EE APIs (create_ee_app builds on create_app) refuse to start. The Temporal worker signs no tokens and still starts without a key.
- Local flows: just dev, dev-backend and dev-backend-ce keep a 32+ byte key from .env or the shell and otherwise use a random key for the run. just demo generates one when neither the shell nor .env has a valid key. docker-compose passes JWT_SECRET_KEY to the api service. .env.example, check-env.sh, test-quickstart.sh, the quickstart and deployment docs require it. The CLI, SDK and notebook only carry API-issued tokens, so they needed no change.
- Tests: the CE and EE conftests setdefault a 32+ byte key, and InsecureKeyLengthWarning dropped to 0 on pyjwt 2.15. New TestSecretKey covers refusal for unset, empty, 31-byte and old-default keys, a 32-byte round trip, and rejection of a token forged with the old key. test_factory.py and the EE test_app cover startup refusal. Red check: with the fallback restored, 6 tests fail.
- Verified: CE 3167 passed, EE 715 passed. On a pgvector:pg16 container on port 55473, CE integration 130 passed and EE integration 16 passed. mypy is clean after syncing the worktree venv, which still had trino 0.336 and pyjwt 2.10 from before #228 and #226.
## Evidence
- Commits: c063e9b3
- Tests: uv run pytest python-packages/dataing/tests --no-cov, uv run pytest python-packages/dataing-ee/tests --no-cov, DATABASE_URL=postgresql://test:test@localhost:55473/test uv run pytest python-packages/dataing/tests/integration -m integration --no-cov, DATABASE_URL=postgresql://test:test@localhost:55473/test uv run pytest python-packages/dataing-ee/tests/integration -m integration --no-cov, uv run mypy python-packages/dataing/src python-packages/dataing-ee/src
- PRs: https://github.com/bordumb/dataing/pull/230
3 changes: 3 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
# Required environment variables:
# ANTHROPIC_API_KEY - Anthropic API key for LLM features
# DATADR_ENCRYPTION_KEY - Fernet key for encrypting datasource credentials
# JWT_SECRET_KEY - 32+ random bytes that sign login tokens (openssl rand -hex 32);
# the api refuses to start without it
#
# Access points:
# Frontend: http://localhost:3000
Expand Down Expand Up @@ -133,6 +135,7 @@ services:
INVESTIGATION_ENGINE: temporal
DATADR_ENCRYPTION_KEY: ${DATADR_ENCRYPTION_KEY:-}
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
JWT_SECRET_KEY: ${JWT_SECRET_KEY:-}
REDIS_HOST: redis
REDIS_PORT: 6379
LLM_MODEL: ${LLM_MODEL:-claude-sonnet-4-20250514}
Expand Down
1 change: 1 addition & 0 deletions docs/docs/development/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ this script already have that login.
| Variable | Value |
|----------|-------|
| `ANTHROPIC_API_KEY` | Your Anthropic API key |
| `JWT_SECRET_KEY` | 32+ random bytes that sign login tokens (`openssl rand -hex 32`); the API refuses to start without it |

### 6. Enable Public URL

Expand Down
12 changes: 12 additions & 0 deletions docs/docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,18 @@
DATADR_ENCRYPTION_KEY=<paste-key-here>
```

Generate the key that signs login tokens (required; the API refuses to start without it):

```bash
openssl rand -hex 32
```

Add it to `.env`:

```bash
JWT_SECRET_KEY=<paste-key-here>
```

Start the stack:

```bash
Expand Down
6 changes: 6 additions & 0 deletions docs/test-quickstart.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,12 @@ if ! grep -q "ANTHROPIC_API_KEY=" .env 2>/dev/null; then
echo "ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}" >> .env
fi

# Generate a JWT signing key unless .env already has one of 32+ bytes
if ! grep -qE '^JWT_SECRET_KEY=.{32,}' .env 2>/dev/null; then
grep -v '^JWT_SECRET_KEY=' .env > .env.tmp && mv .env.tmp .env
echo "JWT_SECRET_KEY=$(openssl rand -hex 32)" >> .env
fi

# Generate encryption key if not present
if ! grep -q "DATADR_ENCRYPTION_KEY=" .env 2>/dev/null; then
KEY=$(python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
Expand Down
9 changes: 9 additions & 0 deletions infra/check-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,15 @@ if [ -z "$ANTHROPIC_API_KEY" ]; then
errors=$((errors + 1))
fi

jwt_key="${JWT_SECRET_KEY:-}"
if [ "${#jwt_key}" -lt 32 ]; then
echo -e "${RED}ERROR: JWT_SECRET_KEY is not set or shorter than 32 bytes${NC}"
echo " The API refuses to start without it: it signs every login token."
echo " Generate with: openssl rand -hex 32"
echo ""
errors=$((errors + 1))
fi

if [ -z "$DATADR_ENCRYPTION_KEY" ]; then
echo -e "${RED}ERROR: DATADR_ENCRYPTION_KEY is not set${NC}"
echo " Datasource credentials cannot be stored without an encryption key."
Expand Down
17 changes: 17 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ dev:
if [ -f .env ]; then
export $(grep -v '^#' .env | xargs)
fi
# The API refuses to start without a JWT signing key of 32+ bytes. Keep the one from
# .env or the shell; otherwise use a fresh one for this run (sign in again after a restart).
jwt_key="${JWT_SECRET_KEY:-}"
if [ "${#jwt_key}" -lt 32 ]; then export JWT_SECRET_KEY="$(openssl rand -hex 32)"; fi

trap 'kill 0' EXIT

Expand All @@ -120,6 +124,10 @@ dev-backend:
export REDIS_PORT=6379
export ENCRYPTION_KEY=ZnxhCyx4-ZjziPWtUguwGOFMMiLNioSwso5-qNPAGZI=
if [ -f .env ]; then export $(grep -v '^#' .env | xargs); fi
# The API refuses to start without a JWT signing key of 32+ bytes. Keep the one from
# .env or the shell; otherwise use a fresh one for this run (sign in again after a restart).
jwt_key="${JWT_SECRET_KEY:-}"
if [ "${#jwt_key}" -lt 32 ]; then export JWT_SECRET_KEY="$(openssl rand -hex 32)"; fi
uv run fastapi dev python-packages/dataing-ee/src/dataing_ee/entrypoints/api/app.py --host 0.0.0.0 --port 8000

# Run CE backend only (no enterprise features). Requires infrastructure.
Expand All @@ -136,6 +144,10 @@ dev-backend-ce:
export REDIS_PORT=6379
export ENCRYPTION_KEY=ZnxhCyx4-ZjziPWtUguwGOFMMiLNioSwso5-qNPAGZI=
if [ -f .env ]; then export $(grep -v '^#' .env | xargs); fi
# The API refuses to start without a JWT signing key of 32+ bytes. Keep the one from
# .env or the shell; otherwise use a fresh one for this run (sign in again after a restart).
jwt_key="${JWT_SECRET_KEY:-}"
if [ "${#jwt_key}" -lt 32 ]; then export JWT_SECRET_KEY="$(openssl rand -hex 32)"; fi
uv run fastapi dev python-packages/dataing/src/dataing/entrypoints/api/app.py --host 0.0.0.0 --port 8000

# Stop dev servers
Expand Down Expand Up @@ -334,6 +346,11 @@ demo: demo-fixtures
#!/usr/bin/env bash
set -euo pipefail
echo "Starting demo stack..."
# The API refuses to start without a JWT signing key of 32+ bytes. docker compose
# takes it from the shell or .env; if neither has one, use a fresh key for this run.
if [ -z "${JWT_SECRET_KEY:-}" ] && ! grep -qsE '^JWT_SECRET_KEY=.{32,}' .env; then
export JWT_SECRET_KEY="$(openssl rand -hex 32)"
fi
# Force recreate db-migrate to ensure seeds run
docker compose -f docker-compose.yml -f demo/docker-compose.demo.yml up -d --build --force-recreate db-migrate
docker compose -f docker-compose.yml -f demo/docker-compose.demo.yml up -d --build
Expand Down
5 changes: 5 additions & 0 deletions python-packages/dataing-ee/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
"""Pytest configuration for dataing-ee tests."""

import os
import sys
from pathlib import Path

import pytest

# The API refuses to sign or verify JWTs without a 32+ byte key (core/auth/jwt.py).
# Set one before any test imports the app.
os.environ.setdefault("JWT_SECRET_KEY", "test-jwt-secret-key-that-is-at-least-32-bytes")

# Add EE package to path for imports
ee_src = Path(__file__).parent.parent / "src"
if str(ee_src) not in sys.path:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import subprocess
import sys

import pytest
from dataing_ee.entrypoints.api.app import app
from fastapi.routing import APIRoute
from fastapi.testclient import TestClient
Expand Down Expand Up @@ -60,3 +61,15 @@ def test_app_has_one_health_route() -> None:
health_routes = [r for r in app.routes if isinstance(r, APIRoute) and r.path == "/health"]

assert len(health_routes) == 1


def test_ee_app_refuses_to_start_without_a_jwt_secret(monkeypatch: pytest.MonkeyPatch) -> None:
"""The EE app, which SSO logins sign tokens for, needs the JWT key as well."""
from dataing_ee.entrypoints.api.app import create_ee_app

from dataing.core.auth.jwt import JWTSecretKeyError

monkeypatch.delenv("JWT_SECRET_KEY", raising=False)

with pytest.raises(JWTSecretKeyError):
create_ee_app()
2 changes: 2 additions & 0 deletions python-packages/dataing/src/dataing/core/auth/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Auth domain types and utilities."""

from dataing.core.auth.jwt import (
JWTSecretKeyError,
TokenError,
create_access_token,
create_refresh_token,
Expand Down Expand Up @@ -29,6 +30,7 @@
"TokenPayload",
"hash_password",
"verify_password",
"JWTSecretKeyError",
"create_access_token",
"create_refresh_token",
"decode_token",
Expand Down
38 changes: 34 additions & 4 deletions python-packages/dataing/src/dataing/core/auth/jwt.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,42 @@ class TokenError(Exception):
pass


class JWTSecretKeyError(RuntimeError):
"""Raised when JWT_SECRET_KEY is missing or too short to sign tokens safely."""


# Configuration
SECRET_KEY = os.environ.get("JWT_SECRET_KEY", "dev-secret-change-in-production")
JWT_SECRET_KEY_ENV = "JWT_SECRET_KEY" # pragma: allowlist secret
# HS256 wants a key at least as long as its 32-byte hash output (RFC 7518, section 3.2)
MIN_SECRET_KEY_BYTES = 32
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 60 * 24 # 24 hours
REFRESH_TOKEN_EXPIRE_DAYS = 7


def jwt_secret_key() -> str:
"""Return the key that signs and verifies JWTs.

There is deliberately no built-in fallback: a key that ships in the source lets
anyone who reads it forge tokens for any user, org and role.

Returns:
The JWT_SECRET_KEY environment variable.

Raises:
JWTSecretKeyError: If JWT_SECRET_KEY is unset or shorter than 32 bytes.
"""
key = os.environ.get(JWT_SECRET_KEY_ENV, "")
size = len(key.encode())
if size < MIN_SECRET_KEY_BYTES:
problem = "is not set" if not key else f"is only {size} bytes"
raise JWTSecretKeyError(
f"{JWT_SECRET_KEY_ENV} {problem}; it must be at least {MIN_SECRET_KEY_BYTES} "
"random bytes. Generate one with: openssl rand -hex 32"
)
return key


def create_access_token(
user_id: str,
org_id: str,
Expand Down Expand Up @@ -50,7 +79,7 @@ def create_access_token(
"iat": int(now.timestamp()),
}

return jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
return jwt.encode(payload, jwt_secret_key(), algorithm=ALGORITHM)


def create_refresh_token(user_id: str) -> str:
Expand All @@ -75,7 +104,7 @@ def create_refresh_token(user_id: str) -> str:
"type": "refresh",
}

return jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
return jwt.encode(payload, jwt_secret_key(), algorithm=ALGORITHM)


def decode_token(token: str) -> TokenPayload:
Expand All @@ -89,9 +118,10 @@ def decode_token(token: str) -> TokenPayload:

Raises:
TokenError: If token is invalid or expired
JWTSecretKeyError: If JWT_SECRET_KEY is missing or too short
"""
try:
payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
payload = jwt.decode(token, jwt_secret_key(), algorithms=[ALGORITHM])
return TokenPayload(
sub=payload["sub"],
org_id=payload["org_id"],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from fastapi.middleware.cors import CORSMiddleware
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor

from dataing.core.auth.jwt import jwt_secret_key
from dataing.telemetry import CorrelationMiddleware, configure_logging, init_telemetry

from .deps import lifespan
Expand All @@ -24,7 +25,14 @@ def create_app() -> FastAPI:

Returns:
Configured FastAPI application instance.

Raises:
JWTSecretKeyError: If JWT_SECRET_KEY is missing or too short. The API
signs and verifies every login token with it, so it does not start
without one.
"""
jwt_secret_key()

# Initialize OpenTelemetry SDK (idempotent, safe to call multiple times)
init_telemetry()

Expand Down
5 changes: 5 additions & 0 deletions python-packages/dataing/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,17 @@
from __future__ import annotations

import logging
import os
import sys
from collections.abc import Iterator
from pathlib import Path

import pytest

# The API refuses to sign or verify JWTs without a 32+ byte key (core/auth/jwt.py).
# Set one before any test imports the app.
os.environ.setdefault("JWT_SECRET_KEY", "test-jwt-secret-key-that-is-at-least-32-bytes")

# Add CE package to path for imports
ce_src = Path(__file__).parent.parent / "src"
if str(ce_src) not in sys.path:
Expand Down
Loading
Loading