______ __ _____ __ _ __ __
/ ____/________ _ ______/ // ___// /_ (_)__ / /____/ /
/ / / ___/ __ \ | /| / / __ / \__ \/ __ \ / / _ \/ // __ /
/ /___/ / / /_/ / |/ |/ / /_/ / ___/ / / / // / __/ // /_/ /
\____/_/ \____/|__/|__/\__,_/ /____/_/ /_//_/\___/_/ \__,_/
Predicting crowd crush risk 6β15 minutes before it occurs β transforming reactive CCTV surveillance into proactive, life-saving automated interventions.
Overview β’ Key Differentiators β’ Architecture β’ Engines Breakdown β’ Data Contract β’ Demo Scenarios β’ Quick Start β’ API Docs β’ Privacy & Ethics β’ Judging Criteria β’ Roadmap β’ Contributing
Every year, catastrophic crowd stampedes and crush disasters at religious pilgrimages, music festivals, transit hubs, and sporting arenas claim hundreds of preventable lives..
Current venue security relies on human monitors looking at dozens of CCTV feeds or reactive density heatmaps. The critical failure mode is universal:
-
Reactive, not predictive: Operators only notice critical density after physical gridlock and panic have already set in (
$>6\text{ people/m}^2$ ). - No actionable escalation: Heatmaps show where it is crowded, but provide zero guidance on which gates to open, where to redirect flow, or how to communicate calmly with attendees.
- Latency kills: Once crowd pressure exceeds the critical physical threshold, evacuation routes choke, shockwaves propagate, and intervention is impossible.
TRADITIONAL SURVEILLANCE:
[ CCTV Feed ] βββΊ [ Operator Looks at Screen ] βββΊ [ Realizes Crush Occurred ] βββΊ [ Evacuation Failed ] (TOO LATE)
CROWDSHIELD PROACTIVE INTERVENTION:
[ Video/Stream ] βββΊ [ Vision Engine ] βββΊ [ Risk Engine ] βββΊ [ 4-Layer LLM ] βββΊ [ Automated SOPs + App Alerts ]
(Density + Flow) (ETA: 6-15 min) (Bilingual PA) (Actuates 10m BEFORE Peak)
CrowdShield continuously extracts crowd density (
Before a crowd reaches crush thresholds, CrowdShield:
- Alerts operators with specific topological standard operating procedures (e.g., "Open Gate 2 as relief corridor, deploy 4 marshals to South Entrance").
- Actuates venue digital twins to monitor real-time gate statuses and diverted crowd corridors.
- Generates calm, contextual bilingual public address announcements (English & Hindi) via a resilient 4-Layer LLM cascade.
- Dispatches push notifications directly to attendees and gate marshals via the mobile app.
- I'm Safe Check-in A floating green shield button on the Home screen that lets citizens mark themselves as safe during a crowd emergency. Tapping it opens a modal showing your current zone, timestamp, and last check-in time. Includes a Share button that sends "I am safe at [zone] as of [time]. Sent via CrowdShield." via any messaging app. The button pulses green for 30 minutes after a check-in. Check-in data is stored locally via AsyncStorage.
- Evacuation Route A "Get Evacuation Route" button on the My Zone screen that gives real-time exit guidance based on your zone's current risk level. Low/medium zones show the nearest exit is safe. High risk zones list specific recommendations from the live feed as numbered steps. Critical zones show a bold EVACUATE NOW alert with the bilingual announcement. Includes a visual venue map with your zone highlighted and a directional exit arrow. Has a Share button to send your evacuation route to family/contacts.
| Feature | Legacy CCTV / Heatmap Systems | CrowdShield Autonomous Platform |
|---|---|---|
| Detection Philosophy | Reactive: Alerts when density is already dangerous | Predictive: Forecasts dangerous surges 6β15 minutes in advance |
| Scoring Explainability | Black-box deep learning or opaque thresholds | Transparent mathematical formulation (density + gradient + flow deceleration + bottleneck penalty) |
| Operator Assistance | Raw video monitoring with no SOP recommendations | Actionable topological recommendations (specific alternate gates & routes) |
| Public Communication | Delayed, manual, panic-inducing megaphone alerts | Real-time bilingual PA generation (English & Hindi) via 4-Layer LLM Cascade |
| Resilience & Cost | Fragile paid API locks, single-point failures | 100% Free-Tier & Zero-Network Local Fallback (never crashes or rate-limits) |
| Privacy Compliance | Stores facial footage and raw video streams | Zero raw video persistence, zero facial recognition (ephemeral edge metadata only) |
| Multi-Stakeholder Sync | Siloed control room monitors | Real-time synchronized WebSocket broadcast across Web Dashboard & Mobile Apps |
CrowdShield operates as a modular, decoupled streaming ecosystem where every component synchronizes over high-speed WebSockets using a strictly typed data contract:;
flowchart TB
subgraph INGESTION["1. SENSORY INGESTION & VISION"]
A1[RTSP Cameras / Sample Video Feeds] --> B1[Vision Engine]
B1 --> B2[Modular Detector\nHOG / YOLOv8 / CSRNet]
B1 --> B3[Lucas-Kanade & Farneback\nOptical Flow Velocity Tracker]
B2 & B3 --> B4[ZoneDensityEstimate\nDensity p/mΒ² + Velocity m/s + Direction]
end
subgraph RISK["2. PREDICTION & REASONING"]
B4 --> C1[Risk Prediction Engine]
C1 --> C2["Explainable Scoring Formula\nSd + St (Gradient) + Sf + Pb (Bottleneck)"]
C2 --> C3["ETA-to-Critical Derivation\n(Minutes Remaining)"]
C3 --> D1[Recommendation Engine]
D1 --> D2[Topological Rule Layer\nGate Adjacency & Rerouting SOPs]
D1 --> D3["4-Layer LLM Announcement Cascade\nGemini β Groq β Cohere β Rule Templates"]
end
subgraph PIPELINE["3. REALTIME PIPELINE SERVER"]
D2 & D3 --> E1[FastAPI Pipeline Hub]
E1 --> E2[Shared In-Memory State]
E1 --> E3[WebSocket Broadcaster\n/ws/risk-events]
E1 --> E4[REST Endpoints\n/events/latest, /ai/summary, /demo/scenario]
end
subgraph CONSUMERS["4. MULTI-STAKEHOLDER CLIENTS"]
E3 & E4 --> F1[Safety Command Dashboard\nNext.js 16 + React 19 + Leaflet + Digital Twin]
E3 & E4 --> F2[Citizen & Marshal Mobile App\nReact Native + Expo SDK 51]
E4 --> F3[AI Voice Assistant & Auto-Dispatch]
end
style INGESTION fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#fff
style RISK fill:#1e293b,stroke:#818cf8,stroke-width:2px,color:#fff
style PIPELINE fill:#1e293b,stroke:#34d399,stroke-width:2px,color:#fff
style CONSUMERS fill:#1e293b,stroke:#f472b6,stroke-width:2px,color:#fff
Monitors video feeds and camera streams at configurable temporal intervals (default:
Note
Reducing VISION_SAMPLE_INTERVAL below 1.0s increases CPU load significantly. The default 2.0s is optimized for standard venue CCTV hardware.
-
Pluggable Crowd Detectors (
detectors.py):- HOG Pedestrian Detector (Default): Ultra-fast, lightweight OpenCV implementation running efficiently on standard CPUs with zero external weights.
-
YOLOv8 Detector (
ultralytics): High-precision person bounding-box detection with centroid tracking. - CSRNet Density Regressor: Fallback deep neural network for dense crowd density-map regression.
-
Optical Flow & Motion Tracker (
flow_tracker.py):- Uses dual-pass Lucas-Kanade Sparse Feature Tracking and Farneback Dense Optical Flow.
- Computes average crowd velocity (
$\text{m/s}$ ) calibrated via pixels-per-meter scale ratio. - Classifies aggregate directional heading:
towards_exit,away_from_exit,stationary, ormixed.
-
Privacy-by-Design Frame Pipeline (
density_estimator.py):- Video frames are processed strictly in RAM and immediately dropped.
- No faces, identities, license plates, or raw recordings are ever written to disk or transmitted over the network.
Ingests rolling-window time series (
Unlike uninterpretable deep neural networks, CrowdShieldβs scoring is mathematical, deterministic, and fully auditable:
Where:
-
$S_d$ (Normalized Density Score): Ratio of current density to critical physical threshold ($D_{\text{critical}} = 5.0\text{ p/m}^2$ ).$$S_d = \min\left(1.0, \frac{\text{density}}{D_{\text{critical}}}\right)$$ -
$S_t$ (Density Gradient / Trend Score): Rolling linear rate of change over time ($\Delta D / \Delta t$ ). A rapid crowd influx triggers early warnings even at moderate density.$$S_t = \max\left(0.0, \min\left(1.0, \frac{\Delta D / \Delta t}{\text{max_trend_rate}}\right)\right)$$ -
$S_f$ (Flow Velocity Impairment): Measures crowd deceleration and impending crush stagnation relative to free-flow walking speed ($V_{\text{free}} = 1.2\text{ m/s}$ ).$$S_f = \max\left(0.0, 1.0 - \frac{\text{flow_speed}}{V_{\text{free}}}\right)$$ -
$P_b$ (Non-linear Bottleneck Penalty): Interaction term that spikes when high density is compounded by zero flow velocity ($S_d \times S_f$ ). -
Default Weights:
$w_d = 0.45$ ,$w_t = 0.25$ ,$w_f = 0.20$ ,$w_b = 0.10$ .
0.0 ββββββββββ 0.35 ββββββββββ 0.60 ββββββββββ 0.80 ββββββββββ 1.0
β LOW β MEDIUM β HIGH β CRITICAL β
β (Routine) β (Pre-Warning) β (Action Req.) β (Emergency) β
-
ETA to Critical (
$T_{\text{critical}}$ ): When density is rising ($\Delta D / \Delta t > 0$ ), computes:$$\text{ETA (minutes)} = \left\lceil \frac{D_{\text{critical}} - D_{\text{current}}}{\Delta D / \Delta t \times 60} \right\rceil$$ Enables operators to intervene 6β15 minutes before physical crowd crush.
When risk rises, CrowdShield automatically derives topological standard operating procedures and calm public address announcements.
flowchart TD
A[Risk Event Triggered] --> B{LLM Enabled?}
B -- Yes --> C[Layer 1: Google Gemini 1.5 Flash]
C -- Success --> G[Return Contextual Bilingual Announcement]
C -- Timeout / Rate Limit / No Key --> D[Layer 2: Groq Llama 3.1 8B Instant]
D -- Success --> G
D -- Timeout / Rate Limit / No Key --> E[Layer 3: Cohere Command R]
E -- Success --> G
E -- Timeout / Rate Limit / No Key --> F[Layer 4: Deterministic Local Rule Templates]
B -- No --> F
F --> H[Return Guaranteed Instant Announcement]
-
Topological Venue Rules: Analyzes connected zone geometries (e.g., South Gate
$\leftrightarrow$ West Corridor) to balance loads, recommend gate actuations, and deploy ground marshals. -
Resilient 4-Layer Multi-Provider Cascade:
-
Google Gemini 1.5 Flash (Primary): Free-tier AI model generating natural, calm bilingual messages (
$15\text{ RPM} / 1\text{,500 RPD}$ ). -
Groq Cloud Llama 3.1 8B (Secondary): Ultra-low inference latency (~$200\text{ms}$,
$30\text{ RPM} / 14\text{,400 RPD}$ ). - Cohere Command R (Tertiary): Free developer tier for robust multi-lingual contextual synthesis.
- Deterministic Local Templates (Quaternary): Pure Python zero-network fallback that guarantees 100% uptime with zero single points of failure.
-
Google Gemini 1.5 Flash (Primary): Free-tier AI model generating natural, calm bilingual messages (
Powered by FastAPI and async Uvicorn, serving as the central nervous system:
-
WebSocket Feed (
/ws/risk-events): Broadcasts a unified array of all 4 zone events every$3\text{ seconds}$ to all connected dashboard and mobile clients simultaneously. - Immediate State Push: Pushes the current snapshot immediately upon WebSocket handshake for instantaneous UI rendering with zero blank states.
-
Graceful Degradation: If
MOCK_MODE=falseis requested but video streams are disconnected, automatically switches to realistic synthetic telemetry rather than crashing. -
Demo Scenario Replay (
/demo/scenario): Allows instant replay of scripted 5-minute before/after crush incidents at accelerated speeds for live presentations.
Caution
Setting BROADCAST_INTERVAL below 1 second in a high-client environment may cause WebSocket congestion. The default 3s is the recommended minimum for stable performance.
A state-of-the-art emergency operations center built with Next.js 16 (App Router), React 19, Tailwind CSS, and Radix UI:
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β π‘οΈ CROWDSHIELD EMERGENCY COMMAND CENTER π’ LIVE STREAM [3s TICK] β
βββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β π NAVIGATION β π’ BILINGUAL MARQUEE: "Attention: Heavy ingress at North Entrance..." β
β ββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββ¬ββββββββββββββββββββ€
β [1] Overview β π ACTIVE ZONE METRICS β πΊοΈ INTERACTIVE MAP β β οΈ LIVE ALERTS β
β [2] Live Map β β’ South: 2.1 p/mΒ² (LOW) β Leaflet Heatmap Overlay β 18:42:01 Gate 3 β
β [3] Incidents β β’ West: 1.8 p/mΒ² (LOW) β Color-coded Zone Nodes β Risk: HIGH (78%) β
β [4] Zones β β’ North: 4.8 p/mΒ² (HIGH)β Real-time Density Layer β ETA: 6 minutes β
β [5] Analytics β β’ East: 1.2 p/mΒ² (LOW) β β Action: Open G5 β
β [6] Digital Twinββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββ΄ββββββββββββββββββββ€
β [7] AI Summary β π ROLLING TIME-SERIES ANALYTICS (Recharts) β
β [8] Settings β Density Gradient (p/mΒ²) & Risk Score Trends over rolling 10-min window β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β ποΈ VOICE CMD β π’ 2D DIGITAL TWIN: SVG Venue Plan with Real-time Gate Actuation & Relief β
βββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
- Overview: High-level executive KPI cards, active incident counters, highest-risk zone spotlight, and quick-dispatch actions.
- Live Map (Leaflet +
leaflet.heat): Real-time venue map with dynamic heatmap overlay calibrated todensity_per_sqmand risk-colored zone markers. - Incident Dispatch: Live incident feed with triage severity tags, marshal dispatch tracking, and field log history.
- Zone Deep-Dive: Individual telemetry cards for all 4 venue zones displaying density meters, flow velocities, bottleneck warnings, and SOP checklists.
- Analytics & Trends (Recharts): Client-side rolling historical time-series analyzing density trends, flow stagnation, and risk evolution.
- 2D Digital Twin Venue: Interactive SVG venue floor plan rendering architectural boundaries, crowd flow vectors, and animated gate opening/closing states.
- AI Executive Summary: One-click on-demand AI executive briefing summarizing current danger points and recommended SOPs in English and Hindi.
- Settings & Simulation Controls: Control panel for adjusting sensitivity thresholds, configuring weights, toggling mock generators, and testing demo scenarios.
Cross-platform mobile application built on React Native and Expo SDK 51:
- Live Zone Safety Roster: Instant overview of all venue sectors with color-coded safety indicators.
- Simulated Proximity & Geofenced Alerts: Select your current gate/concourse and receive local push notifications via Expo Notifications the instant your area crosses into high risk.
-
Instant English
$\leftrightarrow$ Hindi Translation: Language toggle updating public safety guidance and evacuation notices. - Citizen Incident Reporting: Rapid reporting interface enabling attendees and field marshals to log overcrowded corridors, blocked exits, or medical emergencies with photos.
- AI Voice Safety Assistant: Natural language voice-and-text query interface allowing operators and attendees to ask "Which zones are congested right now?" and receive concise, calm guidance.
All components in CrowdShield (Vision, Risk, Pipeline, Web Dashboard, Mobile App) communicate via this standardized, strictly typed JSON contract broadcast every 3 seconds:
{
"zone_id": "gate_3",
"zone_name": "North Entrance",
"timestamp": "2026-08-22T18:42:00Z",
"density_per_sqm": 4.8,
"flow_speed_mps": 0.35,
"risk_score": 0.78,
"risk_level": "high",
"eta_minutes": 6,
"recommendations": [
"open_gate_2",
"redirect_flow_west",
"deploy_staff_gate_3"
],
"announcement": {
"en": "Attention visitors near North Entrance: Movement is heavy. Please divert calmly towards Gate 2.",
"hi": "ΰ€¨ΰ₯ΰ€°ΰ₯ΰ€₯ ΰ€ΰ€ΰ€ΰ₯ΰ€°ΰ₯ΰ€ΰ€Έ ΰ€ΰ₯ ΰ€ͺΰ€Ύΰ€Έ ΰ€¦ΰ€°ΰ₯ΰ€Άΰ€ ΰ€ΰ₯ΰ€ͺΰ€―ΰ€Ύ ΰ€§ΰ₯ΰ€―ΰ€Ύΰ€¨ ΰ€¦ΰ₯ΰ€: ΰ€ΰ₯ΰ€‘ΰ€Ό ΰ€
ΰ€§ΰ€Ώΰ€ ΰ€Ήΰ₯ΰ₯€ ΰ€ΰ₯ΰ€ͺΰ€―ΰ€Ύ ΰ€Άΰ€Ύΰ€ΰ€€ΰ€Ώΰ€ͺΰ₯ΰ€°ΰ₯ΰ€΅ΰ€ ΰ€ΰ₯ΰ€ 2 ΰ€ΰ₯ ΰ€ΰ€° ΰ€¬ΰ€’ΰ€Όΰ₯ΰ€ΰ₯€"
}
}| Field | Type | Description |
|---|---|---|
zone_id |
string |
Unique machine-readable identifier (gate_1, gate_2, gate_3, gate_4). |
zone_name |
string |
Human-readable zone label displayed in UI. |
timestamp |
string (ISO 8601) |
Precise UTC timestamp of sensor snapshot generation. |
density_per_sqm |
float |
Estimated crowd density in people per square metre ( |
flow_speed_mps |
float |
Estimated average crowd movement velocity in metres per second ( |
risk_score |
float [0.0 - 1.0] |
Calculated composite risk score ( |
risk_level |
enum |
Categorical classification: "low", "medium", "high", or "critical". |
eta_minutes |
integer | null |
Estimated minutes remaining until density crosses critical threshold ( |
recommendations |
string[] |
Ordered list of machine-readable operator action tokens. |
announcement |
object |
Bilingual public address announcement strings (en and hi). |
CrowdShield includes a built-in simulation engine demonstrating the life-saving impact of predictive crowd intervention over a 5-minute scenario (accelerated to
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β SCENARIO A: WITHOUT CROWDSHIELD (Unmanaged Crowd Surge) β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Min 0: Density = 2.0 p/mΒ² | Flow = 1.2 m/s | Risk: LOW (25%) β Normal ingress β
β Min 1: Density = 3.2 p/mΒ² | Flow = 0.8 m/s | Risk: MEDIUM (48%) β Influx accelerates β
β Min 2: Density = 4.8 p/mΒ² | Flow = 0.4 m/s | Risk: HIGH (78%) β Bottleneck forms β
β Min 3: Density = 6.5 p/mΒ² | Flow = 0.1 m/s | Risk: CRITICAL β Physical gridlock β
β Min 4: Density = 8.0 p/mΒ² | Flow = 0.0 m/s | CRUSH EVENT β FATAL STAMPEDE β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β SCENARIO B: WITH CROWDSHIELD (Proactive Early Intervention) β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Min 0: Density = 2.0 p/mΒ² | Flow = 1.2 m/s | Risk: LOW (25%) β Normal ingress β
β Min 1: Density = 3.2 p/mΒ² | Flow = 0.8 m/s | Risk: MEDIUM (48%) β Early warning alert β
β β‘ CrowdShield Action: Open Gate 2 relief gate + Bilingual Diversion Broadcast β
β Min 2: Density = 3.5 p/mΒ² | Flow = 0.7 m/s | Risk: MEDIUM (45%) β Surge stabilizes β
β Min 3: Density = 2.8 p/mΒ² | Flow = 1.0 m/s | Risk: LOW (32%) β Crowd diverted β
β Min 4: Density = 2.1 p/mΒ² | Flow = 1.2 m/s | Risk: LOW (22%) β AREA FULLY SECURED β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
- Git: 2.x or higher
- Python: 3.11 or higher
- Node.js: 18.x or 20.x+ (LTS) &
npm/pnpm - Expo CLI: (for mobile testing)
npm install -g expo-cli
CrowdShield includes an automated startup script that creates virtual environments, installs all Python & Node.js dependencies, and starts both the FastAPI pipeline server and the Next.js Dashboard:
# 1. Clone the repository
git clone https://github.com/Swapnil-Ghosh06/Crowdshield.git
cd Crowdshield/crowdshield
# 2. Run one-click bootstrapper (installs all deps & starts all services)
bash start.sh- Command Center Dashboard: http://localhost:3000
- FastAPI Pipeline Server: http://localhost:8000
- Interactive Swagger API Docs: http://localhost:8000/docs
- Live WebSocket Stream:
ws://localhost:8000/ws/risk-events
cd crowdshield/pipeline
# Create and activate Python virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install pipeline dependencies
pip install -r requirements.txt
# Start FastAPI server on port 8000 (--reload enables hot-reloading in dev)
uvicorn main:app --host 0.0.0.0 --port 8000 --reloadcd crowdshield/dashboard
# Install npm dependencies (or use pnpm: pnpm install)
npm install
# Start Next.js development server
npm run devcd crowdshield/mobile-app
# Install dependencies
npm install
# Launch Expo development server
npx expo startNote
Scan the QR code with the Expo Go app on iOS or Android, or press w to open the web simulator.
Copy .env.example to .env in the project root or pipeline/ directory:
cp .env.example .env# ---------------------------------------------------------------------------
# PIPELINE CONFIGURATION
# ---------------------------------------------------------------------------
MOCK_MODE=true # Set to "false" to use live OpenCV/YOLO video feeds
VIDEO_PATH=sample.mp4 # Path to video file or RTSP stream URI
BROADCAST_INTERVAL=3 # WebSocket broadcast cadence in seconds
# ---------------------------------------------------------------------------
# VISION ENGINE CONFIGURATION
# ---------------------------------------------------------------------------
ZONE_AREA_SQM=50.0 # Physical square metres of monitored zone
VISION_SAMPLE_INTERVAL=2.0 # Frame sampling rate in seconds
DETECTOR_BACKEND=hog # Options: hog, yolo, csrnet
# ---------------------------------------------------------------------------
# 4-LAYER MULTI-PROVIDER LLM CASCADE (OPTIONAL)
# ---------------------------------------------------------------------------
LLM_CASCADE_ENABLED=true # Toggles AI announcement generation
GEMINI_API_KEY=your_gemini_key # Free Tier Layer 1 (Google AI Studio)
GROQ_API_KEY=your_groq_key # Free Tier Layer 2 (Groq Cloud)
COHERE_API_KEY=your_cohere_key # Free Tier Layer 3 (Cohere Trial)Tip
Zero-Key Resilience: API keys are completely optional. If keys are omitted, CrowdShield will automatically and silently utilize its Deterministic Layer 4 Rule Engine, producing perfect bilingual announcements locally with zero network calls and zero cost.
-
Endpoint:
ws://localhost:8000/ws/risk-events -
Behavior:
- On connection, immediately pushes the current multi-zone snapshot array (
list[RiskEvent]). - Broadcasts updated telemetry for all 4 zones every
$3\text{ seconds}$ . - Supports automatic reconnection and client pruning.
- On connection, immediately pushes the current multi-zone snapshot array (
| Protocol | Route | Method | Description |
|---|---|---|---|
GET |
/events/latest |
REST | Returns latest in-memory risk event dictionary keyed by zone_id. |
GET |
/health |
REST | Diagnostic probe returning server mode (mock/live), active WebSocket connections, and per-engine status. |
GET |
/ai/summary |
REST | Generates an AI-synthesized bilingual executive incident briefing for the highest-risk zone. |
GET/POST |
/demo/scenario?mode=without_intervention |
REST | Returns a 10-step simulated time-series showing an unmitigated crowd surge. |
GET/POST |
/demo/scenario?mode=with_intervention |
REST | Returns a 10-step simulated time-series demonstrating proactive CrowdShield gate relief. |
GET/POST |
/demo/scenario?type=before |
REST | Triggers live accelerated 5-stage unmanaged crush replay over the WebSocket feed. |
GET/POST |
/demo/scenario?type=after |
REST | Triggers live accelerated 5-stage proactive intervention replay over the WebSocket feed. |
GET |
/docs |
HTTP | Interactive Swagger UI API exploration interface. |
CrowdShield maintains a comprehensive, automated test suite covering all modules:
# Run the full test suite with pytest (recommended β uses pytest.ini config)
cd crowdshield && pytest -v
# --- OR run individual module suites manually: ---
# Run all Risk Engine and 4-Layer LLM Cascade tests
python3 -m unittest discover -s crowdshield/risk-engine/tests -p "test_*.py" -v
# Run Vision Engine and Optical Flow tests
python3 -m unittest discover -s crowdshield/vision-engine/tests -p "test_*.py" -v
# Run FastAPI Pipeline and WebSocket broadcast tests
python3 -m unittest discover -s crowdshield/pipeline/tests -p "test_*.py" -v- β
Risk Formulation Tests: Verifies linear gradient sensitivity, stagnation penalties, and boundary conditions (
$0.0 \le \text{score} \le 1.0$ ). - β
LLM Cascade Tests: Validates seamless failover from Gemini
$\rightarrow$ Groq$\rightarrow$ Cohere$\rightarrow$ Fallback Rule Templates on mock network failures. - β Vision Tests: Validates Lucas-Kanade velocity tracking, bounding box density estimation, and transient RAM memory release.
- β WebSocket Tests: Confirms 3-second broadcast cadence, immediate connection push, client disconnection cleanup, and REST snapshot persistence.
CrowdShield was built from the ground up to respect human privacy and adhere strictly to international data ethics principles:
- Zero Raw Video Persistence: Video frames are analyzed transiently in volatile memory and immediately dropped. No video files or CCTV recordings are ever written to disk or sent to the cloud.
- Zero Facial Recognition or Biometrics: Person detection is strictly aggregate counting. No face detection, facial recognition, gender/age estimation, or identity profiling is performed.
-
Directional Vector Fields over Individual Tracking: Crowd movement is analyzed as aggregate optical velocity vectors (
$\text{m/s}$ ), ensuring individuals cannot be singled out or tracked across cameras. - Pure In-Memory Real-Time Stream: Visitor telemetry is held transiently in RAM queues for rolling rate-of-change math and never saved to a persistent database.
- Calm, Transparent Public Communication: Multilingual announcements are crafted using calming, clear behavioral psychology prompts to prevent mass panic while guiding safe egress.
Important
CrowdShield never stores video, never identifies individuals, and never transmits raw footage over any network. All crowd data is reduced to anonymous aggregate metrics before leaving the edge device.
| Criterion | Weight | How CrowdShield Excels |
|---|---|---|
| Innovation | 40% | Moves beyond passive CCTV heatmaps to predictive 6β15 min early warning, combined with a 4-Layer Resilient LLM Cascade, Interactive 2D Digital Twin, and Voice-Enabled Command Center. |
| Technical Feasibility | 30% | Built with an explainable, deterministic mathematical formulation (no black box), modular OpenCV/YOLO vision engines, 100% free-tier sustainability, and |
| User Friendliness | 20% | Dual-stakeholder experience: Next.js 16 Command Operations Dashboard for venue directors and Expo React Native Mobile App with instant English |
| Data Ethics & Privacy | 10% | Privacy-by-Design Architecture: Zero video storage, zero biometric profiling, transient in-memory frame processing, and ethical crowd flow vectorization. |
Crowdshield/
βββ 00_PROJECT_PLAN.md # Master project requirements & milestone plan
βββ 01_SWAPNIL_PLAN.md # Vision, Risk & Pipeline architecture plan
βββ 02_ZAHID_PLAN.md # Command Dashboard & Backend plan
βββ 03_HARIPRIYA_PLAN.md # Mobile App, Multilingual & Docs plan
βββ FREE_TIER_MIGRATION_CHANGES.md # 4-Layer Multi-Provider LLM Cascade technical report
βββ crowdshield/
β βββ start.sh # One-command bootstrapper for all services
β βββ sample.mp4 # Sample crowd footage for vision engine testing
β βββ pipeline/ # Realtime FastAPI & WebSocket Service
β β βββ main.py # FastAPI server, WS broadcaster, REST endpoints
β β βββ models.py # Pydantic schemas for shared wire contract
β β βββ mock_generator.py # Realistic multi-zone synthetic event generator
β β βββ real_engine.py # Ingests live vision-engine & risk-engine outputs
β β βββ demo_scenarios.py # Before/After scripted crush replay generator
β β βββ requirements.txt # Python dependencies for pipeline
β β βββ tests/ # Automated pipeline & WebSocket test suite
β βββ vision-engine/ # Computer Vision & Motion Tracking Module
β β βββ density_estimator.py # Frame processor & density per mΒ² calculator
β β βββ detectors.py # HOG, YOLOv8, and CSRNet detection backends
β β βββ flow_tracker.py # Lucas-Kanade & Farneback optical flow velocity
β β βββ models.py # Vision data schemas & directional enums
β β βββ tests/ # Automated vision engine test suite
β βββ risk-engine/ # Predictive Risk & Recommendation System
β β βββ risk_engine.py # Mathematical scoring formula & ETA derivation
β β βββ recommendations.py # Topological venue rules & 4-Layer LLM Cascade
β β βββ rules.py # Deterministic fallback bilingual templates
β β βββ explainability.py # Explainable score decomposition & attribution
β β βββ config.py # Physical threshold & scoring weight settings
β β βββ tests/ # Automated risk engine & cascade test suite
β βββ dashboard/ # Next.js 16 Emergency Operations Command Center
β β βββ app/ # Next.js App Router root & entry page
β β βββ components/dashboard/ # Command Center UI Modules
β β β βββ sections/ # 8 Dedicated Views (Overview, Map, Digital Twin, etc.)
β β β βββ map/ # Leaflet map & Heatmap layers
β β β βββ charts/ # Recharts time-series analytics
β β β βββ multilingual-ticker.tsx # Live scrolling bilingual PA marquee
β β β βββ voice-command-button.tsx# AI Voice Command Assistant interface
β β β βββ sidebar.tsx # Collapsible primary navigation
β β βββ package.json # Dashboard dependencies & Next.js scripts
β βββ mobile-app/ # React Native (Expo SDK 51) Citizen Mobile App
β β βββ screens/ # Home, MyZone, Incident Report, Assistant screens
β β βββ components/ # Reusable mobile UI components & Zone Cards
β β βββ hooks/ # WebSocket & state subscription hooks
β β βββ package.json # React Native Expo dependencies
β βββ docs/ # Documentation, PRD, Schemas & Technical Specifications
β βββ prd.md # Product Requirements Document
β βββ techstack.md # Technology choices & rationale
β βββ schema.md # Complete JSON data dictionary
β βββ requirements.md # Functional & Non-functional requirements
| Team Member | Core Focus & Ownership | Primary Tech Stack | GitHub |
|---|---|---|---|
| Swapnil Ghosh | Vision Engine, Predictive Risk Engine, 4-Layer LLM Cascade, Realtime FastAPI Pipeline | Python, OpenCV, FastAPI, Pydantic, Gemini, Groq | @Swapnil-Ghosh06 |
| Zahid | Next.js Command Operations Dashboard, Leaflet Map/Heatmap, 2D Digital Twin, Recharts | Next.js 16, React 19, TypeScript, Tailwind CSS | β |
| Haripriya | Citizen Mobile App, Expo Notifications, Bilingual Delivery, Incident Reporting, Docs | React Native, Expo SDK 51, JavaScript | β |
- RTSP Live Camera Integration β Connect to real IP cameras via RTSP streams in production mode
- Multi-Venue Support β Scale from a single 4-zone venue to multi-stadium, multi-site deployments
- Native iOS & Android Builds β Publish standalone apps to App Store and Google Play
- Historical Incident Analytics β Persistent database (TimescaleDB) for post-event safety analysis
- Voice PA Auto-Dispatch β Automatic text-to-speech output via PA system hardware integrations
- Drone Camera Ingestion β Support for aerial RTSP feeds from UAV surveillance drones
Contributions, issues, and feature requests are welcome! Feel free to check the issues page.
- Fork the repository
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'feat: add AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
This project is licensed under the MIT License β see the LICENSE file for details.
Developed with β€οΈ for TechNova Hackathon 2026 π β to make public gatherings, transit hubs, and cultural events safer for everyone worldwide.