A real-time driver monitoring system that detects fatigue and cognitive distraction using facial landmark analysis, temporal feature engineering, and ensemble machine learning.
Driver fatigue and distraction are among the leading causes of road accidents worldwide. Existing monitoring systems often rely on isolated indicators such as eye closure or steering patterns, limiting their ability to accurately capture a driver's cognitive state.
This project presents a vision-based driver monitoring framework that combines computer vision, behavioral feature extraction, machine learning, and deep learning techniques to classify driver states in real time.
The system analyzes facial behavior including eye movements, blinking patterns, yawning activity, and head pose dynamics to identify signs of fatigue and reduced attentiveness.
Develop a non-intrusive driver monitoring system capable of detecting fatigue and cognitive distraction from video streams without requiring physiological sensors such as EEG or ECG.
The system should:
- Operate in real time
- Extract meaningful behavioral features from facial landmarks
- Learn temporal fatigue patterns
- Provide reliable driver state classification
Facial landmarks are extracted using MediaPipe to track key facial regions including:
- Eyes
- Mouth
- Nose
- Jawline
The following fatigue-related indicators are computed:
| Feature | Description |
|---|---|
| EAR | Eye Aspect Ratio |
| Blink Rate | Number of blinks over time |
| Eye Closure Duration | Consecutive closed-eye frames |
| MAR | Mouth Aspect Ratio |
| Yawning Frequency | Yawning behavior over time |
| Head Pose | Driver head orientation |
| Eye Asymmetry | Fatigue-related eye behavior |
Frame-level observations are aggregated into temporal windows to capture fatigue progression and behavioral trends.
The engineered features are used to train multiple classifiers:
- Random Forest
- XGBoost
- CNN-LSTM
Predictions from individual models are combined using an ensemble framework to improve robustness and reduce false detections.
Video Stream
│
▼
Face Detection
│
▼
MediaPipe Landmark Extraction
│
▼
Feature Engineering
(EAR, MAR, Blink Dynamics,
Head Pose, Eye Asymmetry)
│
▼
Temporal Feature Generation
│
▼
Machine Learning Models
(Random Forest, XGBoost, CNN-LSTM)
│
▼
Ensemble Decision Layer
│
▼
Driver State Classification
| Class | Description |
|---|---|
| Alert | Driver is attentive and focused |
| Fatigued | Fatigue-related behavioral patterns detected |
| Distracted | Signs of cognitive distraction detected |
The project utilizes publicly available driver monitoring datasets including:
- NTHU Driver Drowsiness Detection Dataset
- YawDD Dataset
Due to dataset licensing and repository size constraints, raw datasets are not included in this repository.
- Python
- OpenCV
- MediaPipe
- Scikit-Learn
- XGBoost
- Random Forest
- TensorFlow
- Keras
- NumPy
- Pandas
- Matplotlib
- Seaborn
.
├── train_data/
├── buildframefeature.py
├── extract_feature.py
├── extract_temporal_features.py
├── extract_window_features.py
├── train_model.py
├── train_short_only.py
├── demo.py
├── requirements.txt
└── README.md
git clone https://github.com/AnweshSinha/driver-cognitive-state-analysis.git
cd driver-cognitive-state-analysis
pip install -r requirements.txtGenerate frame-level features:
python buildframefeature.pyExtract temporal features:
python extract_temporal_features.pyTrain the classifier:
python train_model.pyRun real-time inference:
python demo.py- Vision Transformer based architectures
- Attention-enhanced temporal modeling
- Edge deployment optimization
- Low-light robustness improvements
- Driver alert generation system
Anwesh Sinha Ayush Thakur Ayush Rawat
Jaypee University of Information Technology
This project is intended for academic and research purposes.