Pi Server — Computer Vision Detection

A Python-based computer vision server that receives images from a Raspberry Pi, detects people using a YOLO model, and sends the resulting decision back to the Raspberry Pi over MQTT.

The project was developed as part of a distributed computer vision system, where the Raspberry Pi handles image capture and hardware control while a more powerful computer performs the computationally expensive machine-learning inference.

System Architecture

                    Raspberry Pi
                         │
                         │  JPEG image
                         │
                         ▼
                   MQTT Broker
                         │
                         ▼
                  ┌───────────────┐
                  │   Pi-Server   │
                  │               │
                  │ Image Decode  │
                  │      ↓        │
                  │ YOLO Detection│
                  │      ↓        │
                  │   Decision    │
                  └───────┬───────┘
                          │
                          │ detection result
                          ▼
                    MQTT Broker
                          │
                          ▼
                    Raspberry Pi
                          │
                          ▼
                   Hardware Control

This architecture allows the Raspberry Pi to act as an edge device while offloading machine-learning inference to a more capable computer.

Computer Vision

The project uses a YOLOv8-based model for person detection. The original development process began by testing the model using a live webcam stream before integrating it into the Raspberry Pi system.

The processing pipeline is:

Input Image
     │
     ▼
Image Processing
     │
     ▼
YOLO Object Detection
     │
     ▼
Person Detected?
     │
 ┌───┴───┐
 │       │
Yes      No
 │       │
 ▼       ▼
True    False
 │       │
 └───┬───┘
     ▼
MQTT Response

MQTT Communication

MQTT is used to communicate between the Raspberry Pi and the computer running the detection model.

The Raspberry Pi publishes captured images to an MQTT broker. The server receives these images, processes them, and publishes the detection result back to the Pi.

This approach avoids running the full computer vision model directly on the Raspberry Pi, which was found to be too slow for the intended real-time application.

Why a Separate Server?

The initial implementation attempted to run the model directly on the Raspberry Pi.

However, inference speed was a significant limitation. The project therefore moved toward a distributed architecture:

Raspberry Pi

• Captures images
• Communicates over the network
• Controls hardware

Computer / Server

• Receives images
• Runs the YOLO model
• Performs image processing
• Sends detection decisions back

This provides significantly more computational resources for machine-learning inference without requiring the Raspberry Pi to run the model locally.

Project Structure

Pi-Server/
│
├── image_processing.py   # Main image-processing and detection pipeline
├── img_process_tools.py  # Supporting image-processing functions
├── mqtt_subscriber.py    # MQTT communication
├── timings.txt           # Recorded processing timings
├── README.md
└── LICENSE

Technologies

• Python
• YOLOv8
• Computer Vision
• OpenCV
• MQTT
• Paho MQTT
• Raspberry Pi
• Image Processing

Development

The project evolved through several stages:

1. Initial model testing - testing YOLO-based person detection using a computer webcam.
2. Raspberry Pi integration - experimenting with running inference directly on the Pi.
3. Hardware integration - connecting the computer vision output to the physical system.
4. Performance testing - measuring inference and processing times.
5. Distributed processing - moving image processing to a separate computer and communicating with the Pi over MQTT.

The final architecture separates sensing, computation and actuation, allowing each component to perform the task it is best suited for.

Performance

Processing timings are recorded in timings.txt to allow the performance of the image-processing pipeline to be analysed and compared.

Moving inference from the Raspberry Pi to a more powerful computer was motivated primarily by the processing-time limitations encountered when running the model directly on the Pi.

Future Improvements

Potential improvements include:

• Benchmarking different YOLO model sizes
• Optimising image compression and transmission
• Reducing MQTT communication latency
• Running inference asynchronously
• Adding confidence thresholds
• Adding structured detection messages
• Implementing automatic MQTT reconnection
• Exporting the model to an optimised inference format
• Running inference on dedicated edge hardware

Related Project

This repository forms the computer vision server component of the Raspberry Pi system.

The Raspberry Pi-side hardware and MQTT communication is contained in the pi-nerf-gun repository.

Together, the two repositories demonstrate a distributed computer vision pipeline combining:

• Raspberry Pi hardware
• Camera input
• Machine-learning inference
• MQTT networking
• Real-time decision making
• Physical actuation