RF-DETR C++ Inference

A Windows-based real-time object detection application using RF-DETR (Roboflow Detection Transformer). Supports both ONNX Runtime and TensorRT backends with CUDA acceleration and OpenCV for image/video I/O.

---

Features

• ONNX Runtime inference with CUDA execution provider
• TensorRT inference from pre-built .engine/.trt files
• Backend auto-selected from model file extension
• Image and video input with optional annotated output saving
• Configurable confidence threshold and input resolution
• No NMS required — RF-DETR is end-to-end

---

Dependencies

Dependency	Version	Notes
Windows	10/11	x64
Visual Studio	2022	Desktop C++ and CUDA workloads required
CUDA Toolkit	11.8+	Download
OpenCV	4.5+	Download
ONNX Runtime	1.16+	GPU (CUDA) build — Download
TensorRT	8.6+	Windows ZIP package — Download
CMake	3.21+	Download

Note: ONNX Runtime and TensorRT are both required at build time even if you only use one backend at runtime.

---

Installation

1. Clone the repository

git clone https://github.com/yeokiwi/Rf_detr_cpp.git
cd Rf_detr_cpp

2. Install dependencies

Extract/install each dependency to a known path, for example:

C:\onnxruntime\        ← ONNX Runtime root (contains include\ and lib\)
C:\TensorRT\           ← TensorRT root (contains include\ and lib\)
C:\opencv\build\       ← OpenCV cmake config directory

Make sure the CUDA Toolkit is installed and nvcc is on your PATH.

3. Configure with CMake

Open x64 Native Tools Command Prompt for VS 2022 and run:

cmake -G "Visual Studio 17 2022" -A x64 ^
      -DONNXRUNTIME_DIR=C:\onnxruntime ^
      -DTENSORRT_DIR=C:\TensorRT ^
      -DOpenCV_DIR=C:\opencv\build ^
      -B build .

4. Build

cmake --build build --config Release

The executable is output to build\Release\rf_detr_inference.exe. onnxruntime.dll is automatically copied next to the executable by a post-build step.

---

Exporting RF-DETR to ONNX

Use the official RF-DETR Python package to export your model:

pip install rfdetr

from rfdetr import RFDETRBase

# Load pretrained weights (downloads automatically on first run)
model = RFDETRBase(pretrain_weights="rf-detr-base.pth")

# Export to ONNX
model.export(format="onnx", destination="model.onnx")

The input size must be divisible by 14 (e.g. 560, 630, 644, 840). The default is 560.

Exporting to TensorRT (optional)

Convert the ONNX model to a TensorRT engine on the same machine you will run inference on:

trtexec.exe --onnx=model.onnx --saveEngine=model.engine --fp16

trtexec.exe is included in the TensorRT package under bin\.

---

Class Names File

Prepare a plain text file with one class name per line. The line index (0-based) is the class ID.

person
bicycle
car
motorcycle
...

A COCO 80-class example is provided at sample_classes\coco.txt.

---

Usage

rf_detr_inference.exe [OPTIONS]

Required:
  --model    <path>    Model file (.onnx  →  ONNX Runtime
                                  .engine or .trt  →  TensorRT)
  --classes  <path>    Text file with class names (one per line)
  --input    <path>    Input image (.jpg, .png, .bmp) or video (.mp4, .avi, ...)

Optional:
  --threshold <float>  Confidence threshold (default: 0.5)
  --input-size <int>   Model input resolution in pixels (default: 560, must be divisible by 14)
  --output   <path>    Save annotated image/video to file
  --no-display         Suppress the OpenCV display window
  --help               Print this message

Examples

Image inference with ONNX Runtime:

rf_detr_inference.exe ^
  --model model.onnx ^
  --classes sample_classes\coco.txt ^
  --input photo.jpg

Image inference with TensorRT, saving result:

rf_detr_inference.exe ^
  --model model.engine ^
  --classes sample_classes\coco.txt ^
  --input photo.jpg ^
  --output result.jpg

Video inference, lower threshold, no display:

rf_detr_inference.exe ^
  --model model.onnx ^
  --classes sample_classes\coco.txt ^
  --input clip.mp4 ^
  --output clip_out.mp4 ^
  --threshold 0.4 ^
  --no-display

Custom resolution (must be divisible by 14):

rf_detr_inference.exe ^
  --model model.onnx ^
  --classes my_classes.txt ^
  --input photo.jpg ^
  --input-size 630

During video playback, press Q or ESC to stop early.

---

Project Structure

Rf_detr_cpp/
├── CMakeLists.txt                  # Build configuration
├── include/
│   ├── rfdetr_detector.hpp         # Abstract base class, Detection and InferConfig structs
│   ├── onnx_backend.hpp            # ONNX Runtime backend declaration
│   ├── tensorrt_backend.hpp        # TensorRT backend declaration
│   └── utils.hpp                   # CLI parsing, drawing, class loading
├── src/
│   ├── main.cpp                    # Entry point — CLI, image/video loop
│   ├── rfdetr_detector.cpp         # Shared preprocessing and postprocessing
│   ├── onnx_backend.cpp            # ONNX Runtime session with CUDA EP
│   ├── tensorrt_backend.cpp        # TensorRT engine deserialization and CUDA buffers
│   └── utils.cpp                   # Helpers, OpenCV drawing, video detection
└── sample_classes/
    └── coco.txt                    # 80-class COCO names

---

Architecture

main.cpp
  └─ createDetector()          auto-selects backend from file extension
       ├─ ONNXBackend           Ort::Session + CUDA Execution Provider
       └─ TensorRTBackend       nvinfer1::IExecutionContext + cudaMalloc buffers
            │
            ▼
       RFDETRDetector (base)
         preprocess()           BGR → RGB → resize → normalize (ImageNet) → NCHW float32
         postprocess()          softmax → argmax → threshold → cxcywh → xyxy (pixels)

Preprocessing pipeline

1. BGR → RGB conversion
2. Resize to input_size × input_size (bilinear)
3. Scale to [0, 1] (÷ 255)
4. Normalize: (pixel − mean) / std with ImageNet stats
   • mean = [0.485, 0.456, 0.406]
   • std = [0.229, 0.224, 0.225]
5. Transpose HWC → NCHW, shape [1, 3, H, W]

Postprocessing pipeline

RF-DETR requires no NMS. Output processing:

1. Softmax over class logits for each query
2. Take argmax over foreground classes (excludes background)
3. Filter by confidence threshold
4. Convert normalized [cx, cy, w, h] → pixel [x1, y1, x2, y2]

---

Troubleshooting

Problem	Solution
onnxruntime.dll not found	Ensure onnxruntime.dll is in the same folder as the .exe (CMake copies it automatically on build) or add %ONNXRUNTIME_DIR%\lib to PATH
CUDA EP not available	Verify ONNX Runtime GPU build is used (filename contains gpu), and CUDA is installed
Could not find 'boxes' and 'labels' output nodes	Re-export the ONNX model; node names must be boxes and labels
TensorRT binding not found	Engine was built with different output names; rebuild with the ONNX model above
input_size must be divisible by 14	Use values like 560, 574, 588, 602, 616, 630, 644
Black/blank output video	Check the --output extension; use .mp4 for best codec compatibility

---

License

This project is provided as-is for research and deployment use. RF-DETR model weights are subject to Roboflow's license.