Skip to content

build: ship Rust mqproxy and retire the C application #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mp0rta merged 3 commits into from
Oct 7, 2026
Merged

build: ship Rust mqproxy and retire the C application #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ef0dd4a
build: ship Rust mqproxy and retire the C application
mp0rta Oct 7, 2026
6425ebb
ci: validate standalone Rust gateway benchmarks
mp0rta Oct 7, 2026
be3a1a8
fix: normalize package permissions and prepare benchmark certificates
mp0rta Oct 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
488 changes: 36 additions & 452 deletions .github/workflows/ci.yml
View file
Open in desktop

Large diffs are not rendered by default.

73 changes: 9 additions & 64 deletions .github/workflows/codeql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
name: CodeQL

on:
# Scoped to the default branch: CodeQL builds xquic (heavy) and uploads to the
# Scoped to the default branch: CodeQL runs build scripts and uploads to the
# repo code-scanning store, so run it on main pushes + PRs targeting main +
# weekly, not on every feature-branch push. (The lightweight fuzz job in
# ci.yml still runs on all branches.)
# rust.yml still runs on all branches.)
push:
branches: [ main ]
pull_request:
Expand All @@ -13,9 +13,6 @@ on:
- cron: '23 4 * * 1' # weekly, Monday 04:23 UTC
workflow_dispatch:

# Cache-key convention mirrors ci.yml: keys carry a trailing `-vN`. Bump in
# lockstep with ci.yml if the cached BoringSSL/xquic config changes.

jobs:
analyze:
runs-on: ubuntu-latest
Expand All @@ -28,68 +25,16 @@ jobs:
- uses: actions/checkout@v6
with:
submodules: recursive
- name: Install dependencies
run: sudo apt-get update && sudo apt-get install -y cmake libevent-dev libcurl4-openssl-dev libnghttp2-dev golang

# --- BoringSSL + xquic: mirror ci.yml `build` so mqproxy compiles for
# extraction. Same cache keys as ci.yml to ride the existing caches. ---
- name: Compute xquic SHA
id: xqsha
run: |
sha=$(git -c safe.directory="$GITHUB_WORKSPACE" rev-parse HEAD:third_party/xquic)
echo "sha=$sha" >> "$GITHUB_OUTPUT"

- name: Cache BoringSSL (gcc fPIC)
id: bssl-cache
uses: actions/cache@v5
with:
path: third_party/xquic/third_party/boringssl/build
key: bssl-0ce57bb-linux-v1
- name: Build BoringSSL (xquic submodule)
if: steps.bssl-cache.outputs.cache-hit != 'true'
run: |
cd third_party/xquic/third_party/boringssl
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_FLAGS="-fPIC" ..
make -j"$(nproc)" ssl crypto

- name: Cache xquic (gcc Release, EVENT_LOG)
id: xquic-cache
uses: actions/cache@v5
with:
path: |
third_party/xquic/build
third_party/xquic/include/xquic/xqc_configure.h
key: xquic-${{ steps.xqsha.outputs.sha }}-linux-gcc-release-eventlog-bssl-0ce57bb-v3
- name: Build xquic
if: steps.xquic-cache.outputs.cache-hit != 'true'
run: |
cd third_party/xquic
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DSSL_TYPE=boringssl \
-DXQC_ENABLE_EVENT_LOG=ON \
-DXQC_ENABLE_BBR2=ON -DXQC_ENABLE_UNLIMITED=ON \
-DXQC_ENABLE_FEC=ON -DXQC_ENABLE_XOR=ON \
-DCMAKE_C_FLAGS="-Wno-dangling-pointer" ..
make -j"$(nproc)"

# CodeQL traces commands run AFTER init — build xquic above (ignored via
# paths-ignore), init here, then build mqproxy so only our code is extracted.
- name: Install native build dependencies
run: sudo apt-get update && sudo apt-get install -y cmake golang
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v4
with:
languages: c-cpp
languages: rust
build-mode: none
queries: security-extended
config-file: ./.github/codeql/codeql-config.yml

- name: Build mqproxy
run: |
# EXACT flags from ci.yml `build` job so extraction compiles
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
-DXQUIC_BUILD_DIR="$PWD/third_party/xquic/build"
cmake --build build -j"$(nproc)"

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v4
with:
category: /language:c-cpp
category: /language:rust
61 changes: 11 additions & 50 deletions .github/workflows/perf-weekly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,70 +35,31 @@ jobs:
run: |
apt-get install -y \
cmake gcc g++ make \
libevent-dev libcurl4-openssl-dev libnghttp2-dev golang \
golang \
iproute2 nftables curl python3 openssl sudo \
iperf3 socat \
python3-pip pipx \
ca-certificates
pipx install awscli
echo "$HOME/.local/bin" >> "$GITHUB_PATH"

- name: Compute xquic SHA
id: xqsha
- name: Install Rust
run: |
sha=$(git -c safe.directory="$GITHUB_WORKSPACE" rev-parse HEAD:third_party/xquic)
echo "sha=$sha" >> "$GITHUB_OUTPUT"

- name: Cache BoringSSL (gcc fPIC)
id: bssl-cache
uses: actions/cache@v5
with:
path: third_party/xquic/third_party/boringssl/build
key: bssl-0ce57bb-linux-v1

- name: Build BoringSSL (xquic submodule)
if: steps.bssl-cache.outputs.cache-hit != 'true'
run: |
cd third_party/xquic/third_party/boringssl
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_FLAGS="-fPIC" ..
make -j"$(nproc)" ssl crypto

- name: Cache xquic (gcc Release, EVENT_LOG)
id: xquic-cache
uses: actions/cache@v5
with:
path: |
third_party/xquic/build
third_party/xquic/include/xquic/xqc_configure.h
key: xquic-${{ steps.xqsha.outputs.sha }}-linux-gcc-release-eventlog-bssl-0ce57bb-v3

- name: Build xquic
if: steps.xquic-cache.outputs.cache-hit != 'true'
run: |
cd third_party/xquic
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DSSL_TYPE=boringssl \
-DXQC_ENABLE_EVENT_LOG=ON \
-DXQC_ENABLE_BBR2=ON -DXQC_ENABLE_UNLIMITED=ON \
-DXQC_ENABLE_FEC=ON -DXQC_ENABLE_XOR=ON \
-DCMAKE_C_FLAGS="-Wno-dangling-pointer" ..
make -j"$(nproc)"

- name: Build mqproxy (Release)
run: |
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
-DXQUIC_BUILD_DIR="$PWD/third_party/xquic/build"
cmake --build build -j"$(nproc)"
curl --proto '=https' -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain none
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Build mqproxy (Rust Release)
run: cargo build --release --locked -p mqproxy
- name: Generate MITM test certificates
run: bash tests/certs/generate-mitm.sh

- name: TCP proxy benchmark
run: sudo bash scripts/ci_benchmarks/ci_bench_tcp_proxy.sh build/mqproxy
run: sudo bash scripts/ci_benchmarks/ci_bench_tcp_proxy.sh target/release/mqproxy

- name: MITM H2 benchmark
run: sudo bash scripts/ci_benchmarks/ci_bench_mitm.sh build/mqproxy
run: sudo bash scripts/ci_benchmarks/ci_bench_mitm.sh target/release/mqproxy

- name: Multi-stream sweep benchmark
run: sudo bash scripts/ci_benchmarks/ci_bench_multi_stream.sh build/mqproxy
run: sudo bash scripts/ci_benchmarks/ci_bench_multi_stream.sh target/release/mqproxy

- name: Upload results
if: always()
Expand Down
59 changes: 10 additions & 49 deletions .github/workflows/perf.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,67 +35,28 @@ jobs:
run: |
apt-get install -y \
cmake gcc g++ make \
libevent-dev libcurl4-openssl-dev libnghttp2-dev golang \
golang \
iproute2 nftables curl python3 openssl sudo \
iperf3 socat \
python3-pip pipx \
ca-certificates
pipx install awscli
echo "$HOME/.local/bin" >> "$GITHUB_PATH"

- name: Compute xquic SHA
id: xqsha
- name: Install Rust
run: |
sha=$(git -c safe.directory="$GITHUB_WORKSPACE" rev-parse HEAD:third_party/xquic)
echo "sha=$sha" >> "$GITHUB_OUTPUT"

- name: Cache BoringSSL (gcc fPIC)
id: bssl-cache
uses: actions/cache@v5
with:
path: third_party/xquic/third_party/boringssl/build
key: bssl-0ce57bb-linux-v1

- name: Build BoringSSL (xquic submodule)
if: steps.bssl-cache.outputs.cache-hit != 'true'
run: |
cd third_party/xquic/third_party/boringssl
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_FLAGS="-fPIC" ..
make -j"$(nproc)" ssl crypto

- name: Cache xquic (gcc Release, EVENT_LOG)
id: xquic-cache
uses: actions/cache@v5
with:
path: |
third_party/xquic/build
third_party/xquic/include/xquic/xqc_configure.h
key: xquic-${{ steps.xqsha.outputs.sha }}-linux-gcc-release-eventlog-bssl-0ce57bb-v3

- name: Build xquic
if: steps.xquic-cache.outputs.cache-hit != 'true'
run: |
cd third_party/xquic
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DSSL_TYPE=boringssl \
-DXQC_ENABLE_EVENT_LOG=ON \
-DXQC_ENABLE_BBR2=ON -DXQC_ENABLE_UNLIMITED=ON \
-DXQC_ENABLE_FEC=ON -DXQC_ENABLE_XOR=ON \
-DCMAKE_C_FLAGS="-Wno-dangling-pointer" ..
make -j"$(nproc)"

- name: Build mqproxy (Release)
run: |
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
-DXQUIC_BUILD_DIR="$PWD/third_party/xquic/build"
cmake --build build -j"$(nproc)"
curl --proto '=https' -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain none
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Build mqproxy (Rust Release)
run: cargo build --release --locked -p mqproxy
- name: Generate MITM test certificates
run: bash tests/certs/generate-mitm.sh

- name: TCP proxy benchmark
run: sudo bash scripts/ci_benchmarks/ci_bench_tcp_proxy.sh build/mqproxy
run: sudo bash scripts/ci_benchmarks/ci_bench_tcp_proxy.sh target/release/mqproxy

- name: MITM H2 benchmark
run: sudo bash scripts/ci_benchmarks/ci_bench_mitm.sh build/mqproxy
run: sudo bash scripts/ci_benchmarks/ci_bench_mitm.sh target/release/mqproxy

- name: Upload results
if: always()
Expand Down
Loading
Loading