Gaussian Merging [BMVC26]

Research code for reducing the number of 3D Gaussian primitives by merging superpixel-grouped reconstructions while preserving novel-view quality. The pipeline supports AnySplat, DepthSplat and Depth Anything 3 (DA3); ReSplat and VolSplat are comparison baselines.

This is an implementation of the Paper Compact Feed-Forward 3D Gaussians via Saliency Guided Primitive Merging [BMVC26].

Abstract: 3D scene reconstruction, modeling, and rendering are highly relevant for numerous tasks, and 3D Gaussian splatting has become a standard choice in this context. Its feed-forward variants provide fast reconstruction from sparse input views but often produce per-pixel primitives, leading to highly redundant and thus inefficient representations. We present a structure-aware merging pipeline that takes per-pixel primitives from any feed-forward method and consolidates them into a compact, content-adaptive Gaussian set while largely retaining visual quality at just 120th of the Gaussians of a per-pixel method. We group spatially coherent Gaussians of similar appearance into variable-size clusters via adaptive superpixel segmentation guided by a saliency map, which allocates fine segments to textured regions and coarse segments to homogeneous areas. We compress each cluster into a compact latent representation through a learned encoder, then match and consolidate representations across views based on geometric overlap and feature similarity via a learned merger. A level-of-detail decoder then produces the final Gaussians at a controllable resolution, enabling a flexible quality-efficiency trade-off at inference. As a post-processing module, the pipeline is backbone-agnostic, leveraging the strengths of existing feed-forward methods. This leads to better and more robust quality than achieved by previous approaches that target a reduction in primitive count, while providing a highly compact representation, that can be rendered efficiently.

This source distribution contains code and configuration for training and evaluating Gaussian merging. Provide your own datasets, weights and external backbones; review configuration values before starting long-running jobs.

Installation

Use Linux, an NVIDIA GPU, a compatible CUDA toolkit and driver, CMake, CUDA-enabled OpenCV development libraries, uv and the Python version in gmerge/.python-version. The Python OpenCV wheel does not replace the C++/CUDA libraries required by BASS. The tested environment used Python 3.14.6, PyTorch 2.11.0+cu130, CUDA 13.0 and an NVIDIA B200; other combinations are unverified. From the repository root:

cd gmerge
uv sync --frozen --no-install-package xformers
source .venv/bin/activate
MAX_JOBS=2 TORCH_CUDA_ARCH_LIST=10.0 uv sync --frozen
cd ..
bash scripts/build_bass.sh

Set TORCH_CUDA_ARCH_LIST for your GPU (10.0 for B200); xformers is built from source against the installed PyTorch/CUDA combination. Activate gmerge/.venv when running CUDA scripts directly. If a BASS build directory has a CMake cache from another checkout, set BASS_BUILD_DIR to a new directory rather than deleting it.

Set up only the external models you intend to run (source, weights and datasets are not included):

• AnySplat: Run bash scripts/setup_anysplat.sh to prepare the pinned AnySplat source in ignored external/AnySplat_integrated. Set ANYSPLAT_REPO to a new empty destination for a different location, both during setup and at runtime. Supply pretrained weights through the model hub or GMERGE_ANYSPLAT_SNAPSHOT.
• DA3: Obtain the Depth Anything 3 source and set DA3_REPO to its checkout. The model name in the base config selects pretrained weights from the model hub; arrange access to those weights before running.
• DepthSplat: Obtain the DepthSplat source, set DEPTHSPLAT_REPO to its checkout, and set GMERGE_DEPTHSPLAT_CHECKPOINT to compatible pretrained weights for the DepthSplat config.
• ReSplat and VolSplat baselines: Install each upstream repository in its own .venv following its upstream instructions. Set RESPLAT_REPO/VOLSPLAT_REPO to those checkouts and RESPLAT_CHECKPOINT/VOLSPLAT_CHECKPOINT to compatible local weights. Their evaluation launchers also require the DL3DV-Bench, MipNeRF360 and Tanks & Temples dataset roots.

See environment variables for the remaining paths and build overrides. The main launcher uses a venv-local CUDA extension cache by default.

Primary workflows

The Bash launcher resolves its location, uses the project environment and accepts absolute or caller-relative paths to its config/checkpoint input. Run the following commands from the repository root. Use absolute paths for dataset roots, generated data and outputs, because the launcher changes into gmerge/. The base training config expands ${VARIABLE} placeholders and fails on missing values; make private copies for other datasets/backbones.

export GMERGE_DL3DV_ROOT=/path/to/DL3DV-10K
export GMERGE_MIPNERF360_ROOT=/path/to/mipNeRF360
export GMERGE_DL3DV_BENCH_ROOT=/path/to/DL3DV-Bench
export GMERGE_TANKS_ROOT=/path/to/TanksAndTemples
export GMERGE_GENERATED_ROOT=/path/to/private/generated_merge_maps

# Full DA3 training: 300,000 steps with NVS supervision and post-training evaluation.
# Requires the DA3 source/weights described above. Use a new run name for each run.
GMERGE_POST_TRAIN_EVAL=1 GMERGE_POST_TRAIN_QUICK=0 bash scripts/run.sh train \
    gmerge/gmerge/configs/merge_pipe_base.json --run-name full_da3 \
    --override compute_merge_maps=true \
    --override "dataset_generated_roots={\"dl3dv\":\"${GMERGE_GENERATED_ROOT}\"}"

# Reconstruct all windows of one scene at the preset's 12-view, 476 resolution.
# Use a trained merging checkpoint with its matching train_config.json.
bash scripts/run.sh stream gmerge/gmerge/configs/streaming_recon/dl3dv.json \
    --merge-checkpoint /path/to/compatible/final.pt --dataset-root "$GMERGE_DL3DV_ROOT" \
    --generated-data-root "$GMERGE_GENERATED_ROOT" --split val --scene 0 \
    --max-steps -1 \
    --out-json /path/to/private/stream_results.json

# Full benchmark evaluation: all scenes and all four context-view counts.
# DA3 source/weights are required for masking, regardless of reconstruction backbone.
bash scripts/run.sh evaluate /path/to/compatible/final.pt \
    --dataset dl3dv-bench "$GMERGE_DL3DV_BENCH_ROOT" \
    --dataset mipnerf360 "$GMERGE_MIPNERF360_ROOT" \
    --dataset tanks_and_temples "$GMERGE_TANKS_ROOT" \
    --num-imgs 3 6 9 12 --resolution 476 --masking-protocol da3_depth

Training automatically evaluates the datasets configured in the training run; the explicit evaluation command above covers all three benchmarks. A full paper reproduction additionally requires matching the experiment's backbone, weights, split, configuration and checkpoint, not just completing these commands.

For full comparison-baseline evaluation, first configure the separate environments, repository paths and checkpoints described under Installation, including DA3 masking dependencies in each environment. Then run:

bash gmerge/scripts/eval/eval_resplat.sh
bash gmerge/scripts/eval/eval_volsplat.sh

These launchers evaluate all three benchmarks at 3, 6, 9 and 12 input views with DA3-depth masking.

Configuration and results

• Base training config: dataset roots from required private variables, reconstruction backbone, output, loss and training settings. Default training uses DA3, 12 context views, resolution 476 and 300,000 steps with NVS supervision. The command above generates missing merge maps in the writable private cache. For other backbones, see the backbone configs; review the selected recipe before launching.
• Streaming presets: per-dataset parameters; dataset and checkpoint must be supplied by CLI.
• Training writes checkpoints and logs under its configured output directory; streaming and evaluation use their configuration/CLI output settings. Data, pretrained weights and trained merging checkpoints are not included.
• The reported evaluation protocol is novel-view PSNR, SSIM, LPIPS and relative Gaussian count on DL3DV-Bench, MipNeRF360 and Tanks & Temples, using 3, 6, 9 and 12 input views.

Note: See env_vars.md for explanations on how to set up and use environment variables.

License

Original project code and original modifications are licensed under GNU AGPL version 3; this does not relicense copied or adapted upstream code. The third-party attribution inventory covers bundled BASS, the nerfview v0.0.3 viewer adaptation, gsplat adaptation and the Tanks & Temples downloader. External reconstruction source, model weights and datasets are not included.

Special thanks to Baysian Adaptive Superpixel Segementation which provides the basis for our Superpixel Segmentation.

Citation

@InProceedings{Faasch2026_GaussianMerging,
    author="Faasch, Tim-Felix
    and Kall, Jochen
    and Stachniss, Cyrill",
    booktitle="Proc.~of British Machine Vision Conf. (BMVC)",
    title="Compact Feed-Forward 3D Gaussians via Saliency Guided Primitive Merging",
    year="2026"
}