Skip to content

About

UniFlow-RS: Unified Generative Prediction for Heterogeneous Remote-Sensing Tasks

Resources

Stars

13 stars

Watchers

1 watching

Forks

Repository files navigation

UniFlow-RS

Unified Generative Prediction
for Heterogeneous Remote-Sensing Tasks

Jianlong He1, Liang Liao2, Jing Xiao1, Chaofeng Chen1, Gui-Song Xia1

1School of Artificial Intelligence, Wuhan University
2Hangzhou Institute of Technology, Xidian University

arXiv · Project Page

UniFlow-RS overview: a shared generative prediction process across heterogeneous remote sensing tasks.

UniFlow-RS formulates heterogeneous remote sensing tasks as a shared generative prediction process. Binary masks, semantic maps, height fields, referring masks, and rotated-object targets are represented in a common structured visual space, predicted through one shared model, and recovered with task-dependent deterministic decoders.

UniFlow-RS prerecorded qualitative demo: task prompts, input images, ground truth, and saved predictions with synchronized detail zooms across six tasks.

Requirements

  • Python 3.10
  • PyTorch with CUDA and BF16 support
  • A CUDA-capable GPU; single-GPU and multi-GPU evaluation are supported

Installation

Run the following commands from the repository root:

python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Direct dependencies are pinned to the observed evaluation environment (Python 3.10.8, PyTorch 2.8.0, torchvision 0.23.0, CUDA 12.8 build). Use a compatible NVIDIA driver. See reproducibility notes for evaluation settings and commands.

Models

Download the UniFlow-RS checkpoint (Hugging Face) and save uniflow_rs.pt under checkpoints/.

Place the base transformer, autoencoder, text encoder, and UniFlow-RS adapter in the following locations:

pretrained/
├── flux2-klein-base-4b/
│   ├── flux-2-klein-base-4b.safetensors
│   └── ae.safetensors
└── qwen3-4b-fp8/
    └── ... text encoder model files ...
checkpoints/
└── uniflow_rs.pt
Download the upstream base-model assets

These commands download third-party assets, not the UniFlow-RS adapter. Upstream access requirements and asset licenses apply.

hf download black-forest-labs/FLUX.2-klein-base-4B \
  flux-2-klein-base-4b.safetensors \
  --local-dir pretrained/flux2-klein-base-4b
hf download black-forest-labs/FLUX.2-dev ae.safetensors \
  --local-dir pretrained/flux2-klein-base-4b
hf download Qwen/Qwen3-4B-FP8 \
  --local-dir pretrained/qwen3-4b-fp8

Inference

Follow docs/DATASETS.md and place prepared datasets below data/. Check the dataset paths in configs/uniflow_rs_512_2step_eval.yaml; keep the model, prompt, palette, preprocessing, and inference settings fixed when comparing results.

The evaluation defaults use two flow steps, BF16, and seed 12345. Batch size and other runtime settings are defined in configs/uniflow_rs_512_2step_eval.yaml. Evaluation defaults to a single GPU. For multi-GPU evaluation, explicitly set NUM_PROCESSES to the desired GPU count (for example, NUM_PROCESSES=2).

One benchmark

METRICS_ONLY=1 \
  bash scripts/eval.sh whu_cd checkpoints/uniflow_rs.pt outputs/whu_cd
All supported dataset keys
whu_building       spacenet6           whu_cd
levir_cd_plus      urbansar_floods     loveda
openearthmap_rgb   openearthmap_sar    potsdam
dfc23_track2       rrsisd              lasers
dior_r

All thirteen benchmarks

Metrics only:

METRICS_ONLY=1 \
  bash scripts/eval.sh all checkpoints/uniflow_rs.pt outputs/all_benchmarks

Metrics, predictions, and comparison panels:

bash scripts/eval.sh all checkpoints/uniflow_rs.pt outputs/all_benchmarks

Leave METRICS_ONLY unset to export images. Keep the process count and batch size fixed when comparing repeated runs.

Results include metrics_summary.json, per_dataset_metrics.json, per-dataset JSON files, and task-specific metrics.json files. Visualization-enabled runs also save inputs, ground truth, probabilities, predictions, and comparison panels as applicable to each task.

Code structure

Start with uniflow_rs/model.py: UniFlowRS assembles the frozen VAE, shared FLUX.2 + LoRA transformer, and text conditions. Evaluation uses this facade for checkpoint loading and prediction.

Image(s) + prompt
  -> visual/text encoding and task-dependent source state
  -> shared FLUX.2 + LoRA flow integration
  -> VAE decoding into a structured visual target
  -> deterministic task recovery

Dataset preprocessing lives in uniflow_rs/data/; evaluation, task recovery, and metrics are coordinated by uniflow_rs/evaluation/ and uniflow_rs/task_runtime.py. Training starts in uniflow_rs/training/engine.py; TMAS handles training-time timestep sampling in uniflow_rs/training/tmas.py.

Citation

@misc{he2026uniflowrs,
  title={UniFlow-RS: Unified Generative Prediction for
         Heterogeneous Remote-Sensing Tasks},
  author={He, Jianlong and Liao, Liang and Xiao, Jing and
          Chen, Chaofeng and Xia, Gui-Song},
  year={2026},
  eprint={2611.14281},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2611.14281}
}

License

The source release is provided under Apache-2.0. The official FLUX.2 implementation and pinned revision are documented in NOTICE and the upstream notice. Model weights, Qwen assets, and datasets remain subject to their respective upstream terms; they are not redistributed in this source package.

About

UniFlow-RS: Unified Generative Prediction for Heterogeneous Remote-Sensing Tasks

Resources

Stars

13 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages