This paper was accepted for publication in IEEE Transactions on Image Processing (https://ieeexplore.ieee.org/document/11653551).
Semantic correspondence seeks to establish matches between different instances of the same category. A common paradigm for this task leverages high-quality features from stable diffusion (SD) and DINOv2. However, we identify a widely overlooked yet critical issue: common feature aggregation disrupts the structural integrity of SD features, degrading semantic matching performance. We revisit and analyze this phenomenon and propose structure-aware aggregation (SAA) for SD features as a direct replacement for common feature aggregation methods. SAA uses filtering to decompose SD features into fine texture details and coarse contour structures. It aggregates only the texture components while preserving the contours. This divide-and-conquer mechanism enables SAA to significantly enhance the performance of state-of-the-art semantic correspondence models without increasing trainable parameters or computational overhead. Extensive qualitative and quantitative experiments confirm our analysis and validate the effectiveness of SAA. Moreover, SAA generalizes well to geometric, cross-species, and cross-family semantic correspondence tasks.
Our proposed SAA fully complies with the experimental and evaluation settings of GeoAware-SC and Tex2Sem. The models folder contains GeoAware-SC and Tex2Sem, both equipped with SAA. Researchers simply need to replace their original methods with these. The weights for the SAA (Gaussian) version are saved in the weights folder.
@ARTICLE{Wang2026RevisitingSC,
author = {Wang, Zenghui and Du, Songlin and Chen, Cheng and Lu, Xiaobo and Xiao, Guobao},
journal = {IEEE Transactions on Image Processing},
title = {Revisiting Semantic Correspondence: When Feature Aggregation Hurts Structural Integrity},
year = {2026},
volume = {35},
number = {},
pages = {8788-8803},
doi = {10.1109/TIP.2026.3721370}
}Our code is largely based on the following open-source projects: GeoAware-SC and Tex2Sem. We are deeply grateful for their spirit of open-source sharing.
