View : 655 Download: 0
Dense Cross-Modal Correspondence Estimation With the Deep Self-Correlation Descriptor
- Title
- Dense Cross-Modal Correspondence Estimation With the Deep Self-Correlation Descriptor
- Authors
- Kim, Seungryong; Min, Dongbo; Lin, Stephen; Sohn, Kwanghoon
- Ewha Authors
- 민동보
- SCOPUS Author ID
- 민동보
- Issue Date
- 2021
- Journal Title
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
- ISSN
- 0162-8828
1939-3539
- Citation
- IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE vol. 43, no. 7, pp. 2345 - 2359
- Keywords
- Strain; Lighting; Estimation; Benchmark testing; Imaging; Robustness; Visualization; Cross-modal correspondence; pyramidal structure; self-correlation; local self-similarity; non-rigid deformation
- Publisher
- IEEE COMPUTER SOC
- Indexed
- SCIE; SCOPUS
- Document Type
- Article
- Abstract
- We present the deep self-correlation (DSC) descriptor for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. We encode local self-similar structure in a pyramidal manner that yields both more precise localization ability and greater robustness to non-rigid image deformations. Specifically, DSC first computes multiple self-correlation surfaces with randomly sampled patches over a local support window, and then builds pyramidal self-correlation surfaces through average pooling on the surfaces. The feature responses on the self-correlation surfaces are then encoded through spatial pyramid pooling in a log-polar configuration. To better handle geometric variations such as scale and rotation, we additionally propose the geometry-invariant DSC (GI-DSC) that leverages multi-scale self-correlation computation and canonical orientation estimation. In contrast to descriptors based on deep convolutional neural networks (CNNs), DSC and GI-DSC are training-free (i.e., handcrafted descriptors), are robust to cross-modality, and generalize well to various modality variations. Extensive experiments demonstrate the state-of-the-art performance of DSC and GI-DSC on challenging cases of cross-modal image pairs having photometric and/or geometric variations.
- DOI
- 10.1109/TPAMI.2020.2965528
- Appears in Collections:
- 인공지능대학 > 컴퓨터공학과 > Journal papers
- Files in This Item:
There are no files associated with this item.
- Export
- RIS (EndNote)
- XLS (Excel)
- XML