Contrastive Learning of Equivariant Image Representations for Multimodal Deformable Registration

• We propose a method for multimodal deformable image registration which combines a powerful deep learning approach to generate CoMIRs, dense image-like representations of multimodal image pairs, with INSPIRE, a robust framework for monomodal deformable image registration. We introduce new equivariance constraints to improve the consistency of CoMIRs under deformation. We evaluate the method on three publicly available multimodal datasets, one remote sensing, one histological, and one cytological. The proposed method demonstrates general applicability and consistently outperforms state-of-the-art registration tools \elastixname and VoxelMorph. We share source code of the proposed method and complete experimental setup as open-source at: https://github.com/MIDA-group/CoMIR_INSPIRE.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

Keyword

image alignment

correlative imaging

representation learning

equivariance

Publication and Content Type

ref (subject category)

kon (subject category)