Search: onr:"swepub:oai:DiVA.org:uu-459429" >
Contrastive Learnin...
Contrastive Learning for Equivariant Multimodal Image Representations
-
- Wetzer, Elisabeth (author)
- Uppsala universitet,Avdelningen för visuell information och interaktion,Bildanalys och människa-datorinteraktion
-
- Pielawski, Nicolas (author)
- Uppsala universitet,Avdelningen för visuell information och interaktion
-
- Breznik, Eva (author)
- Uppsala universitet,Bildanalys och människa-datorinteraktion,Avdelningen för visuell information och interaktion
-
show more...
-
- Öfverstedt, Johan (author)
- Uppsala universitet,Avdelningen för visuell information och interaktion,Bildanalys och människa-datorinteraktion
-
Lu, Jiahao (author)
-
- Wählby, Carolina, professor, 1974- (author)
- Uppsala universitet,Bildanalys och människa-datorinteraktion,Science for Life Laboratory, SciLifeLab,Avdelningen för visuell information och interaktion
-
- Lindblad, Joakim (author)
- Uppsala universitet,Bildanalys och människa-datorinteraktion,Avdelningen för visuell information och interaktion
-
- Sladoje, Natasa (author)
- Uppsala universitet,Avdelningen för visuell information och interaktion,Bildanalys och människa-datorinteraktion
-
show less...
-
(creator_code:org_t)
- Cambridge University, 2021
- 2021
- English.
- Related links:
-
https://urn.kb.se/re...
Abstract
Subject headings
Close
- Combining the information of different imaging modalities offers complimentary information about the properties of the imaged specimen. Often these modalities need to be captured by different machines, which requires that the resulting images need to be matched and registered in order to map the corresponding signals to each other. This can be a very challenging task due to the varying appearance of the specimen in different sensors.We have recently developed a method which uses contrastive learning to find representations of both modalities, such that the images of different modalities are mapped into the same representational space. The learnt representations (referred to as CoMIRs) are abstract and very similar with respect to a selected similarity measure. There are requirements which these representations need to fulfil for downstream tasks such as registration - e.g rotational equivariance or intensity similarity. We present a hyperparameter free modification of the contrastive loss, which is based on InfoNCE, to produce equivariant, dense-like image representations. These representations are similar enough to be considered in a common space, in which monomodal methods for registration can be exploited.
Subject headings
- TEKNIK OCH TEKNOLOGIER -- Medicinteknik -- Medicinsk bildbehandling (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Medical Engineering -- Medical Image Processing (hsv//eng)
Keyword
- Computerized Image Processing
- Datoriserad bildbehandling
Publication and Content Type
- vet (subject category)
- kon (subject category)
To the university's database