Sökning: onr:"swepub:oai:DiVA.org:uu-510271" >
Can Representation ...
Can Representation Learning for Multimodal Image Registration be Improved by Supervision of Intermediate Layers?
-
- Wetzer, Elisabeth (författare)
- Uppsala universitet,Bildanalys och människa-datorinteraktion
-
- Lindblad, Joakim (författare)
- Uppsala universitet,Bildanalys och människa-datorinteraktion,Avdelningen Vi3
-
- Sladoje, Nataša (författare)
- Uppsala universitet,Bildanalys och människa-datorinteraktion,Avdelningen Vi3
-
(creator_code:org_t)
- Springer, 2023
- 2023
- Engelska.
-
Ingår i: IbPRIA 2023: Pattern Recognition and Image Analysis. - : Springer. - 9783031366154 - 9783031366161 ; , s. 261-275
- Relaterad länk:
-
https://doi.org/10.1...
-
visa fler...
-
https://urn.kb.se/re...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- Multimodal imaging and correlative analysis typically require image alignment. Contrastive learning can generate representations of multimodal images, reducing the challenging task of multimodal image registration to a monomodal one. Previously, additional supervision on intermediate layers in contrastive learning has improved biomedical image classification. We evaluate if a similar approach improves representations learned for registration to boost registration performance. We explore three approaches to add contrastive supervision to the latent features of the bottleneck layer in the U-Nets encoding the multimodal images and evaluate three different critic functions. Our results show that representations learned without additional supervision on latent features perform best in the downstream task of registration on two public biomedical datasets. We investigate the performance drop by exploiting recent insights in contrastive learning in classification and self-supervised learning. We visualize the spatial relations of the learned representations by means of multidimensional scaling, and show that additional supervision on the bottleneck layer can lead to partial dimensional collapse of the intermediate embedding space.
Ämnesord
- TEKNIK OCH TEKNOLOGIER -- Medicinteknik -- Medicinsk bildbehandling (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Medical Engineering -- Medical Image Processing (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
Nyckelord
- Contrastive learning; Multimodal image registration; Digital pathology
- Computerized Image Processing
- Datoriserad bildbehandling
- Machine learning
- Maskininlärning
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)
Hitta via bibliotek
Till lärosätets databas