Gigapixel end-to-end training using streaming and attention

Funding Agencies|Innovative Medicines Initiative 2 Joint Undertaking [945358]; European Union; EFPIA, Belgium
Current hardware limitations make it impossible to train convolutional neural networks on gigapixel image inputs directly. Recent developments in weakly supervised learning, such as attention-gated multiple instance learning, have shown promising results, but often use multi-stage or patch-wise training strategies risking suboptimal feature extraction, which can negatively impact performance. In this paper, we propose to train a ResNet-34 encoder with an attention-gated classification head in an end-to-end fashion, which we call StreamingCLAM, using a streaming implementation of convolutional layers. This allows us to train end-to-end on 4-gigapixel microscopic images using only slide-level labels.We achieve a mean area under the receiver operating characteristic curve of 0.9757 for metastatic breast cancer detection (CAMELYON16), close to fully supervised approaches using pixel-level annotations. Our model can also detect MYC-gene translocation in histologic slides of diffuse large B-cell lymphoma, achieving a mean area under the ROC curve of 0.8259. Furthermore, we show that our model offers a degree of interpretability through the attention mechanism.

Pinckaers, HansRadboud Univ Nijmegen, Netherlands (author)
Aswolinskiy, WitaliRadboud Univ Nijmegen, Netherlands (author)
Hebeda, KonnieRadboud Univ Nijmegen, Netherlands (author)
Jarkman, SofiaLinköpings universitet,Avdelningen för neurobiologi,Medicinska fakulteten,Centrum för medicinsk bildvetenskap och visualisering, CMIV,Region Östergötland, Klinisk patologi(Swepub:liu)sofja84 (author)
van der Laak, JeroenLinköpings universitet,Avdelningen för diagnostik och specialistmedicin,Medicinska fakulteten,Centrum för medicinsk bildvetenskap och visualisering, CMIV,Region Östergötland, Klinisk patologi,Radboud Univ Nijmegen, Netherlands(Swepub:liu)jerva26 (author)
Litjens, GeertRadboud Univ Nijmegen, Netherlands (author)
Radboud Univ Nijmegen, NetherlandsAvdelningen för neurobiologi (creator_code:org_t)

By the author/editor: Dooper, Stephan; Pinckaers, Hans; Aswolinskiy, Wit ...; Hebeda, Konnie; Jarkman, Sofia; van der Laak, Je ...; show more...; Litjens, Geert; show less...

About the subject

NATURAL SCIENCES: NATURAL SCIENCES; and Computer and Inf ...; and Computer Science ...

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.