Restormer: Efficient Transformer for High-Resolution Image Restoration

Funding Agencies|NSF CAREER grant [1149783]; ARC DECRA Fellowship [DE200101100]
Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from largescale data, these models have been extensively applied to image restoration and related tasks. Recently, another class of neural architectures, Transformers, have shown significant performance gains on natural language and high-level vision tasks. While the Transformer model mitigates the shortcomings of CNNs (i.e., limited receptive field and inadaptability to input content), its computational complexity grows quadratically with the spatial resolution, therefore making it infeasible to apply to most image restoration tasks involving high-resolution images. In this work, we propose an efficient Transformer model by making several key designs in the building blocks (multi-head attention and feed-forward network) such that it can capture long-range pixel interactions, while still remaining applicable to large images. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks, including image deraining, single-image motion deblurring, defocus deblurring (single-image and dual-pixel data), and image denoising (Gaussian grayscale/color denoising, and real image denoising). The source code and pre-trained models are available at https://github.com/swz30/Restormer.

Arora, AdityaIncept Inst AI, U Arab Emirates (författare)
Khan, SalmanMohamed Bin Zayed Univ AI, U Arab Emirates (författare)
Hayat, MunawarMohamed Bin Zayed Univ AI, U Arab Emirates; Monash Univ, Australia (författare)
Khan, FahadLinköpings universitet,Datorseende,Tekniska fakulteten,Mohamed Bin Zayed Univ AI, U Arab Emirates(Swepub:liu)fahkh30 (författare)
Yang, Ming-HsuanUniv Calif Merced, CA USA; Yonsei Univ, South Korea; Google Res, CA USA (författare)
Incept Inst AI, U Arab EmiratesMohamed Bin Zayed Univ AI, U Arab Emirates (creator_code:org_t)

Ingår i:2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022): IEEE COMPUTER SOC, s. 5718-572997816654694639781665469470

Om ämnet

NATURVETENSKAP: NATURVETENSKAP; och Data och informa ...; och Datorseende och ...

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.