A generative appearance model for end-to-end video object segmentation

Johnander, JoakimLinköpings universitet,Datorseende,Tekniska fakulteten,Zenuity, Sweden (författare)

One of the fundamental challenges in video object segmentation is to find an effective representation of the target and background appearance. The best performing approaches resort to extensive fine-tuning of a convolutional neural network for this purpose. Besides being prohibitively expensive, this strategy cannot be truly trained end-to-end since the online fine-tuning procedure is not integrated into the offline training of the network. To address these issues, we propose a network architecture that learns a powerful representation of the target and background appearance in a single forward pass. The introduced appearance module learns a probabilistic generative model of target and background feature distributions. Given a new image, it predicts the posterior class probabilities, providing a highly discriminative cue, which is processed in later network modules. Both the learning and prediction stages of our appearance module are fully differentiable, enabling true end-to-end training of the entire segmentation pipeline. Comprehensive experiments demonstrate the effectiveness of the proposed approach on three video object segmentation benchmarks. We close the gap to approaches based on online fine-tuning on DAVIS17, while operating at 15 FPS on a single GPU. Furthermore, our method outperforms all published approaches on the large-scale YouTube-VOS dataset.

Danelljan, Martin,1989-Linköpings universitet,Datorseende,Tekniska fakulteten,ETH Zurich, Switzerland(Swepub:liu)marda26 (författare)
Brissman, EmilLinköpings universitet,Datorseende,Tekniska fakulteten,Saab, Sweden(Swepub:liu)emibr12 (författare)
Khan, Fahad Shahbaz,1983-Linköpings universitet,Datorseende,Tekniska fakulteten,IIAI, UAE(Swepub:liu)fahkh30 (författare)
Felsberg, Michael,1974-Linköpings universitet,Datorseende,Tekniska fakulteten(Swepub:liu)micfe03 (författare)
Linköpings universitetDatorseende (creator_code:org_t)

Ingår i:2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR): Institute of Electrical and Electronics Engineers (IEEE), s. 8945-895497817281329389781728132945

Av författaren/redakt...: Johnander, Joaki ...; Danelljan, Marti ...; Brissman, Emil; Khan, Fahad Shah ...; Felsberg, Michae ...

Om ämnet

NATURVETENSKAP: NATURVETENSKAP; och Data och informa ...; och Datorseende och ...

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.