Sökning: onr:"swepub:oai:DiVA.org:hj-63652" >
A confidence-based ...
A confidence-based interface for neuro-symbolic visual question answering
-
- Eiter, Thomas (författare)
- Institute of Logic and Computation, Vienna University of Technology, Vienna, Austria
-
- Higuera, Nelson (författare)
- Institute of Logic and Computation, Vienna University of Technology, Vienna, Austria
-
- Oetsch, Johannes (författare)
- Institute of Logic and Computation, Vienna University of Technology, Vienna, Austria
-
visa fler...
-
- Pritz, Michael (författare)
- Institute of Logic and Computation, Vienna University of Technology, Vienna, Austria
-
visa färre...
-
(creator_code:org_t)
- 2022
- 2022
- Engelska.
-
Ingår i: Combining learning and reasoning: Programming languages, formalisms, and representations.
- Relaterad länk:
-
https://openreview.n...
-
visa fler...
-
https://urn.kb.se/re...
-
visa färre...
Abstract
Ämnesord
Stäng
- We present a neuro-symbolic visual question answering (VQA) approach for the CLEVR dataset that is based on the combination of deep neural networks and answer-set programming (ASP), a logic-based paradigm for declarative problem solving. We provide a translation mechanism for the questions included in CLEVR to ASP programs. By exploiting choice rules, we consider deterministic and non-deterministic scene encodings. In addition, we introduce a confidence-based interface between the ASP module and the neural network which allows us to restrict the non-determinism to objects classified by the network with high confidence. Our experiments show that the non-deterministic scene encoding achieves good results even if the neural networks are trained rather poorly in comparison with the deterministic approach. This is important for building robust VQA systems if network predictions are less-than perfect.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Nyckelord
- neuro-symbolic reasoning
- visual-question answering
- answer-set programming
- deep learning
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)