| 1. |
- Eiter, Thomas, et al.
(författare)
-
A confidence-based interface for neuro-symbolic visual question answering
- 2022
-
Ingår i: Combining learning and reasoning: Programming languages, formalisms, and representations.
-
Konferensbidrag (refereegranskat)abstract
- We present a neuro-symbolic visual question answering (VQA) approach for the CLEVR dataset that is based on the combination of deep neural networks and answer-set programming (ASP), a logic-based paradigm for declarative problem solving. We provide a translation mechanism for the questions included in CLEVR to ASP programs. By exploiting choice rules, we consider deterministic and non-deterministic scene encodings. In addition, we introduce a confidence-based interface between the ASP module and the neural network which allows us to restrict the non-determinism to objects classified by the network with high confidence. Our experiments show that the non-deterministic scene encoding achieves good results even if the neural networks are trained rather poorly in comparison with the deterministic approach. This is important for building robust VQA systems if network predictions are less-than perfect.
|
|
| 2. |
- Eiter, Thomas, et al.
(författare)
-
A Neuro-Symbolic ASP Pipeline for Visual Question Answering
- 2022
-
Ingår i: Theory and Practice of Logic Programming. - : Cambridge University Press. - 1471-0684 .- 1475-3081. ; 22:5, s. 739-754
-
Tidskriftsartikel (refereegranskat)abstract
- We present a neuro-symbolic visual question answering (VQA) pipeline for CLEVR, which is a well-known dataset that consists of pictures showing scenes with objects and questions related to them. Our pipeline covers (i) training neural networks for object classification and bounding-box prediction of the CLEVR scenes, (ii) statistical analysis on the distribution of prediction values of the neural networks to determine a threshold for high-confidence predictions, and (iii) a translation of CLEVR questions and network predictions that pass confidence thresholds into logic programmes so that we can compute the answers using an answer-set programming solver. By exploiting choice rules, we consider deterministic and non-deterministic scene encodings. Our experiments show that the non-deterministic scene encoding achieves good results even if the neural networks are trained rather poorly in comparison with the deterministic approach. This is important for building robust VQA systems if network predictions are less-than perfect. Furthermore, we show that restricting non-determinism to reasonable choices allows for more efficient implementations in comparison with related neuro-symbolic approaches without losing much accuracy.
|
|
| 3. |
- Sallinger, Katja, et al.
(författare)
-
Spatial tumour gene signature discriminates neoplastic from non-neoplastic compartments in colon cancer : unravelling predictive biomarkers for relapse
- 2023
-
Ingår i: Journal of Translational Medicine. - 1479-5876. ; 21:1
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Opting for or against the administration of adjuvant chemotherapy in therapeutic management of stage II colon cancer remains challenging. Several studies report few survival benefits for patients treated with adjuvant therapy and additionally revealing potential side effects of overtreatment, including unnecessary exposure to chemotherapy-induced toxicities and reduced quality of life. Predictive biomarkers are urgently needed. We, therefore, hypothesise that the spatial tissue composition of relapsed and non-relapsed colon cancer stage II patients reveals relevant biomarkers.Methods: The spatial tissue composition of stage II colon cancer patients was examined by a novel spatial transcriptomics technology with sub-cellular resolution, namely in situ sequencing. A panel of 176 genes investigating specific cancer-associated processes such as apoptosis, proliferation, angiogenesis, stemness, oxidative stress, hypoxia, invasion and components of the tumour microenvironment was designed to examine differentially expressed genes in tissue of relapsed versus non-relapsed patients. Therefore, FFPE slides of 10 colon cancer stage II patients either classified as relapsed (5 patients) or non-relapsed (5 patients) were in situ sequenced and computationally analysed.Results: We identified a tumour gene signature that enables the subclassification of tissue into neoplastic and non-neoplastic compartments based on spatial expression patterns obtained through in situ sequencing. We developed a computational tool called Genes-To-Count (GTC), which automates the quantification of in situ signals, accurately mapping their position onto the spatial tissue map and automatically identifies neoplastic and non-neoplastic tissue compartments. The GTC tool was used to quantify gene expression of biological processes upregulated within the neoplastic tissue in comparison to non-neoplastic tissue and within relapsed versus non-relapsed stage II colon patients. Three differentially expressed genes (FGFR2, MMP11 and OTOP2) in the neoplastic tissue compartments of relapsed patients in comparison to non-relapsed patients were identified predicting recurrence in stage II colon cancer.Conclusions: In depth spatial in situ sequencing showed potential to provide a deeper understanding of the underlying mechanisms involved in the recurrence of disease and revealed novel potential predictive biomarkers for disease relapse in colon cancer stage II patients. Our open-access GTC-tool allowed us to accurately capture the tumour compartment and quantify spatial gene expression in colon cancer tissue.
|
|
