| 1. |
- Norlund, Tobias, 1991, et al.
(författare)
-
Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?
- 2021
-
Ingår i: Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 149-162, Punta Cana, Dominican Republic. - : Association for Computational Linguistics.
-
Konferensbidrag (refereegranskat)abstract
- Large language models are known to suffer from the hallucination problem in that they are prone to output statements that are false or inconsistent, indicating a lack of knowledge. A proposed solution to this is to provide the model with additional data modalities that complements the knowledge obtained through text. We investigate the use of visual data to complement the knowledge of large language models by proposing a method for evaluating visual knowledge transfer to text for uni- or multimodal language models. The method is based on two steps, 1) a novel task querying for knowledge of memory colors, i.e. typical colors of well-known objects, and 2) filtering of model training data to clearly separate knowledge contributions. Additionally, we introduce a model architecture that involves a visual imagination step and evaluate it with our proposed method. We find that our method can successfully be used to measure visual knowledge transfer capabilities in models and that our novel model architecture shows promising results for leveraging multimodal knowledge in a unimodal setting.
|
|
| 2. |
- Lindén, Joakim, et al.
(författare)
-
Evaluating the Robustness of ML Models to Out-of-Distribution Data Through Similarity Analysis
- 2023
-
Ingår i: Commun. Comput. Info. Sci.. - : Springer Science and Business Media Deutschland GmbH. - 9783031429408 ; , s. 348-359, s. 348-359
-
Konferensbidrag (refereegranskat)abstract
- In Machine Learning systems, several factors impact the performance of a trained model. The most important ones include model architecture, the amount of training time, the dataset size and diversity. We present a method for analyzing datasets from a use-case scenario perspective, detecting and quantifying out-of-distribution (OOD) data on dataset level. Our main contribution is the novel use of similarity metrics for the evaluation of the robustness of a model by introducing relative Fréchet Inception Distance (FID) and relative Kernel Inception Distance (KID) measures. These relative measures are relative to a baseline in-distribution dataset and are used to estimate how the model will perform on OOD data (i.e. estimate the model accuracy drop). We find a correlation between our proposed relative FID/relative KID measure and the drop in Average Precision (AP) accuracy on unseen data.
|
|
| 3. |
- Suchan, Jakob, et al.
(författare)
-
Commonsense Visual Sensemaking for Autonomous Driving : On Generalised Neurosymbolic Online Abduction Integrating Vision and Semantics
- 2021
-
Ingår i: Artificial Intelligence. - : Elsevier. - 0004-3702 .- 1872-7921. ; 299
-
Tidskriftsartikel (refereegranskat)abstract
- We demonstrate the need and potential of systematically integrated vision and semantics solutions for visual sensemaking in the backdrop of autonomous driving. A general neurosymbolic method for online visual sensemaking using answer set programming (ASP) is systematically formalised and fully implemented. The method integrates state of the art in visual computing, and is developed as a modular framework that is generally usable within hybrid architectures for realtime perception and control. We evaluate and demonstrate with community established benchmarks KITTIMOD, MOT-2017, and MOT-2020. As use-case, we focus on the significance of human-centred visual sensemaking —e.g., involving semantic representation and explainability, question-answering, commonsense interpolation— in safety-critical autonomous driving situations. The developed neurosymbolic framework is domain-independent, with the case of autonomous driving designed to serve as an exemplar for online visual sensemaking in diverse cognitive interaction settings in the backdrop of select human-centred AI technology design considerations.
|
|
| 4. |
- Somanath, Sanjay, 1994, et al.
(författare)
-
Towards Urban Digital Twins: A Workflow for Procedural Visualization Using Geospatial Data
- 2024
-
Ingår i: Remote Sensing. - 2072-4292. ; 16:11
-
Tidskriftsartikel (refereegranskat)abstract
- A key feature for urban digital twins (DTs) is an automatically generated detailed 3D representation of the built and unbuilt environment from aerial imagery, footprints, LiDAR, or a fusion of these. Such 3D models have applications in architecture, civil engineering, urban planning, construction, real estate, Geographical Information Systems (GIS), and many other areas. While the visualization of large-scale data in conjunction with the generated 3D models is often a recurring and resource-intensive task, an automated workflow is complex, requiring many steps to achieve a high-quality visualization. Methods for building reconstruction approaches have come a long way, from previously manual approaches to semi-automatic or automatic approaches. This paper aims to complement existing methods of 3D building generation. First, we present a literature review covering different options for procedural context generation and visualization methods, focusing on workflows and data pipelines. Next, we present a semi-automated workflow that extends the building reconstruction pipeline to include procedural context generation using Python and Unreal Engine. Finally, we propose a workflow for integrating various types of large-scale urban analysis data for visualization. We conclude with a series of challenges faced in achieving such pipelines and the limitations of the current approach. However, the steps for a complete, end-to-end solution involve further developing robust systems for building detection, rooftop recognition, and geometry generation and importing and visualizing data in the same 3D environment, highlighting a need for further research and development in this field.
|
|
| 5. |
- Blanch, Krister, 1991
(författare)
-
Beyond-application datasets and automated fair benchmarking
- 2023
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Beyond-application perception datasets are generalised datasets that emphasise the fundamental components of good machine perception data. When analysing the history of perception datatsets, notable trends suggest that design of the dataset typically aligns with an application goal. Instead of focusing on a specific application, beyond-application datasets instead look at capturing high-quality, high-volume data from a highly kinematic environment, for the purpose of aiding algorithm development and testing in general. Algorithm benchmarking is a cornerstone of autonomous systems development, and allows developers to demonstrate their results in a comparative manner. However, most benchmarking systems allow developers to use their own hardware or select favourable data. There is also little focus on run time performance and consistency, with benchmarking systems instead showcasing algorithm accuracy. By combining both beyond-application dataset generation and methods for fair benchmarking, there is also the dilemma of how to provide the dataset to developers for this benchmarking, as the result of a high-volume, high-quality dataset generation is a significant increase in dataset size when compared to traditional perception datasets. This thesis presents the first results of attempting the creation of such a dataset. The dataset was built using a maritime platform, selected due to the highly dynamic environment presented on water. The design and initial testing of this platform is detailed, as well as as methods of sensor validation. Continuing, the thesis then presents a method of fair benchmarking, by utilising remote containerisation in a way that allows developers to present their software to the dataset, instead of having to first locally store a copy. To test this dataset and automatic online benchmarking, a number of reference algorithms were required for initial results. Three algorithms were built, using the data from three different sensors captured on the maritime platform. Each algorithm calculates vessel odometry, and the automatic benchmarking system was utilised to show the accuracy and run-time performance of these algorithms. It was found that the containerised approach alleviated data management concerns, prevented inflated accuracy results, and demonstrated precisely how computationally intensive each algorithm was.
|
|
| 6. |
- Liu, Yuqi, et al.
(författare)
-
Integration of Multi-scale Spatial Digital Twins in Metaverse Based on Multi-dimensional Hash Geocoding
- 2024
-
Ingår i: IMX '24. - : Association for Computing Machinery (ACM). - 9798400705038 ; , s. 56-63
-
Konferensbidrag (refereegranskat)abstract
- With the popularization of the metaverse, virtual reality mapping technology based on digital twins has generated a large amount of spatial data. These data are multidimensional, multi-scale, mobile, and distributed. In order to fully utilize these data, we propose a non mutation multidimensional hash geocoding that can organize and store data with geographic features, and achieve data mapping at different scales from macro to micro. The mapping between them can achieve joint utilization of data of various scales. On this basis, we propose a block network secure storage mapping model for spatial digital twins, which can securely and reliably organize and map spatial data. This article also looks forward to the possible emergence of digital twins of different dimensions and scales in the future metaverse, and proposes an adaptive 3D reconstruction method based on this to adapt to digital twins models of different scales in the metaverse. On the basis of our work, we will further promote the development of the spatial digital twin metaverse.
|
|
| 7. |
- Ali, Muhaddisa Barat, 1986
(författare)
-
Deep Learning Methods for Classification of Gliomas and Their Molecular Subtypes, From Central Learning to Federated Learning
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The most common type of brain cancer in adults are gliomas. Under the updated 2016 World Health Organization (WHO) tumor classification in central nervous system (CNS), identification of molecular subtypes of gliomas is important. For low grade gliomas (LGGs), prediction of molecular subtypes by observing magnetic resonance imaging (MRI) scans might be difficult without taking biopsy. With the development of machine learning (ML) methods such as deep learning (DL), molecular based classification methods have shown promising results from MRI scans that may assist clinicians for prognosis and deciding on a treatment strategy. However, DL requires large amount of training datasets with tumor class labels and tumor boundary annotations. Manual annotation of tumor boundary is a time consuming and expensive process. The thesis is based on the work developed in five papers on gliomas and their molecular subtypes. We propose novel methods that provide improved performance. The proposed methods consist of a multi-stream convolutional autoencoder (CAE)-based classifier, a deep convolutional generative adversarial network (DCGAN) to enlarge the training dataset, a CycleGAN to handle domain shift, a novel federated learning (FL) scheme to allow local client-based training with dataset protection, and employing bounding boxes to MRIs when tumor boundary annotations are not available. Experimental results showed that DCGAN generated MRIs have enlarged the original training dataset size and have improved the classification performance on test sets. CycleGAN showed good domain adaptation on multiple source datasets and improved the classification performance. The proposed FL scheme showed a slightly degraded performance as compare to that of central learning (CL) approach while protecting dataset privacy. Using tumor bounding boxes showed to be an alternative approach to tumor boundary annotation for tumor classification and segmentation, with a trade-off between a slight decrease in performance and saving time in manual marking by clinicians. The proposed methods may benefit the future research in bringing DL tools into clinical practice for assisting tumor diagnosis and help the decision making process.
|
|
| 8. |
- Lindgren, Helena, Professor, et al.
(författare)
-
The wasp-ed AI curriculum : A holistic curriculum for artificial intelligence
- 2023
-
Ingår i: INTED2023 Proceedings. - : IATED. - 9788409490264 ; , s. 6496-6502
-
Konferensbidrag (refereegranskat)abstract
- Efforts in lifelong learning and competence development in Artificial Intelligence (AI) have been on the rise for several years. These initiatives have mostly been applied to Science, Technology, Engineering and Mathematics (STEM) disciplines. Even though there has been significant development in Digital Humanities to incorporate AI methods and tools in higher education, the potential for such competences in Arts, Humanities and Social Sciences is far from being realised. Furthermore, there is an increasing awareness that the STEM disciplines need to include competences relating to AI in humanity and society. This is especially important considering the widening and deepening of the impact of AI on society at large and individuals. The aim of the presented work is to provide a broad and inclusive AI Curriculum that covers the breadth of the topic as it is seen today, which is significantly different from only a decade ago. It is important to note that with the curriculum we mean an overview of the subject itself, rather than a particular education program. The curriculum is intended to be used as a foundation for educational activities in AI to for example harmonize terminology, compare different programs, and identify educational gaps to be filled. An important aspect of the curriculum is the ethical, legal, and societal aspects of AI and to not limit the curriculum to the STEM subjects, instead extending to a holistic, human-centred AI perspective. The curriculum is developed as part of the national research program WASP-ED, the Wallenberg AI and transformative technologies education development program.
|
|
| 9. |
- Lv, Zhihan, Dr. 1984-, et al.
(författare)
-
5G for mobile augmented reality
- 2022
-
Ingår i: International Journal of Communication Systems. - : John Wiley & Sons. - 1074-5351 .- 1099-1131. ; 35:5
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)
|
|
| 10. |
- Lv, Zhihan, Dr. 1984-, et al.
(författare)
-
Editorial : 5G for Augmented Reality
- 2022
-
Ingår i: Mobile Networks and Applications. - : Springer. - 1383-469X .- 1572-8153.
-
Tidskriftsartikel (refereegranskat)
|
|
