| 1. |
- Hagström, Lovisa, 1995, et al.
(author)
-
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
- 2023
-
In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5457–5476, Singapore. - : Association for Computational Linguistics.
-
Conference paper (peer-reviewed)abstract
- Large Language Models (LLMs) make natural interfaces to factual knowledge, but their usefulness is limited by their tendency to deliver inconsistent answers to semantically equivalent questions. For example, a model might supply the answer “Edinburgh” to “Anne Redpath passed away in X.” and “London” to “Anne Redpath’s life ended in X.” In this work, we identify potential causes of inconsistency and evaluate the effectiveness of two mitigation strategies: up-scaling and augmenting the LM with a passage retrieval database. Our results on the LLaMA and Atlas models show that both strategies reduce inconsistency but that retrieval augmentation is considerably more efficient. We further consider and disentangle the consistency contributions of different components of Atlas. For all LMs evaluated we find that syntactical form and task artifacts impact consistency. Taken together, our results provide a better understanding of the factors affecting the factual consistency of language models.
|
|
| 2. |
- Ahrendt, Wolfgang, 1967, et al.
(author)
-
TriCo—Triple Co-piloting of Implementation, Specification and Tests
- 2022
-
In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - Cham : Springer International Publishing. - 1611-3349 .- 0302-9743. ; 13701 LNCS, s. 174-187, s. 174-187
-
Conference paper (peer-reviewed)abstract
- This white paper presents the vision of a novel methodology for developing safety-critical software, which is inspired by late developments in learning based co-piloting of implementations. The methodology, called TriCo, integrates formal methods with learning based approaches to co-pilot the agile, simultaneous development of three artefacts: implementation, specification, and tests. Whenever the user changes any of these, a TriCo empowered IDE would suggest changes to the other two artefacts in such a way that the three are kept consistent. The user has the final word on whether the changes are accepted, rejected, or modified. In the latter case, consistency will be checked again and re-established. We discuss the emerging trends which put the community in a good position to realise this vision, describe the methodology and workflow, as well as challenges and possible solutions for the realisation of TriCo.
|
|
| 3. |
- Atterfors, Johan, 1998, et al.
(author)
-
Machine Learning of Pacing Patterns for Half Marathon
- 2022
-
Journal article (other academic/artistic)abstract
- Every year over 40 000 runners participate in Gothenburg Half Marathon, one of the world’s largest half-marathons. Based on publicly available results data (423 496 entries) for ten years (2010 – 2019), we investigate machine learning models for two tasks: prediction of finishing times and identification of runners risking hitting the wall. Our models improve results over the current baseline on finish time prediction and manage to correctly identify many of the runners who later hit the wall, although it also misclassifies many who do not.
|
|
| 4. |
- Bonafilia, Brian, et al.
(author)
-
Sudden Semantic Shifts in Swedish NATO Discourse
- 2023
-
In: Association for Computational Linguistics . Annual Meeting Conference Proceedings. - 0736-587X. ; 4, s. 184-193
-
Conference paper (peer-reviewed)abstract
- In this paper, we investigate a type of semantic shift that occurs when a sudden event radically changes public opinion on a topic. Looking at Sweden's decision to apply for NATO membership in 2022, we use word embeddings to study how the associations users on Twitter have regarding NATO evolve. We identify several changes that we successfully validate against real-world events. However, the low engagement of the public with the issue often made it challenging to distinguish true signals from noise. We thus find that domain knowledge and data selection are of prime importance when using word embeddings to study semantic shifts.
|
|
| 5. |
- Bruinsma, Sebastianus Cornelis Jacobus, 1991, et al.
(author)
-
Finding the structure of parliamentary motions in the Swedish Riksdag 1971–2015
- 2023
-
In: Quality and Quantity. - 1573-7845 .- 0033-5177. ; In press
-
Journal article (peer-reviewed)abstract
- The current increase in the number of large, open sets of unstructured textual data has created both opportunities and challenges for social scientists. Here, we explore if and how we can use such data by looking at a dataset of over 144,000 documents used by parliamentary committees in Sweden. Of these, we aim to understand: (a) the topical content of these motions, (b) how these topics have changed over time, and (c) how these topics differ across political parties. To do so, we use a Structural Topic Model, which allows us to not only find the topics using the textual data itself, but also to include the documents’ metadata, such as authorship and date of publication. Doing so, we find 30 topics, which we combine into 9 broader themes. We find that these themes often rise and fall in popularity in line with historical events, and relate to the various political parties as we would expect. Throughout our analysis, we provide a step-by-step overview of how to use structural topic models in practice and also how to handle the type of dataset we use here.
|
|
| 6. |
- Einarsdóttir, Sólrún, 1991, et al.
(author)
-
Template-based Theory Exploration: Discovering Properties of Functional Programs by Testing
- 2020
-
In: ACM International Conference Proceeding Series. - New York, NY, USA : ACM. ; , s. 67-78
-
Conference paper (peer-reviewed)abstract
- We present RoughSpec, a template-based extension of the theory exploration tool QuickSpec. QuickSpec uses testing to automatically discover equational properties about functions in a Haskell program. These properties can help the user understand the program or be used as a source of possible lemmas in proofs of the program's correctness. In RoughSpec, the user supplies templates, which describe families of laws such as associativity and distributivity, and we only consider properties that match the templates. This restriction limits the search space and ensures that only relevant properties are discovered. In this way, we sacrifice broad search for more direction towards desirable property patterns, which makes theory exploration tractable and scalable. We also combine RoughSpec with QuickSpec, using QuickSpec to perform a complete search for smaller term sizes, while using templates for larger, more complex properties, in order to leverage the strengths of both systems.
|
|
| 7. |
- Eriksson, Rikard, et al.
(author)
-
Generating Weekly Training Plans in the Style of a Professional Swimming Coach Using Genetic Algorithms and Random Trees
- 2022
-
In: Proceedings of the 9th International Performance Analysis Workshop and Conference & 5th IACSS Conference (Advances in Intelligent Systems and Computing 1426). - Cham : Springer International Publishing. - 2194-5365 .- 2194-5357. - 9783030993337 - 9783030993320 ; 1426, s. 61-68
-
Conference paper (peer-reviewed)abstract
- Optimal training planning is a combination of art and sci- ence, a time-consuming task that requires expert knowledge. As such, it is often exclusively available to top tier athletes. Many athletes outside the elite do not have access or cannot afford to hire a professional coach to help them create their training plans. In this study, we investigate if it is possible to use the historical training logs of elite swimmers to con- struct detailed weekly training plans similar to how a specific professional coach would have planned. We present a software system based on machine learning and genetic algorithms for generation of detailed weekly training plans based on desired volume, intensity, training frequency, and athlete characteristics. The system schedules training sessions from a library extracted from training plans written by a professional swimming coach. Results show that the proposed system is able to generate highly accurate training plans in terms of training load, types of sessions, and structure, compared to the human coach.
|
|
| 8. |
- Jergéus, Erik, et al.
(author)
-
Towards Learning Abstractions via Reinforcement Learning
- 2022
-
In: CEUR Workshop Proceedings. - 1613-0073. ; 3400, s. 120-126
-
Conference paper (peer-reviewed)abstract
- In this paper we take the first steps in studying a new approach to synthesis of efficient communication schemes in multi-agent systems, trained via reinforcement learning. We combine symbolic methods with machine learning, in what is referred to as a neuro-symbolic system. The agents are not restricted to only use initial primitives: reinforcement learning is interleaved with steps to extend the current language with novel higher-level concepts, allowing generalisation and more informative communication via shorter messages. We demonstrate that this approach allow agents to converge more quickly on a small collaborative construction task.
|
|
| 9. |
- Johansson, Moa, 1981, et al.
(author)
-
Automated Conjecturing in QuickSpec
- 2021
-
In: 1 st Mathematical Reasoning in General Artificial Intelligence Workshop, ICLR 2021..
-
Conference paper (peer-reviewed)abstract
- A key component of mathematical reasoning is the ability to formulate interesting conjectures about a problem domain at hand. This task has not yet been widely studied by the automated reasoning and AI communities, but we believe interest is growing. In this paper, we give a brief overview of a theory exploration system called QuickSpec, able to automatically discover interesting conjectures about a given set of functions. QuickSpec works by interleaving term generation with random testing to form candidate equational conjectures. This is made tractable by starting from small sizes and ensuring that only terms that are irreducible with respect to already discovered equalities are considered. QuickSpec has been successfully applied to generate lemmas for automated inductive theorem proving as well as to generate specifications of functional programs. We also give a small survey of different approaches to conjecture discovery, and speculate about future directions combining symbolic methods and machine learning.
|
|
| 10. |
- Johansson, Moa, 1981, et al.
(author)
-
Conjectures, tests and proofs: An overview of theory exploration
- 2021
-
In: Electronic Proceedings in Theoretical Computer Science, EPTCS. - 2075-2180. ; 341, s. 1-16
-
Conference paper (peer-reviewed)abstract
- A key component of mathematical reasoning is the ability to formulate interesting conjectures about a problem domain at hand. In this paper, we give a brief overview of a theory exploration system called QuickSpec, which is able to automatically discover interesting conjectures about a given set of functions. QuickSpec works by interleaving term generation with random testing to form candidate conjectures. This is made tractable by starting from small sizes and ensuring that only terms that are irreducible with respect to already discovered conjectures are considered. QuickSpec has been successfully applied to generate lemmas for automated inductive theorem proving as well as to generate specifications of functional programs. We give an overview of typical use-cases of QuickSpec, as well as demonstrating how to easily connect it to a theorem prover of the user’s choice.
|
|
