| 1. |
- Berman, Alexander, 1979, et al.
(författare)
-
"Apparently acousticness is positively correlated with neuroticism": Conversational explanations of model predictions
- 2022
-
Ingår i: Proceedings of the 26th Workshop on the Semantics and Pragmatics of Dialogue, August, 22-24, 2022, Dublin, Ireland - Poster Abstracts.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This paper describes an experiment that collects human dialogues about predictions of participants’ personality traits on the basis of their music preferences, and presents preliminary results. This type of data can inform the design of explanatory dialogue systems, and the method can straightforwardly be adapted to other domains and statistical models.
|
|
| 2. |
|
|
| 3. |
- Berman, Alexander, 1979, et al.
(författare)
-
Counterfactual reasoning capabilities of GPT: Preliminary findings
- 2023
-
Ingår i: Proceedings of the 18th SweCog Conference, Göteborg, 5-6 October, 2023. - 1653-2325. - 9789198903805
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Recently, there has been a large interest in large language models (LLMs) such as GPT and their ability to engage in human-like dialogue and use commonsense reasoning. We experimentally investigate specific aspects of these abilities, namely counterfactual reasoning and explanations. These abilities are particularly important when using LLMs to assist high-stake decisions and assessments such as credit approval or medical diagnostics. For example, if a loan applicant is denied credit, a counterfactual explanation conveys the conditions under which the credit would have been granted. By injecting a decision-making algorithm into the model's prompt and systematically probing and annotating responses for carefully chosen inputs, we study potential patterns in GPT's selection of counterfactual examples. Preliminary results indicate that when GPT 3.5 provides counterfactual explanations, it does not consider causal relations between variables in a way that one would expect from a model with strong commonsense reasoning capabilities. We discuss potential implications of these results for real-world applications and future research.
|
|
| 4. |
- Berman, Alexander, 1979, et al.
(författare)
-
Explaining predictions with enthymematic counterfactuals
- 2022
-
Ingår i: Proceedings of the 1st Workshop on Bias, Ethical AI, Explainability and the role of Logic and Logic Programming, BEWARE-22. - : CEUR Workshop Proceedings. - 1613-0073.
-
Konferensbidrag (refereegranskat)abstract
- When people are subject to high-stakes decisions informed by computer models, they have a legitimate interest in understanding the basis for the model’s judgements and whether actions can be taken to turn a dispreferred decision into a preferred one. For example, if an application for a loan is denied by the model, the applicant has an interest in understanding the conditions that would yield an approval. In this paper, we argue that these kinds of counterfactual (or contrastive) explanations rest on domain-specific and commonsensical principles that can be negotiated, and sketch a method for incorporating such principles in an explanatory dialogue system using enthymematic reasoning.
|
|
| 5. |
- Berman, Alexander, 1979, et al.
(författare)
-
Trustworthy AI in the public sector: An empirical analysis of a Swedish labor market decision-support system
- 2024
-
Ingår i: Technology in society. - 0160-791X. ; 76:2024, s. 1-15
-
Tidskriftsartikel (refereegranskat)abstract
- This paper investigates the deployment of Artificial Intelligence (AI) in the Swedish Public Employment Service (PES), focusing on the concept of trustworthy AI in public decision-making. Despite Sweden’s advanced digitalization efforts and the widespread application of AI in the public sector, our study reveals significant gaps between theoretical ambitions and practical outcomes, particularly in the context of AI’s trustworthiness. We employ a robust theoretical framework comprising Institutional Theory, the Resource-Based View (RBV), and Ambidexterity Theory, to analyze the challenges and discrepancies in AI implementation within PES. Our analysis shows that while AI promises enhanced decision-making efficiency, the reality is marred by issues of transparency, interpretability, and stakeholder engagement. The opacity of the neural network used by the agency to assess jobseekers’ need for support and the lack of comprehensive technical understanding among PES management contribute to the challenges in achieving transparent and interpretable AI systems. Economic pressures for efficiency often overshadow the need for ethical considerations and stakeholder involvement, leading to decisions that may not be in the best interest of jobseekers. We propose recommendations for enhancing AI’s trustworthiness in public services, emphasizing the importance of stakeholder engagement, particularly involving jobseekers in the decision-making process. Our study advocates for a more nuanced balance between the use of advanced AI technologies and the leveraging of internal resources such as skilled personnel and organizational knowledge. We also highlight the need for improved AI literacy among both management and personnel to effectively navigate AI’s integration into public decision-making processes. Our findings contribute to the ongoing debate on trustworthy AI, offering a detailed case study that bridges the gap between theoretical exploration and practical application. By scrutinizing the AI implementation in the Swedish PES, we provide valuable insights and guidelines for other public sector organizations grappling with the integration of AI into their decision-making processes.
|
|
| 6. |
- Berman, Alexander, 1979
(författare)
-
Why does the AI say that I am too far away from the job market?
- 2023
-
Ingår i: Proceedings of the Weizenbaum Conference 2023: AI, Big Data, Social Media, and People on the Move, 19-20 June 2023, Berlin / Berendt Bettina, Krzywdzinski Martin, Kuznetsova Elizaveta (ed.). - Berlin : Weizenbaum Institute for the Networked Society - The German Internet Institute. - 2510-7666.
-
Konferensbidrag (refereegranskat)
|
|
| 7. |
- Khojah, Ranim, et al.
(författare)
-
Evaluating N-best Calibration of Natural Language Understanding for Dialogue Systems
- 2022
-
Ingår i: Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2022, 07-09 September 2022, Edinburgh, UK. - Stroudsburg, PA : Association for Computational Linguistics. - 9781955917667
-
Konferensbidrag (refereegranskat)abstract
- A Natural Language Understanding (NLU) component can be used in a dialogue system to perform intent classification, returning an N-best list of hypotheses with corresponding confidence estimates. We perform an in-depth evaluation of 5 NLUs, focusing on confidence estimation. We measure and visualize calibration for the 10 best hypotheses on model level and rank level, and also measure classification performance. The results indicate a trade-off between calibration and performance. In particular, Rasa (with Sklearn classifier) had the best calibration but the lowest performance scores, while Watson Assistant had the best performance but a poor calibration.
|
|
| 8. |
- Kronlid, Fredrik, 1973, et al.
(författare)
-
Comparing system-driven and free dialogue in in-vehicle interaction
- 2011
-
Ingår i: Proceedings of Interspeech 2011. - 9781618392701
-
Konferensbidrag (refereegranskat)abstract
- It is widely held that a free, natural dialogue model is more efficient and less distracting than system-initiative, state based dialogue. This paper describes an evaluation of two systems - one using system-directed dialogue and one using a more “free” dialogue - focusing on distraction and efficiency. The level of distraction is measured using an automotive industry standard test (LCT), and the efficiency is measured by counting the number of completed tasks. The efficiency is increased by 42 % using the free, natural dialogue model while the LCT results are unclear. Using a free dialogue model increases the efficiency and reduces the distraction in some cases.
|
|
| 9. |
- Larsson, Staffan, 1969, et al.
(författare)
-
Adding a Speech Cursor to a Multimodal Dialogue System
- 2011
-
Ingår i: Proceedings of Interspeech 2011. - 9781618392701
-
Konferensbidrag (refereegranskat)abstract
- This paper describes an in-vehicle dialogue system demonstrating a novel combination of flexible multimodal menu-based dialogue and a "speech cursor" which enables menu navigation as well as browsing long list using haptic input and spoken output.
|
|
| 10. |
- Larsson, Staffan, 1969, et al.
(författare)
-
Domain-specific and general syntax and semantics in the Talkamatic Dialogue Manager
- 2015
-
Ingår i: CSSP '15 (Colloque de Syntaxe et Sémantique à Paris), October 8-10, 2015, Paris.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- We have presented a design philosophy for dialogue system development, where domain-specific domain knowledge is clearly separated from the logic for generic dialogue capabilities. We hope that this provide a useful illustration of how one may approach the division of labour between general and domain specific syntax, semantics and pragmatics.
|
|
