| 1. |
- Booth, Frederick, et al.
(author)
-
A Mental Health and Well-Being Chatbot : User Event Log Analysis
- 2023
-
In: JMIR mhealth and uhealth. - : JMIR Publications. - 2291-5222. ; 11
-
Journal article (peer-reviewed)abstract
- Background: Conversational user interfaces, or chatbots, are becoming more popular in the realm of digital health and well-being. While many studies focus on measuring the cause or effect of a digital intervention on people’s health and well-being (outcomes), there is a need to understand how users really engage and use a digital intervention in the real world.Objective: In this study, we examine the user logs of a mental well-being chatbot called ChatPal, which is based on the concept of positive psychology. The aim of this research is to analyze the log data from the chatbot to provide insight into usage patterns, the different types of users using clustering, and associations between the usage of the app’s features.Methods: Log data from ChatPal was analyzed to explore usage. A number of user characteristics including user tenure, unique days, mood logs recorded, conversations accessed, and total number of interactions were used with k-means clustering to identify user archetypes. Association rule mining was used to explore links between conversations.Results: ChatPal log data revealed 579 individuals older than 18 years used the app with most users being female (n=387, 67%). User interactions peaked around breakfast, lunchtime, and early evening. Clustering revealed 3 groups including “abandoning users” (n=473), “sporadic users” (n=93), and “frequent transient users” (n=13). Each cluster had distinct usage characteristics, and the features were significantly different (P<.001) across each group. While all conversations within the chatbot were accessed at least once by users, the “treat yourself like a friend” conversation was the most popular, which was accessed by 29% (n=168) of users. However, only 11.7% (n=68) of users repeated this exercise more than once. Analysis of transitions between conversations revealed strong links between “treat yourself like a friend,” “soothing touch,” and “thoughts diary” among others. Association rule mining confirmed these 3 conversations as having the strongest linkages and suggested other associations between the co-use of chatbot features.Conclusions: This study has provided insight into the types of people using the ChatPal chatbot, patterns of use, and associations between the usage of the app’s features, which can be used to further develop the app by considering the features most accessed by users.
|
|
| 2. |
- Boyd, Kyle, et al.
(author)
-
Usability testing and trust analysis of a mental health and wellbeing chatbot
- 2022
-
In: Proceedings of the 33rd European Conference on Cognitive Ergonomics (ECCE 2022). - New York, NY, USA : Association for Computing Machinery.
-
Conference paper (peer-reviewed)abstract
- Mental health chatbots are particularly useful for those who are isolated and may have difficulty attending services or for those who are reluctant to speak to a professional. In this study, the usability and trust of a chatbot known as ’ChatPal’ has been assessed. ChatPal has been developed by an interdisciplinary team encompassing health service providers, local authorities, charities and universities to promote positive mental wellbeing among individuals in rural areas across Europe. This study employed a usability test protocol to recruit representative users to complete a set of tasks using the ChatPal chatbot. Usability issues were assessed along with trust and users’ satisfaction on the System Usability Scale and the Chatbot Usability Questionnaire. The study shows the usability issues and trust with a mental health chatbot and highlights recommendations for improvement.
|
|
| 3. |
- Potts, Courtney, et al.
(author)
-
A Multilingual Digital Mental Health and Well-Being Chatbot (ChatPal): Pre-Post Multicenter Intervention Study
- 2023
-
In: Journal of Medical Internet Research. - : JMIR Publications. - 1438-8871. ; 25
-
Journal article (peer-reviewed)abstract
- Background: In recent years, advances in technology have led to an influx of mental health apps, in particular the development of mental health and well-being chatbots, which have already shown promise in terms of their efficacy, availability, and accessibility. The ChatPal chatbot was developed to promote positive mental well-being among citizens living in rural areas. ChatPal is a multilingual chatbot, available in English, Scottish Gaelic, Swedish, and Finnish, containing psychoeducational content and exercises such as mindfulness and breathing, mood logging, gratitude, and thought diaries.Objective: The primary objective of this study is to evaluate a multilingual mental health and well-being chatbot (ChatPal) to establish if it has an effect on mental well-being. Secondary objectives include investigating the characteristics of individuals that showed improvements in well-being along with those with worsening well-being and applying thematic analysis to user feedback.Methods: A pre-post intervention study was conducted where participants were recruited to use the intervention (ChatPal) for a 12-week period. Recruitment took place across 5 regions: Northern Ireland, Scotland, the Republic of Ireland, Sweden, and Finland. Outcome measures included the Short Warwick-Edinburgh Mental Well-Being Scale, the World Health Organization-Five Well-Being Index, and the Satisfaction with Life Scale, which were evaluated at baseline, midpoint, and end point. Written feedback was collected from participants and subjected to qualitative analysis to identify themes.Results: A total of 348 people were recruited to the study (n=254, 73% female; n=94, 27% male) aged between 18 and 73 (mean 30) years. The well-being scores of participants improved from baseline to midpoint and from baseline to end point; however, improvement in scores was not statistically significant on the Short Warwick-Edinburgh Mental Well-Being Scale (P=.42), the World Health Organization-Five Well-Being Index (P=.52), or the Satisfaction With Life Scale (P=.81). Individuals that had improved well-being scores (n=16) interacted more with the chatbot and were significantly younger compared to those whose well-being declined over the study (P=.03). Three themes were identified from user feedback, including “positive experiences,” “mixed or neutral experiences,” and “negative experiences.” Positive experiences included enjoying exercises provided by the chatbot, while most of the mixed, neutral, or negative experiences mentioned liking the chatbot overall, but there were some barriers, such as technical or performance errors, that needed to be overcome.Conclusions: Marginal improvements in mental well-being were seen in those who used ChatPal, albeit nonsignificant. We propose that the chatbot could be used along with other service offerings to complement different digital or face-to-face services, although further research should be carried out to confirm the effectiveness of this approach. Nonetheless, this paper highlights the need for blended service offerings in mental health care.
|
|
| 4. |
- Potts, Courtney, et al.
(author)
-
Insights and lessons learned from trialling a mental health chatbot in the wild
- 2021
-
In: 2021 IEEE Symposium on Computers and Communications (ISCC). - : IEEE. ; , s. 1-6
-
Conference paper (peer-reviewed)abstract
- This study reports on the development and ‘in the wild’ trialling of a chatbot (ChatPal) which promotes good mental wellbeing. A stakeholder-centered approach for design was adopted where end users, mental health professionals and service users were involved in the design which was centered around positive psychology. In the wild usage of the chatbot was investigated from Jul-20-Mar-21. Exploratory analyses of usage metrics were carried out using the event log data. User tenure, unique usage days, total chatbot interactions and average daily interactions were used in K-means clustering to identify user archetypes. The chatbot was used by a variety of age groups (18-65+) and genders, mainly those living in Ireland. K-means clustering identified three clusters: sporadic users (n=4), frequent transient users (n=38) and abandoning users (n=169) each with distinct usage characteristics. This study highlights the importance of event log data analysis for making improvements to the mental health chatbot.
|
|
| 5. |
- Sweeney, Colm, et al.
(author)
-
Can Chatbots Help Support a Person’s Mental Health? Perceptions and Views from Mental Healthcare Professionals and Experts
- 2021
-
In: ACM Transactions on Computing for Healthcare. - : ACM Digital Library. - 2691-1957 .- 2637-8051. ; 2:3
-
Journal article (peer-reviewed)abstract
- The objective of this study was to understand the attitudes of professionals who work in mental health regarding the use of conversational user interfaces, or chatbots, to support people’s mental health and wellbeing. This study involves an online survey to measure the awareness and attitudes of mental healthcare professionals and experts. The findings from this survey show that more than half of the participants in the survey agreed that there are benefits associated with mental healthcare chatbots (65%, p < 0.01). The perceived importance of chatbots was also relatively high (74%, p < 0.01), with more than three-quarters (79%, p < 0.01) of respondents agreeing that mental healthcare chatbots could help their clients better manage their own health, yet chatbots are overwhelmingly perceived as not adequately understanding or displaying human emotion (86%, p < 0.01). Even though the level of personal experience with chatbots among professionals and experts in mental health has been quite low, this study shows that where they have been used, the experience has been mostly satisfactory. This study has found that as years of experience increased, there was a corresponding increase in the belief that healthcare chatbots could help clients better manage their own mental health.
|
|
