| 1. |
- Georgsson, Mattias
(författare)
-
A Modified User-Oriented Heuristic Evaluation of a Mobile Health System for Diabetes Self-management Support
- 2016
-
Ingår i: Computers, Informatics, Nursing. - : Lippincott Williams & Wilkins. - 1538-2931 .- 1538-9774. ; 34:2, s. 77-84
-
Tidskriftsartikel (refereegranskat)abstract
- Mobile health platforms offer significant opportunities for improving diabetic self-care, but only if adequate usability exists. Expert evaluations such as heuristic evaluation can provide distinct usability information about systems. The purpose of this study was to complete a usability evaluation of a mobile health system for diabetes patients using a modified heuristic evaluation technique of (1) dual-domain experts (healthcare professionals, usability experts), (2) validated scenarios and user tasks related to patients' self-care, and (3) in-depth severity factor ratings. Experts identified 129 usability problems with 274 heuristic violations for the system. The categories Consistency and Standards dominated at 24.1% (n = 66), followed by Match Between System and Real World at 22.3% (n = 61). Average severity ratings across system views were 2.8 (of 4), with 9.3% (n = 12) rated as catastrophic and 53.5% (n = 69) as major. The large volume of violations with severe ratings indicated clear priorities for redesign. The modified heuristic approach allowed evaluators to identify unique and important issues, including ones related to self-management and patient safety. This article provides a template for one type of expert evaluation adding to the informaticists' toolbox when needing to conduct a fast, resource-efficient and user-oriented heuristic evaluation.
|
|
| 2. |
|
|
| 3. |
- Georgsson, Mattias, 1969-, et al.
(författare)
-
Employing a user-centered cognitive walkthrough to evaluate a mHealth diabetes self-management application : A case study and beginning method validation
- 2019
-
Ingår i: Journal of Biomedical Informatics. - : Elsevier. - 1532-0464 .- 1532-0480. ; 91
-
Tidskriftsartikel (refereegranskat)abstract
- Introduction: Self-management of chronic diseases using mobile health (mHealth) systems and applications is becoming common. Current evaluation methods such as formal usability testing can be very costly and time-consuming; others may be more efficient but lack a user focus. We propose an enhanced cognitive walkthrough (CW) method, the user-centered CW (UC-CW), to address identified deficiencies in the original technique and perform a beginning validation with think aloud protocol (TA) to assess its effectiveness, efficiency and user acceptance in a case study with diabetes patient users on a mHealth self-management application. Materials and methods: A total of 12 diabetes patients at University of Utah Health, USA, were divided into UC-CW and think aloud (TA) groups. The UC-CW method included: making the user the main evaluator for detecting usability problems, having a dual domain facilitator, and using three other improved processes: validated task development, higher level tasks and a streamlined evaluation process. Users interacted with the same mHealth application for both methods. Post-evaluation assessments included the NASA RTLX instrument and a set of brief interview questions. Results: Participants had similar demographic characteristics. A total of 26 usability problems were identified with the UC-CW and 20 with TA. Both methods produced similar ratings: severity across all views (UC-CW = 2.7 and TA = 2.6), numbers of problems in the same views (Main View [UC-CW = 11, TA = 10], Carbohydrate Entry View [UC-CW = 4, TA = 3] and List View [UC-CW = 3, TA = 3]) with similar heuristic violations (Match Between the System and Real World [UC-CW = 19, TA = 16], Consistency and Standards [UC-CW = 17, TA = 15], and Recognition Rather than Recall [UC-CW = 13, TA = 10]). Both methods converged on eight usability problems, but the UC-CW group detected five critical issues while the TA group identified two. The UC-CW group identified needed personalized features for patients’ disease needs not identified with TA. UC-CW was more efficient on average time per identified usability problem and on the total evaluation process with patients. NASA RTLX scores indicated that participants experienced the UC-CW half as cognitively demanding. Common themes from interviews indicated the UC-CW as enjoyable and easy to perform while TA was considered somewhat awkward and more cognitively challenging. Conclusions: UC-CW was effective for finding severe, recurring usability problems and it highlighted the need for personalized user features. The method was also efficient and had high user acceptance. These results indicate UC-CW's utility and user acceptance in evaluating a mHealth self-management application. It provides an additional usability evaluation technique for researchers. © 2019
|
|
| 4. |
- Georgsson, Mattias, et al.
(författare)
-
Patients' Perceptions and Experiences of a mHealth Diabetes Self-management System
- 2017
-
Ingår i: Computers, Informatics, Nursing. - : LIPPINCOTT WILLIAMS & WILKINS. - 1538-2931 .- 1538-9774. ; 35:3, s. 122-130
-
Tidskriftsartikel (refereegranskat)abstract
- Chronic diseases, including diabetes, constitute a substantial disease burden around the world. Mobile self-management systems now play a significant and increasingly important role in patients' disease management. Yet, patients' perceptions of these systems after longer-term use are largely unexplored. A random sample of 10 diabetes patients was assessed immediately after they exited a larger, 6-month randomized controlled trial on the use of a mHealth system called Care4Life. This descriptive, exploratory study assessed patients' perceptions and experiences of mHealth using a questionnaire and semistructured interview whose development was guided by the Technology Acceptance Model. Results indicated that patients saw clear benefits in using the technology and had favorable behavioral disease outcomes after using Care4Life. Suggestions for improving the system were highly individual despite the apparent homogeneity of the patient group. The study begins to fill the gap about the longer-term use of mHealth systems in chronic disease management and reflects the significance of individual needs for mHealth systems.
|
|
| 5. |
- Georgsson, Mattias
(författare)
-
Quantifying usability : an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics
- 2016
-
Ingår i: JAMIA Journal of the American Medical Informatics Association. - : Oxford University Press. - 1067-5027 .- 1527-974X. ; 23:1, s. 5-11
-
Tidskriftsartikel (refereegranskat)abstract
- Objective Mobile health (mHealth) systems are becoming more common for chronic disease management, but usability studies are still needed on patients' perspectives and mHealth interaction performance. This deficiency is addressed by our quantitative usability study of a mHealth diabetes system evaluating patients' task performance, satisfaction, and the relationship of these measures to user characteristics. Materials and Methods We used metrics in the International Organization for Standardization (ISO) 9241-11 standard. After standardized training, 10 patients performed representative tasks and were assessed on individual task success, errors, efficiency (time on task), satisfaction (System Usability Scale [SUS]) and user characteristics. Results Tasks of exporting and correcting values proved the most difficult, had the most errors, the lowest task success rates, and consumed the longest times on task. The average SUS satisfaction score was 80.5, indicating good but not excellent system usability. Data trends showed males were more successful in task completion, and younger participants had higher performance scores. Educational level did not influence performance, but a more recent diabetes diagnosis did. Patients with more experience in information technology (IT) also had higher performance rates. Discussion Difficult task performance indicated areas for redesign. Our methods can assist others in identifying areas in need of improvement. Data about user background and IT skills also showed how user characteristics influence performance and can provide future considerations for targeted mHealth designs. Conclusion Using the ISO 9241-11 usability standard, the SUS instrument for satisfaction and measuring user characteristics provided objective measures of patients' experienced usability. These could serve as an exemplar for standardized, quantitative methods for usability studies on mHealth systems.
|
|
| 6. |
- Georgsson, Mattias, et al.
(författare)
-
Revisiting Heuristic Evaluation Methods to Improve the Reliability of Findings
- 2014
-
Konferensbidrag (refereegranskat)abstract
- The heuristic evaluation (HE) method is one of the most common in the suite of tools for usability evaluations because it is a fast, inexpensive and resource-efficient process in relation to the many usability issues it generates. The method emphasizes completely independent initial expert evaluations. Inter-rater reliability and agreement coefficients are not calculated. The variability across evaluators, even dual domain experts, can be significant as is seen in the case study here. The implications of this wide variability mean that results are unique to each HE, results are not readily reproducible and HE research on usability is not yet creating a uniform body of knowledge. We offer recommendations to improve the science by incorporating selected techniques from qualitative research: calculating inter-rater reliability and agreement scores, creating a codebook to define concepts/categories and offering crucial information about raters' backgrounds, agreement techniques and the evaluation setting.
|
|
| 7. |
- Georgsson, Mattias
(författare)
-
Toward Patient-centered, Standardized, and Reproducible Approaches of Evaluating the Usability of mHealth Chronic Disease Self-management Systems for Diabetes
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Diabetes is a chronic disease affecting 422 million patients worldwide according to World Health Organization data with 30.3 million in the United States and 64 million in Europe. The prevalence speaks to the need for improved ways to support patients in disease self-management. mHealth solutions are increasingly used for this; however, usability is a current challenge affecting patients’ mHealth use. Recent literature emphasizes an increased focus on patient-centeredness in diabetes care, user-centeredness in chronic disease mHealth system design and standardized, systematic approaches for usability evaluation. The aim of this thesis and its individual studies was to incorporate these foci into the evaluation of two mobile health self-management systems for diabetes.Study I used ISO standard 9241-11 to examine the relationship between selected group characteristics of diabetes patients on specific interaction outcomes to quantitatively identify needed system modifications. Study II utilized a multi-method design to assess diabetes patients’ mHealth usage and combined two novel analytic methods to structure and analyze results. Study III used a modified, user-oriented heuristic evaluation (HE) method, validated tasks and in-depth severity factor ratings to identify critical problems from patients’ point of view. By developing and employing a modified, user-centered cognitive walkthrough method (UC-CW), study IV assessed its effectiveness and efficiency in finding relevant usability problems for users as well as patients’ acceptance. The modified CW was validated against the golden-standard user test with Think Aloud.Study I emphasized the importance of considering user characteristics in mHealth performance as these influenced interaction outcomes. All patients had difficulties with multiple-step tasks. Patients more recently diagnosed were able to perform tasks more successfully, with fewer errors and at faster times and had higher satisfaction scores; similar outcomes to the more experienced users. Educational level did not, however, seem to influence performance. In study II, the usability test with Think-Aloud (TA), in-depth interviews and questionnaires contributed to 19 consolidated issues, and triangulated on 5 critical usability problems for users. The combined analysis methods resulted in structured, categorized descriptions to aid in problem-solving. In Study III, the disease-related, critical information deficiencies found by expert evaluators using the modified, structured method also converged on and highlighted potentially adverse user concerns. Study IV demonstrated that the UC-CW found more critical user problems compared to the user test with TA despite both methods producing similar major average severity ratings and violations of heuristic categories. The modified method was more efficient per detected problem and experienced as less cognitively demanding and with a higher ease of use.These studies offer different approaches that include patient-centered, efficient and user-acceptable methods and method modifications to detect critical usability issues for users. Importantly, improved mHealth designs for users could mean improvement in interactions, interaction performance, increased adoption, and long-term perhaps even increased adherence to interventions for chronic conditions.
|
|
