Reliable and valid assessment of procedural skills in resuscitative endovascular balloon occlusion of the aorta

• BACKGROUND: Valid and reliable assessment of skills is essential for improved and evidence-based training concepts. In a recent study, we presented a novel tool to assess procedural skills in resuscitative endovascular balloon occlusion of the aorta (REBOA), REBOA-RATE, based on international expert consensus. Although expert consensus is a strong foundation, the performance of REBOA-RATE has not been explored. The study aimed to examine the reliability and validity of REBOA-RATE.METHODS: This was an experimental simulation-based study. We enrolled doctors with three levels of expertise to perform two REBOA procedures in a simulated scenario of out-of-hospital cardiac arrest. Procedures were video-recorded, and videos were blinded and randomized. Three clinical experts independently rated all procedures using REBOA-RATE. Data were analyzed using Messick's framework for validity evidence, including generalizability analysis of reliability and determination of a pass/fail standard.RESULTS: Forty-two doctors were enrolled: 16 novices, 13 anesthesiologists, and 13 endovascular experts. They all performed two procedures, yielding 84 procedures and 252 ratings. The REBOA-RATE assessment tool showed high internal consistency (Cronbach's alpha = 0.95) and excellent interrater reliability (intraclass correlation coefficient, 0.97). Assessment using one rater and three procedures could ensure overall reliability suitable for high-stakes testing (G-coefficient >0.80). Mean scores (SD) for the three groups in the second procedure were as follows: novices, 32% (24%); anesthesiologists, 55% (29%); endovascular experts, 93% (4%) (p < 0.001). The pass/fail standard was set at 81%, which all experts but no novices passed.CONCLUSION: Data strongly support the reliability and validity of REBOA-RATE, which successfully discriminated between all experience levels. The REBOA-RATE assessment tool requires minimal instruction, and one rater is sufficient for reliable assessment. Together, these are strong arguments for the use of REBOA-RATE to assess REBOA skills, allowing for competency-based training and certification concepts.

Ämnesord

MEDICIN OCH HÄLSOVETENSKAP  -- Klinisk medicin -- Kirurgi (hsv//swe)

MEDICAL AND HEALTH SCIENCES  -- Clinical Medicine -- Surgery (hsv//eng)

Nyckelord

Resuscitative endovascular balloon occlusion of the aorta

vascular access

assessment tool

skills

clinical competence

validation

validity evidence

simulation

clinical competence

Resuscitative endovascular balloon occlusion of the aorta

simulation

skills

validation

validity evidence

vascular access, assessment tool

Publikations- och innehållstyp

ref (ämneskategori)

art (ämneskategori)