Table 4 Comparison of scale validations between the SHORT, the DASH and the OSAD
From: A new tool for assessing short debriefings after immersive simulation: validity of the SHORT scale
Rating scales | SHORT | DASH | OSAD |
|---|---|---|---|
Acronym signification | Simulation in Healthcare retrOaction Rating Tool | Debriefing Assessment for Simulation in Healthcare | Objective Structured Assessment of Debriefing |
Specifically-aimed aimed debriefing length | Short (10–15 min) | Undefined length | Long (30–45 min) |
Countries of origin | Canada, France | USA, Canada | UK, USA, Australia |
Targeted educational population | General healthcare practitioners | General healthcare practitioners | Surgeons |
Summary of the methodology of the scale design and validation | 1. Existing literature review 2. Focus group to identify needs 3. Scale elaboration with experts 4. Pre-test on 2 debriefings 5. Rating of recorded debriefings 6. Assessment of reliability and validity | 1. Existing Literature review 2. Iterative process of theory elaboration 3. Scale elaboration with experts 4. 5 h-webinars with international healthcare educators 5. Assessment of reliability and validity | 1. Existing literature review 2. Semi-structured interviews 3. Scale elaboration with experts 4 Rating of recorded debriefings 5. Retest with a new rating of randomized recorded sessions 6 months later 6. Assessment of reliability and validity |
Approach to validity and reliability measurement | Evidence of validity (test content, internal structure, testing consequences and response process) and reliability with generalizability theory: sources of variance for the scores, G and D studies with generalizability coefficients φ and φ-λ3 | Reliability by interrater reliability (intraclass correlation coefficients) and internal consistency with Cronbach α. Validity by 1-way repeated-measures of variance | Reliability by inter-rater intraclass correlation coefficient (ICC) and test–retest ICC. Validity by face and content validation |
Number of raters | 2 | 114 | 2 |
Only expert raters assessing the scale? | Yes | No | Yes |
Other raters assessing the scale than the authors? | No | Yes | No |
Number of rated debriefings for each rater | 22 | 3 | 20 |
Mean debriefings’ length | 10:35 | Unknown | Unknown |
Type of scale | Weighted, criterion referenced behaviorally anchored rating scale | 6-element, unweighted, criterion referenced behaviorally anchored rating scale (BARS) | 5-point Likert rating scale |
Number of items to be rated in the scale | 6 | 6 | 8 |
Anchors to help rating each item? | Yes | Yes | Yes |
Detailed notice for scale use | Yes | Yes | Yes |
Estimated mean time to complete the scale | 2 min | Unknown | 5 min |
Available translations of the scale | English, French | English, French, Japanese, German, Spanish | English |
Broad scale testing before publishing | No | Yes | No |