A comparison of the American Board of Anesthesiology's in-person and virtual objective structured clinical examinations
Authors' conclusion
Our retrospective analyses of first-time OSCEs found comparable candidate performance and examiner grading severity between the in-person and virtual formats, despite the virtual scenarios being more difficult than the in-person scenarios.
Study context
- Population
- First-time candidates of the American Board of Anesthesiology's Objective Structured Clinical Examination (OSCE) for initial certificationfirst-time candidates
- Participants
- 2,934 vs 3,235
- Exposure
- Virtual vs in-person OSCE
- Virtualn = 2,934
- In-personn = 3,235reference
- Outcome
- Candidate performance, examiner grading severity and scenario difficulty (logits)
Methodological detailsMany-Facet Rasch Model, equated by common examiners and common scenarios
- Many-Facet Rasch Model
- Equated by common examiners and common scenarios
Main finding
As reported in the abstract
| Group | Value | Scenario difficulty, n scenarios | Examiner severity, n examiners | Candidate performance |
|---|---|---|---|---|
| Virtual | mean 0.21 (SD 0.38) logits | 35 | ||
| In-person | mean 0.00 (SD 0.69) logits | 93 | ||
| In-person | mean −0.01 (SD 0.82) | 390 | ||
| Virtual | mean −0.02 (SD 0.93) | 304 | ||
| In-person | mean 2.19 (SD 0.93) | 2.19 | ||
| Virtual | mean 2.18 (SD 0.92) | 2.18 |
Value
Virtualmean 0.21 (SD 0.38) logits
In-personmean 0.00 (SD 0.69) logits
In-personmean −0.01 (SD 0.82)
Virtualmean −0.02 (SD 0.93)
In-personmean 2.19 (SD 0.93)
Virtualmean 2.18 (SD 0.92)
Scenario difficulty, n scenarios
Virtual35
In-person93
In-person— not stated in the abstract
Virtual— not stated in the abstract
In-person— not stated in the abstract
Virtual— not stated in the abstract
Examiner severity, n examiners
Virtual— not stated in the abstract
In-person— not stated in the abstract
In-person390
Virtual304
In-person— not stated in the abstract
Virtual— not stated in the abstract
Candidate performance
Virtual— not stated in the abstract
In-person— not stated in the abstract
In-person— not stated in the abstract
Virtual— not stated in the abstract
In-person2.19
Virtual2.18
- Virtual vs In-person: P = 0.01 — scenario difficulty; virtual more difficult; Welch's partially overlapping samples t-test
- In-person vs Virtual: P = 0.81 — examiner severity; no statistically significant difference
- In-person vs Virtual: P = 0.83 — candidate performance; no statistically significant difference
Observational data: associations, not effects.
Further findings
Candidate characteristics
- Comparable in age, sex, race/ethnicity and US medical school graduation
Original publication
A comparison of the American Board of Anesthesiology's in-person and virtual objective structured clinical examinations. J Clin Anesth. Published 19 September 2023. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →