Perioperative medicine & outcomes
Large Language Model Capabilities in Perioperative Risk Prediction and Prognostication
Authors' conclusion
Current general-domain large language models may assist clinicians in perioperative risk stratification on classification tasks but are inadequate for numerical duration predictions.
Study context
- Population
- Patients who had a surgery or procedure with anesthesia and ≥1 clinician-written preoperative note in the electronic health recordtask-specific datasets of 1000 cases
- Comparison
- Prompting strategies for GPT-4 Turbo: original notes, note summaries, few-shot and chain-of-thought prompting
- Outcome
- F1 score for binary and categorical outcomes; mean absolute error for numerical duration outcomes
Methodological detailsretrospective electronic health records data, single quaternary care center (3 academic hospitals), Data analyzed from November to December 2023
- Retrospective electronic health records data
- Single quaternary care center (3 academic hospitals)
- Data analyzed from November to December 2023
Performance
As reported in the abstract
Best predictive performance per task
- F1 score, American Society of Anesthesiologists Physical Status (ASA-PS)
- 0.50 (95% CI, 0.47-0.53)
- F1 score, hospital admission
- 0.64 (95% CI, 0.61-0.67)
- F1 score, intensive care unit (ICU) admission
- 0.81 (95% CI, 0.78-0.83)
- F1 score, unplanned admission (949 cases)
- 0.61 (95% CI, 0.58-0.64)
- F1 score, hospital mortality (576 cases)
- 0.86 (95% CI, 0.83-0.89)
- Mean absolute error, postanesthesia care unit (PACU) phase 1 duration
- 49 minutes (95% CI, 46-51 minutes)
- Mean absolute error, hospital duration
- 4.5 days (95% CI, 4.2-5.0 days)
- Mean absolute error, ICU duration
- 1.1 days (95% CI, 0.9-1.3 days)
Original publication
Large Language Model Capabilities in Perioperative Risk Prediction and Prognostication. JAMA Surg. Published 1 August 2024. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →