Cardiothoracic & vascular anesthesia
Automated structured data extraction from intraoperative echocardiography reports using large language models
Authors' conclusion
A consensus-based LLM ensemble successfully generated structured data from unstructured text contained in intraoperative transoesophageal reports.
Study context
- Population
- Intraoperative TOE reports of adults undergoing cardiac surgery at two University of Pennsylvania Healthcare System hospitals; extraction of LV ejection fraction, RV systolic function and tricuspid regurgitation500 reports for testing
- Participants
- 600 participants
- Exposure
- Unanimous vs Supermajority vs Majority vs Plurality
- Outcome
- Raw accuracy, consensus accuracy, error rate and yield vs reference standard
Methodological detailstwo hospitals, large language model (LLM) ensembles (five open-source LLMs, four voting strategies), 600 reports randomly sampled from 7106 adult patients; 100 for prompt engineering, 500 for testing
- Two hospitals
- LLM ensembles (five open-source LLMs, four voting strategies)
- 600 reports randomly sampled from 7106 adult patients; 100 for prompt engineering, 500 for testing
Main finding
As reported in the abstract
Consensus accuracy, error rate, yield, raw accuracy (presurgical; postsurgical)
| Group | Consensus accuracy presurgical | Consensus accuracy postsurgical | Error rate presurgical | Error rate postsurgical | Yield presurgical | Yield postsurgical | Raw accuracy presurgical | Raw accuracy postsurgical |
|---|---|---|---|---|---|---|---|---|
| Unanimous | 99.4% | 97.9% | 0.6% | 2.1% | 81.7% | 80.5% | 81.2% | 78.9% |
| Plurality | 96.7% | 94.7% | 3.3% | 5.3% | 99.4% | 98.9% | 96.1% | 93.7% |
Consensus accuracy presurgical
Unanimous99.4%
Plurality96.7%
Consensus accuracy postsurgical
Unanimous97.9%
Plurality94.7%
Error rate presurgical
Unanimous0.6%
Plurality3.3%
Error rate postsurgical
Unanimous2.1%
Plurality5.3%
Yield presurgical
Unanimous81.7%
Plurality99.4%
Yield postsurgical
Unanimous80.5%
Plurality98.9%
Raw accuracy presurgical
Unanimous81.2%
Plurality96.1%
Raw accuracy postsurgical
Unanimous78.9%
Plurality93.7%
Observational data: associations, not effects.
Original publication
Automated structured data extraction from intraoperative echocardiography reports using large language models. Br J Anaesth. Published 3 March 2025. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →