Validation of a natural language processing algorithm using national reporting data to improve identification of anesthesia-related ADVerse evENTs: The “ADVENTURE” study
Authors' conclusion
This unsupervised ML method provides an accurate, automated, AI-supported search algorithm that ranks and helps to understand complex risk patterns and has greater speed, precision, and clarity when compared to manual human data extraction.
Study context
- Population
- AE reports by clinicians and healthcare systems to the French National Health accreditor (HAS), January 1, 2009, to December 31, 2020AE reports
- Participants
- 9,559 reports
- Comparison
- Natural language processing (NLP)/machine learning (ML) model categorisation of AE reports vs expert anesthesiologist validation
- Outcome
- Categorisation of reported AE
Methodological detailsmachine learning, natural language processing, national reporting data, validation by independent expert anesthesiologists, 135,000 unique de-identified AE reports labelling validated
- Machine learning
- Natural language processing
- National reporting data
- Validation by independent expert anesthesiologists
- 135,000 unique de-identified AE reports labelling validated
Performance
As reported in the abstract
| Test or model | Value | Sensitivity, difficult intubation | Specificity, difficult intubation | Sensitivity, medication error | Specificity, medication error |
|---|---|---|---|---|---|
| Artificial Intelligence (AI) model | 88% | 70.9% | 96.6% | 43.2% | 98.9% |
Artificial Intelligence (AI) model
Value88%
Sensitivity, difficult intubation70.9%
Specificity, difficult intubation96.6%
Sensitivity, medication error43.2%
Specificity, medication error98.9%
Further findings
Most frequent AE types
- Difficult orotracheal intubation
- 16.9%
- Medication error
- 10.5%
- Post-induction hypotension
- 6.9%
Original publication
Validation of a natural language processing algorithm using national reporting data to improve identification of anesthesia-related ADVerse evENTs: The “ADVENTURE” study. Anaesth Crit Care Pain Med. Published 6 May 2024. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →