Next Article in Journal
Retrospective Study of Dupilumab Treatment for Moderate to Severe Atopic Dermatitis in Korea: Efficacy and Safety of Dupilumab in Real-World Practice
Previous Article in Journal
Disturbances of Continuous Sleep and Circadian Rhythms Account for Behavioral Difficulties in Children with Autism Spectrum Disorder
Open AccessArticle

Clinical Validation of a Deep Learning Algorithm for Detection of Pneumonia on Chest Radiographs in Emergency Department Patients with Acute Febrile Respiratory Illness

1
Department of Radiology, Armed Forces Goyang Hospital, 215, Hyeeum-ro, Deogyang-gu, Goyang-si, Gyeonggi-do 10271, Korea
2
Department of Internal Medicine, Armed Forces Goyang Hospital, 215, Hyeeum-ro, Deogyang-gu, Goyang-si, Gyeonggi-do 10271, Korea
3
Division of Infectious Diseases and Geographic Medicine, Stanford University School of Medicine, 291 Campus Drive, Stanford, CA 94305, USA
4
Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2020, 9(6), 1981; https://doi.org/10.3390/jcm9061981
Received: 18 June 2020 / Accepted: 22 June 2020 / Published: 24 June 2020
(This article belongs to the Section Emergency Medicine)
Early identification of pneumonia is essential in patients with acute febrile respiratory illness (FRI). We evaluated the performance and added value of a commercial deep learning (DL) algorithm in detecting pneumonia on chest radiographs (CRs) of patients visiting the emergency department (ED) with acute FRI. This single-centre, retrospective study included 377 consecutive patients who visited the ED and the resulting 387 CRs in August 2018–January 2019. The performance of a DL algorithm in detection of pneumonia on CRs was evaluated based on area under the receiver operating characteristics (AUROC) curves, sensitivity, specificity, negative predictive values (NPVs), and positive predictive values (PPVs). Three ED physicians independently reviewed CRs with observer performance test to detect pneumonia, which was re-evaluated with the algorithm eight weeks later. AUROC, sensitivity, and specificity measurements were compared between “DL algorithm” vs. “physicians-only” and between “physicians-only” vs. “physicians aided with the algorithm”. Among 377 patients, 83 (22.0%) had pneumonia. AUROC, sensitivity, specificity, PPV, and NPV of the algorithm for detection of pneumonia on CRs were 0.861, 58.3%, 94.4%, 74.2%, and 89.1%, respectively. For the detection of ‘visible pneumonia on CR’ (60 CRs from 59 patients), AUROC, sensitivity, specificity, PPV, and NPV were 0.940, 81.7%, 94.4%, 74.2%, and 96.3%, respectively. In the observer performance test, the algorithm performed better than the physicians for pneumonia (AUROC, 0.861 vs. 0.788, p = 0.017; specificity, 94.4% vs. 88.7%, p < 0.0001) and visible pneumonia (AUROC, 0.940 vs. 0.871, p = 0.007; sensitivity, 81.7% vs. 73.9%, p = 0.034; specificity, 94.4% vs. 88.7%, p < 0.0001). Detection of pneumonia (sensitivity, 82.2% vs. 53.2%, p = 0.008; specificity, 98.1% vs. 88.7%; p < 0.0001) and ‘visible pneumonia’ (sensitivity, 82.2% vs. 73.9%, p = 0.014; specificity, 98.1% vs. 88.7%, p < 0.0001) significantly improved when the algorithm was used by the physicians. Mean reading time for the physicians decreased from 165 to 101 min with the assistance of the algorithm. Thus, the DL algorithm showed a better diagnosis of pneumonia, particularly visible pneumonia on CR, and improved diagnosis by ED physicians in patients with acute FRI. View Full-Text
Keywords: acute febrile respiratory illness; emergency department; chest radiograph; artificial intelligence; deep learning algorithm acute febrile respiratory illness; emergency department; chest radiograph; artificial intelligence; deep learning algorithm
Show Figures

Figure 1

MDPI and ACS Style

Kim, J.H.; Kim, J.Y.; Kim, G.H.; Kang, D.; Kim, I.J.; Seo, J.; Andrews, J.R.; Park, C.M. Clinical Validation of a Deep Learning Algorithm for Detection of Pneumonia on Chest Radiographs in Emergency Department Patients with Acute Febrile Respiratory Illness. J. Clin. Med. 2020, 9, 1981.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop