Start Submission Become a Reviewer

Reading: Performance of an NLP Tool to extract PFT reports from Structured and Semi-Structured VA data

Download

A- A+
dyslexia friendly

Case study

Performance of an NLP Tool to extract PFT reports from Structured and Semi-Structured VA data

Authors:

Brian C. Sauer ,

Barbara E. Jones,

Gary Globe,

Jianwei Leng,

Chao-Chin Lu,

Tao He,

Chia-Chen Teng,

Patrick Sullivan,

Qing Zeng

Abstract

Introduction/Objective: Pulmonary function tests (PFTs) are objective estimates of lung function, but are not reliably stored within the Veteran Health Affairs data systems as structured data. The aim of this study was to validate the natural language processing (NLP) tool we developed—which extracts spirometric values and responses to bronchodilator administration—against expert review, and to estimate the number of additional spirometric tests identified beyond the structured data.

Methods: All patients at seven Veteran Affairs Medical Centers with a diagnostic code for asthma Jan 1, 2006–Dec 31, 2012 were included. Evidence of spirometry with a bronchodilator challenge (BDC) was extracted from structured data as well as clinical documents. NLP’s performance was compared against a human reference standard using a random sample of 1,001 documents.

Results: In the validation set NLP demonstrated a precision of 98.9 percent [d1] (95 percent confidence intervals (CI): 93.9 percent, 99.7 percent), recall of 97.8 percent (95 percent CI: 92.2 percent, 99.7 percent), and an F-measure of 98.3 percent for the forced vital capacity pre- and post pairs and precision of 100 percent (95 percent CI: 96.6 percent, 100 percent), recall of 100 percent (95 percent CI: 96.6 percent, 100 percent), and an F-measure of 100 percent for the forced expiratory volume in one second pre- and post pairs for bronchodilator administration. Application of the NLP increased the proportion identified with complete bronchodilator challenge by 25 percent.

Discussion/Conclusion: This technology can improve identification of PFTs for epidemiologic research. Caution must be taken in assuming that a single domain of clinical data can completely capture the scope of a disease, treatment, or clinical test.

Key words: asthma, bronchodilator challenge, natural language processing, pulmonary function

How to Cite: Sauer BC, Jones BE, Globe G, Leng J, Lu C-C, He T, et al.. Performance of an NLP Tool to extract PFT reports from Structured and Semi-Structured VA data. eGEMs (Generating Evidence & Methods to improve patient outcomes). 2016;4(1):10. DOI: http://doi.org/10.13063/2327-9214.1217
2
Views
Published on 01 Jun 2016.
Peer Reviewed

Downloads

  • PDF (EN)