Start Submission Become a Reviewer

Reading: Can Electronic Health Records Be Used for Population Health Surveillance? Validating Populat...

Download

A- A+
dyslexia friendly

Empirical research

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data

Authors:

Katharine H. McVeigh ,

NYC Department of Health and Mental Hygiene
About Katharine H.
PhD, MPH
X close

Remle Newton-Dame,

NYC Department of Health and Mental Hygiene (at the time of the study)
About Remle
MPH
X close

Pui Ying Chan,

NYC Department of Health and Mental Hygiene
About Pui Ying
MPH
X close

Lorna E. Thorpe,

NYU School of Medicine, Department of Population Health
About Lorna E.
PhD
X close

Lauren Schreibstein,

NYC Department of Health and Mental Hygiene
About Lauren
MS
X close

Kathleen S. Tatem,

NYC Department of Health and Mental Hygiene
X close

Claudia Chernov,

NYC Department of Health and Mental Hygiene
About Claudia
MPH
X close

Elizabeth Lurie-Moroni,

NYC Department of Health and Mental Hygiene
X close

Sharon E. Perlman

NYC Department of Health and Mental Hygiene
About Sharon E.
MPH
X close

Abstract

Introduction: Electronic health records (EHRs) offer potential for population health surveillance but EHR-based surveillance measures require validation prior to use. We assessed the validity of obesity, smoking, depression, and influenza vaccination indicators from a new EHR surveillance system, the New York City (NYC) Macroscope. This report is the second in a 3-part series describing the development and validation of the NYC Macroscope. The first report describes in detail the infrastructure underlying the NYC Macroscope; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. This second report, which addresses concerns related to sampling bias and data quality, describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods described in this report to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia.

Methods: NYC Macroscope prevalence estimates, overall and stratified by sex and age group, were compared to reference survey estimates for adult New Yorkers who reported visiting a doctor in the past year. Agreement was evaluated against 5 a priori criteria. Sensitivity and specificity were assessed by examining individual EHR records in a subsample of 48 survey participants.

Results: Among adult New Yorkers in care, the NYC Macroscope prevalence estimate for smoking (15.2%) fell between estimates from NYC HANES (17.7 %) and CHS (14.9%) and met all 5 a priori criteria. The NYC Macroscope obesity prevalence estimate (27.8%) also fell between the NYC HANES (31.3%) and CHS (24.7%) estimates, but met only 3 a priori criteria. Sensitivity and specificity exceeded 0.90 for both the smoking and obesity indicators. The NYC Macroscope estimates of depression and influenza vaccination prevalence were more than 10 percentage points lower than the estimates from either reference survey. While specificity was > 0.90 for both of these indicators, sensitivity was < 0.70.

Discussion: Through this work we have demonstrated that EHR data from a convenience sample of providers can produce acceptable estimates of smoking and obesity prevalence among adult New Yorkers in care; gained a better understanding of the challenges involved in estimating depression prevalence from EHRs; and identified areas for additional research regarding estimation of influenza vaccination prevalence. We have also shared lessons learned about how EHR indicators should be constructed and offer methodologic suggestions for validating them.

Conclusions: This work adds to a rapidly emerging body of literature about how to define, collect and interpret EHR-based surveillance measures and may help guide other jurisdictions.

How to Cite: McVeigh KH, Newton-Dame R, Chan PY, Thorpe LE, Schreibstein L, Tatem KS, et al.. Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data. eGEMs (Generating Evidence & Methods to improve patient outcomes). 2016;4(1):27. DOI: http://doi.org/10.13063/2327-9214.1267
4
Views
2
Downloads
Published on 15 Dec 2016.
Peer Reviewed

Downloads

  • PDF (EN)