Reading: Analytical Methods for a Learning Health System: 2. Design of Observational Studies


A- A+
Alt. Display


Analytical Methods for a Learning Health System: 2. Design of Observational Studies


Michael Stoto ,

Georgetown University
About Michael
X close

Michael Oakes,

University of Minnesota
About Michael
X close

Elizabeth Stuart,

Johns Hopkins University
About Elizabeth
X close

Elisa L. Priest,

Baylor Health Care System
About Elisa L.
X close

Lucy Savitz

Kaiser Permanente Center for Health Research
About Lucy
X close


The second paper in a series on how learning health systems can use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning, this review summarizes study design approaches, including choosing appropriate data sources, and methods for design and analysis of natural and quasi-experiments.

The primary strength of study design approaches described in this section is that they study the impact of a deliberate intervention in real-world settings, which is critical for external validity. These evaluation designs address estimating the counterfactual – what would have happened if the intervention had not been implemented. At the individual level, epidemiologic designs focus on identifying situations in which bias is minimized. Natural and quasi-experiments focus on situations where the change in assignment breaks the usual links that could lead to confounding, reverse causation, and so forth. And because these observational studies typically use data gathered for patient management or administrative purposes, the possibility of observation bias is minimized. The disadvantages are that one cannot necessarily attribute the effect to the intervention (as opposed to other things that might have changed), and the results do not indicate what about the intervention made a difference.

Because they cannot rely on randomization to establish causality, program evaluation methods demand a more careful consideration of the “theory” of the intervention and how it is expected to play out. A logic model describing this theory can help to design appropriate comparisons, account for all influential variables in a model, and help to ensure that evaluation studies focus on the critical intermediate and long-term outcomes as well as possible confounders.
How to Cite: Stoto M, Oakes M, Stuart E, Priest EL, Savitz L. Analytical Methods for a Learning Health System: 2. Design of Observational Studies. eGEMs (Generating Evidence & Methods to improve patient outcomes). 2017;5(1):29. DOI:
Published on 07 Dec 2017.
Peer Reviewed


  • PDF (EN)

  • XML (EN)

    comments powered by Disqus