Reading: Calculating Power by Bootstrap, with an Application to Cluster-randomized Trials


A- A+
Alt. Display

Empirical research

Calculating Power by Bootstrap, with an Application to Cluster-randomized Trials


Ken Kleinman ,

University of Massachusetts Amherst, School of Public Health and Health Sciences
X close

Susan S. Huang

University of California, Irvine
X close


Background: A key requirement for a useful power calculation is that the calculation mimic the data analysis that will be performed on the actual data, once it is observed. Close approximations may be difficult to achieve using analytic solutions, however, and thus Monte Carlo approaches, including both simulation and bootstrap resampling, are often attractive. One setting in which this is particularly true is cluster-randomized trial designs. However, Monte Carlo approaches are useful in many additional settings as well. Calculating power for cluster-randomized trials using analytic or simulation-based methods is frequently unsatisfactory due to the complexity of the data analysis methods to be employed and to the sparseness of data to inform the choice of important parameters in these methods.

Methods: We propose that among Monte Carlo methods, bootstrap approaches are most likely to generate data similar to the observed data. In bootstrap approaches, real data are re-sampled to build complete data sets based on real data that resemble the data for the intended analyses. In contrast, simulation methods would use the real data to estimate parameters for the data and then simulate data using these parameters. Means of implementing bootstrap power calculation are described.

Results: We demonstrate bootstrap power calculation for a cluster-randomized trial with a censored survival outcome and a baseline observation period.

Conclusions: Bootstrap power calculation is a natural application of resampling methods. It provides a relatively simple solution to power calculation that is likely to be more accurate than analytic solutions or simulation-based calculations, in the sense that the bootstrap approach does not rely on the assumptions inherent in analytic calculations. It has several important strengths. Notably, it is simple to achieve great fidelity to the proposed data analysis method and there is no requirement for parameter estimates, or estimates of their variability, from outside settings. So, for example, for cluster-randomized trials, power calculations need not depend on intracluster correlation coefficient estimates from outside studies. In contrast, bootstrap power calculation requires initial data resembling data to be used in the planned study. We are not aware of bootstrap power calculation being previously proposed or explored for cluster-randomized trials, but it can also be applied for other study designs. We show with a simulation study that bootstrap power calculation can replicate analytic power in cases where analytic power can be accurately calculated. We also demonstrate power calculations for correlated censored survival outcomes in a cluster randomized trial setting, for which we are unaware of an analytic alternative. The method can easily be used when preliminary data is available, as is likely to be the case when research is performed in health delivery systems or other settings where electronic medical records can be obtained.

How to Cite: Kleinman K, Huang SS. Calculating Power by Bootstrap, with an Application to Cluster-randomized Trials. eGEMs (Generating Evidence & Methods to improve patient outcomes). 2017;4(1):32. DOI:
Published on 09 Feb 2017.
Peer Reviewed


  • PDF (EN)

    comments powered by Disqus