Determine whether hospitals are increasing the duration of observation stays following index admission for heart failure to avoid potential payment penalties from the Hospital Readmission Reduction Program.

The Hospital Readmission Reduction Program applies a 30-day cutoff after which readmissions are no longer penalized. Given this seemingly arbitrary cutoff, we use regression discontinuity design, a quasi-experimental research design that can be used to make causal inferences.

The High Value Healthcare Collaborative includes member healthcare systems covering 57% of the nation’s hospital referral regions. We used Medicare claims data including all patients residing within these regions. The study included patients with index admissions for heart failure from January 1, 2012 to June 30, 2015 and a subsequent observation stay within 60 days. We excluded hospitals with fewer than 25 heart failure readmissions in a year or fewer than 5 observation stays in a year and patients with subsequent observation stays at a different hospital.

Overall, there was no discontinuity at the 30-day cutoff in the duration of observation stays, the percent of observation stays over 12 hours, or the percent of observation stays over 24 hours. In the sub-analysis, the discontinuity was significant for non-penalized.

The findings reveal evidence that the HRRP has resulted in an increase in the duration of observation stays for some non-penalized hospitals.

Due to their ability to eliminate bias and estimate causal treatment effects, randomized controlled trials (RCTs) are considered the gold standard in most published hierarchies or levels of evidence [

The challenges associated with RCTs make observational studies common. However, in most cases, these studies provide a lower quality of evidence and are rife with potential bias [

Established as part of the Affordable Care Act, the HRRP reduces Medicare prospective payments (up to 3 percent) for hospitals with more than expected 30-day readmissions [

Unsurprisingly, hospital leaders have also expressed concern about the HRRP, particularly regarding the size of penalties, the risk adjustment process, and hospitals’ inability to significantly impact patient adherence [

Given the challenges hospitals face in reducing readmissions and the financial pressure of the HRRP, there is concern that hospitals may seek other ways to avoid potential payment reduction penalties. Some have suggested that the HRRP creates a financial incentive to reduce even necessary readmissions, which could result in increased mortality [

Much of the work focused on observation stays uses observational, nonexperimental research designs that limit the ability to draw firm causal links. Additionally, while the decrease in readmissions exceeds the increase in observation stays, the effect on observation stays may not be insignificant. It is also possible that some hospitals work to shift patients to observation status more than others. In fact, hospital characteristics, such as ownership, size, and location, are among the most significant predictors of both the frequency and duration of observation stays among Medicare beneficiaries [

We focus on HF for several key reasons. First, HF affects 17 percent of the Medicare population over the age of 65 and the prevalence is increasing [

In this article, we use RD to estimate the impact of the HRRP on the duration of observation stays following hospitalization for HF. We hypothesize that the HRRP increases the average duration of observation stays for HF, particularly near the near day 30, where the potential to avoid a 30-day readmission increases. Also, we expect greater evidence of an impact of the HRRP among non-penalized hospitals (ERR ≤ 1) because they may have a more comprehensive strategy to reduce readmissions. Thus, we conduct a sub-group analysis comparing penalized and non-penalized hospitals.

This work was conducted through a partnership with the High Value Healthcare Collaborative (HVHC), a network of 12 learning health care systems across the United States. We included all Medicare patients in hospital referral regions (HRR) represented by HVHC members, 57 percent of the nation’s HRRs. We limited the analysis to patients with an index admission for heart failure from July 1, 2012 to June 30, 2015. The analysis was also limited to patients that had an observation stay between 6 and 55 days after discharge from the hospital. We excluded hospitals with fewer than 25 readmissions in a calendar year as they are not affected by the HRRP. We also excluded hospitals with fewer than 5 observation stays because we were interested in hospitals using observation stays frequently enough to potentially impact their ERR scores. Lastly, we excluded observation stays that occurred at a different hospital than the index admission. The new hospital would not necessarily be incentivized to use observation status to avoid readmissions. After exclusions, the sample consisted of 19,815 observation stays across the 3-year period. Overall, the dataset included 419 hospitals.

RD can be used to make causal inferences, even when randomization is not possible or practical [^{th} day following an index admission are likely not very different from patients readmitted on the 31^{st} day. As a result, the HRRP is a prime example of a policy that can be evaluated using RD.

While several statistical programs (e.g., Stata, R) include packages for conducting RD analysis, the analysis can also be conducted using simple regression equations. Equation 1 shows the general linear form of the RD equation. In Equation 1, _{1} is the slope of the line before the cutoff, _{2} is the slope of the line after the cutoff, and

In addition to functional form, the bandwidth of data used for the analysis can also impact RD results. Because RD is only valid for subjects with assignment variable scores near the cutoff value, the inclusion of data far from the cutoff value could bias results [

In addition to requiring a seemingly arbitrary cutoff, RD analysis typically involves testing two key assumptions. First, research subjects must be unable to self-sort [

We used the duration of observation stays in hours as our primary outcome measure. Secondary outcome measures included the percent of observation stays that were greater than 12 hours and 24 hours in duration. We conducted RD analysis with the three outcome variables using days from discharge as our assignment variable. We set the cutoff value to 30.5 days, corresponding to the HRRP. The analyses were conducted using linear regression analysis in Stata; we used the ‘sureg’ command for seemingly unrelated regressions to improve the efficiency of the standard errors. We conducted nonparametric RD in R using the ‘rdrobust’ and ‘rdd’ packages. We set alpha to 0.05 for hypothesis tests and applied the Benjamini-Hochberg (BH) procedure to correct the p-value limit for multiple tests [

Limiting the analysis to patients with an observation stay between 6 and 55 days after the index admission largely eliminated nonlinearities in the data. Likelihood ratio (LR) tests confirmed that linear models fit the data better than higher-order models for each outcome variable [

The results from the main RD analyses are shown in Table

Results from regression discontinuity analysis.

coef (t) | p-value | 95% CI(LL, UL) | BHp-value limit | |
---|---|---|---|---|

Effect of HRRP on all hospitals |
||||

1 – Average duration of observation stays | 0.465 | 0.149 | (–0.166, 1.095) | 0.067 |

2 – Percent of observation stays > 12 hrs | 0.366 | 0.159 | (–0.143, 0.876) | 0.083 |

3 – Percent of observation stays > 24 hrs | 0.236 | 0.208 | (–0.131, 0.604) | 0.100 |

Effect of HRRP on penalized hospitals (ERR > 1) |
||||

4 – Average duration of observation stays | –0.189 | 0.724 | (–1.237, 0.860) | 0.033 |

5 – Percent of observation stays > 12 hrs | –0.148 | 0.678 | (–0.847, 0.551) | 0.042 |

6 – Percent of observation stays > 24 hrs | –0.116 | 0.666 | (–0.643, 0.411) | 0.050 |

Effect of HRRP on non-penalized hospitals (ERR ≤ 1) |
||||

7 – Average duration of observation stays | 0.996 | 0.022* | (0.143, 1.850) | 0.025 |

8 – Percent of observation stays > 12 hrs | 0.752 | 0.008* | (0.193, 1.310) | 0.008 |

9 – Percent of observation stays > 24 hrs | 0.591 | 0.019 | (0.095, 1.087) | 0.017 |

* indicates significance using the BH corrected p-value with α = 0.05.

The BH p-value limit is equal to [α/n * r] where n is the number of tests and r is the p-value significance rank from smallest to largest. For the overall analysis n = 3; for the sub-analysis, n = 6.

Results from Regression Discontinuity Analyses.

Legend

Blue regression lines, “o” plot points: Penalized hospitals (ERR > 1).

Red regression lines, “•” plot points: Non-penalized hospitals (ERR ≤ 1).

Results from regression discontinuity analysis using the average duration of observation stays in hours as the outcome variable.

Results from the regression discontinuity analysis using the percent of observation stays over 12 hours as the outcome variable.

Results from the regression discontinuity analysis using the percent of observation stays over 24 hours as the outcome variable.

These findings show that the HRRP has had an effect on the duration of observation stays for some hospitals. The overall results show that the HRRP has had no impact on the duration of observation stays following index admissions for HF, but the sub-group analysis provides additional granularity. Although there was no effect among penalized hospitals, there was a moderate effect among non-penalized hospitals. Specifically, the results show that the HRRP increases the duration of observation stays at non-penalized hospitals, particularly near the 30-day cutoff.

This study suggests that the use or extension of observation status may be one strategy hospitals adopt to help reduce readmissions for HF. Observation status is certainly appropriate under the right circumstances, but the use of observation status solely to avoid penalties could adversely affect the cost or quality of patient care. Whereas the cost for inpatient care is capped at the inpatient deductible, hospitals bill observation care as an outpatient service with a 20 percent coinsurance. As a result, longer observation stays are associated with higher costs; observation stays greater than 24 hours are significantly higher costs than a corresponding inpatient stay [

This analysis has a few key limitations. First, the results are not significant in alternative nonparametric models and models using a shorter bandwidth of data. Although LR tests show that the linear model fits the data well, robust results that remain significant across alternative model specifications are more reliable and resistant to bias [

This paper has described the use of RD, a powerful quasi-experimental method used to evaluate interventions or policies where treatment is assigned based on a variable with a seemingly arbitrary cutoff value. Under these circumstances, RD provides an unbiased estimate of the treatment effect. Additionally, RD offers several advantages for the evaluation of complex interventions [

This article demonstrates the use of RD to evaluate the impact of the HRRP on the duration of observation stays. The results provide evidence that some hospitals may extend observation stays to avoid payment penalties. Given the potential impact on patients, additional work evaluating the impact of the HRRP is warranted. Additionally, we believe the analytical method described here is underutilized in health services research. RD has many advantages over the traditional randomized experiment. We recommend other researchers consider the RD design for the evaluation of complex health interventions.

The authors have no competing interests to declare.