Introduction

Patient Reported Outcome Measures

Patient-reported outcome measures (PROMs) refer to standardized instruments designed to measure particular phenomena or constructs of individuals’ health status in defined populations [1] including symptoms specific to a single condition, general physical or emotional health, or general quality of life measures. While PROMs have long been collected in research [2], recent examples of successful implementation in diverse clinical settings (e.g., oncology, behavioral health, neurology, orthopaedics) are emerging as PROMs are recognized by United States providers as valuable information to guide patient care [3, 4, 5]. Beyond clinical care, PROMs are now included in quality and outcome metrics and payer mandates [6], as well as population health and research studies.

PROMs in Total Joint Replacement surgery (TJR): an early case study

Total joint replacement surgery (TJR) is a prime example where PROM adoption spans use in the clinic, quality monitoring, payer mandates, and research. Patients elect TJR to relieve advanced knee or hip pain and alleviate the associated functional limitations [7]. Validated PROMs reliably quantify knee or hip arthritic pain, joint-related functional limitations, and physical health status both before and after TJR [8, 9, 10]. Recently the Center for Medicaid and Medicare Innovation (CMMI) instituted a national bundled payment program that, in addition to costs, will compare hospital quality using traditional measures of 30-day post-TJR readmission and 90-day complication rates, as well as PROMs collected before and after TJR [11]. Thus, state, regional, and national TJR quality programs now collect PROMs [12, 13]. Beyond CMMI, private health care payers, (e.g., Blue Cross/Blue Shield of Massachusetts, the Minnesota Community Measurement network, and the Pacific Business Group on Health) are requiring the collection of PROMs for participation in value-based payment programs [14]. Anticipating the need for national TJR outcome norms, in 2010 the Agency for Healthcare Research and Quality (AHRQ) funded a national research registry/cohort of TJR patients with the primary outcome of patient-reported pain and physical function. (P50HS018910; FORCE-TJR) [15, 16, 17] The rapid expansion of PROMs to assess TJR outcomes foreshadows future applications of PROMs in other procedures and conditions and offers important early lessons on PROMs implementation in clinical care.

Challenges in PROM implementation

PROM implementation includes significant challenges (e.g., altered clinic workflow; limited web-based tools to support real-time scoring and trending across time and clinical settings, visualization tools for diverse stakeholder use). Thus, successful PROM implementation requires clinical, operational, and analytic resources and expertise. In the TJR example above, PROM implementation efforts that relied solely on hospital-generated information technology (e.g., web-based surveys or emails) failed to reach patients and resulted in disappointing collection rates below 40 percent [18]. Thus, a framework to guide successful PROM implementation is critically important to assure meaningful PROMs data collection, interpretation, and use.

Need for implementation framework

As the value of measuring PROMs is increasingly recognized, a consistent framework is needed to guide successful implementation planning, collection, and use of PROMs to serve multiple goals and stakeholders. To define this framework, we identified unifying themes from successful PROM users in behavioral health, primary care, oncology, orthopaedics, pediatric gastroenterology, and neurology, as well as health researchers. Our framework integrates the varying uses of the key stakeholders ranging from individual patient care decisions, to quality improvement initiatives, payer mandates, and population health research. The proposed framework can guide future PROM implementation efforts across learning health care systems to assure that complete PROMs are captured at the correct time, and with associated risk factors, to generate meaningful information to serve diverse stakeholders.

Methods

A team of experts from four leading orthopedic registries that collect PROMs (FORCE-TJR, CERTAIN, the Kaiser Permanente Total Joint Registry and the California Joint Replacement Registry) [19, 20, 21, 22], together with diverse stakeholders (clinicians, patients, electronic medical record vendors, hospital administrators, health services and outcomes researchers, and policymakers), identified generalizable PROM implementation lessons. These lessons informed a framework and toolkit of promising strategies to accelerate collection and use of PROMs in health care.

The team identified best practices and examples of successful PROM implementation across clinical disciplines, as well as the challenges and barriers to PROM collection and use, through a scan of recent literature, synthesis of their collective experience, and key informant interviews of 46 individuals actively engaged in the use of PROMs in diverse clinical settings and specialties. Interviewees represented 38 organizations (15 academic research centers, 6 regional delivery systems, 6 professional societies, 3 health information vendors, 4 payers, 3 policy makers, and 2 federal funders). Experts were identified through the literature search and involvement in other forums examining PROM implementation in clinical settings.

Next, the team engaged over 40 stakeholders through two interactive web-based discussions followed by an in-person workshop with 25 participants, held June 22, 2015 in Washington, DC. Participants in both the webinar and workshop included clinicians, patients, researchers, policymakers, research funders, payers and professional societies with interest and/or experience with PROM collection initiatives. (See Appendix 1 for participants). The webinar included facilitated discussion grounded in case examples of current PROM implementation from prior interviews (e.g., knee and hip arthritis and TJR, oncology, depression, pediatric chronic care) as well a reactor panel with patient and EHR vendor perspectives. Input from the subsequent workshop discussion clarified and refined the final framework.

Results

During the workshop the team reviewed and refined the core dimensions of the framework and codified six key steps for implementation (Figure 1). At each step, key system design considerations are outlined, including examples of lessons learned from successful PROMs implementations.

Figure 1 

Collection and use of PROMs in the Learning Health Care System

1. Why PROMs? Identifying the value for diverse stakeholders

In a true learning health system, PROM collection will meet the value proposition for all stakeholders, leading to efficient collection with the greatest utility of research-quality data. Multiple stakeholders value the collection and use of PROMs, but identify different primary goals or value for use of the PROMs. We identified four primary use cases for PROMs that create unique value propositions for key stakeholders. These include 1) individual patient care decisions, 2) quality improvement, 3) value-based payment, and 4) population health and research. Prior to implementing PROM collection strategies, it is important to establish what value PROMs will hold for each stakeholder involved in the process. (See Table 1.)

Table 1

Shared Value of PROMs by User Groups

PROMS USERS SHARED VALUE FOR PROMS

1. Patients and Clinicians Individual patient care decisions:
Individual patient-centered decisions to prioritize, treat, and monitor disease symptoms and health status
2. Hospital Leaders and Clinicians Quality improvement:
Monitor and improve aggregate patient outcomes as compared to national best practice and benchmarks
3. Insurers and Hospital Leaders Value-based payment:
Measure outcomes as compared to costs and utilization to optimize health care value
4. Researchers, Policy makers, and Funders Population health and research:
Generate new evidence for best clinical practices across patients to achieve optimal health status over time.

A comprehensive approach that brings stakeholders together at the outset to delineate goals for the use of PROMs will guide implementation such that the data are collected once and meet each group’s needs [23]. Below we provide case examples of PROMs use highlighting shared value for PROM implementation efforts.

Use 1. Value in Clinic: PROMs Facilitate Dialogue Between Patients and Clinicians to Support Individual Care Decisions

The team identified clinical examples where scored, trended PROMs guide shared decision making between the patient and clinician at the time of the office visit [24]. For example, on arrival at one oncology clinic, patients report the side effects of their chemotherapeutic regimen and rank the most distressing symptoms. These data are then available to the clinician to guide medication adjustments, in addition to managing biologic markers of disease. The patient has the ability to prioritize topics for discussion at office visits and to document symptoms change over time. Clinicians value the ability to systematically monitor symptoms and efficiently identify issues of greatest concern to the patient. Shared decisions about changes in medications or other treatment modalities are guided by scored PROMs available at each office visit.

Use 2. Value to Hospital Leaders and Clinicians: PROMs to Guide Quality Improvement

Aggregate hospital-level PROMs are increasingly important to guide improvement activities for both patient safety and outcomes. For example, annual, aggregate PROMs for total hip replacement care in every hospital in Sweden are publicly reported and compared and are an integral aspect of the national and regional quality monitoring programs [25]. In the United States, CMS began public reporting of risk-adjusted 30-day readmissions and 90-day all cause complications after TJR by hospital in 2013 and 2014, respectively and efforts are underway to evaluate the inclusion of PROMs as part of overall quality monitoring [26]. Both the National Quality Foundation and CMS endorse the use of PROMs for quality monitoring and in value payment programs [27]. At a regional level, the California Joint Replacement Registry in 2015 publicly reported risk-adjusted PROM collection rates and comparative scores for participating hospitals [28].

Some expert interviews identified PROM use within individual health systems as a way to evaluate the effectiveness of treatment plans and patient experience. For example, Cincinnatti Children’s Hospital [29] developed different platforms for parents and children to collect and submit PROMs for symptoms of chronic conditions, such as pediatric irritable bowel disease. Over time, these results were used to improve the design of patient visits so that they are focused on the symptoms and factors most important to patients and their parents. At some hospitals participating in both the FORCE-TJR and California registries, PROM results at the individual physician level are reviewed at quarterly clinical meetings and evaluated to see about any indicated changes in practice.

Use 3. Value to Insurers and Hospital Leaders: Drive Value-Based Payments with PROMs

Government and commercial payers are increasingly turning to PROMs to evaluate quality of care to include in value-based payment models. Today, CMS requires global quality of life measures to meet end-stage renal disease and dialysis quality and reimbursement mandates. The United Kingdom is measuring and publicly reporting PROMs after TJR and links regional health care budgets to PROMs [30]. The Pacific Business Group on Health requires that participants in its travel surgery programs (for hip and knee replacement and spine) collect and share PROM results [31]. Whether PROMs are used for quality monitoring or to meet value-based payment expectations, government quality agencies, payers, hospital quality programs and clinicians are beginning to monitor PROs in quality improvement and cost management programs.

Use 4. Value to Policy Makers and Researchers: Population Health Outcomes

PROMs have a long-standing history as a key outcome measure in population health, policy, and research. The seminal Medical Outcomes Study experimented with diverse reimbursement models while developing PROMs to measure and monitor global health status across patient conditions [32]. Since that time, hundreds of pharmaceutical, device, and clinical trials included PROMs as one outcome measure. Global health status measures enable impact comparisons across interventions, as well as cost-effectiveness analyses, while disease-specific measures monitor the impact on the specific condition [33]. Both the Patient Centered Outcomes Research Institute (PCORI) and the Agency for Healthcare Research and Quality (AHRQ) prioritize the patient’s input in design and outcome measurement of comparative effectiveness research (CER) [34]. Future dissemination and implementation research, in addition to CER is encouraged to include PROMs as a key endpoint.

Value based payment programs compare cost and outcomes of health care utilization. Policy makers and funders are increasingly including PROMs in the outcome measures through which to evaluate value and cost-effectiveness. No longer is the absence of a complication an adequate assessment of quality of care. PROMs clarify the patient’s perspective of the benefits of care in the value equation.

As the learning health system adopts population management strategies, annual health status assessments may screen for previously undetected conditions, such as depression, or decline in physical and/or emotional health. Annual PROMs may prove an effective complement to existing population health management strategies and are being tested now [35].

Following a clear value proposition that defines the specific PROM uses, health care systems should address the Who, When, Where, What, and How of PROM implementation in order to assure optimal collection, interpretation, and use.

2. Who? Priority populations for PROM collection

PROMs are generally collected to monitor conditions with chronic symptoms for which the patient is the best source of information and patient assessment of symptom severity informs treatment decisions. Beyond disease symptoms, the patient is the expert in tracking the side effects of treatment [35] such as the nausea or insomnia associated with cancer therapy A clear definition of the patient population is critical prior to collecting PROMs, since the disease, symptoms, or procedure of interest will inform the frequency and duration of the PROM collection.

3. When, Where? Timing PROM collection to meet diverse uses

As mentioned above, the timing and location of PROM collection will vary according to the condition or disease being tracked. In general, leaders who successfully implemented PROMs to date concur that PROM collection is most likely to be complete if collection is embedded in the standard clinic workflow, serves patient and clinician decisions, and is supported by electronic means. Increasingly efforts to capture PROMs are supported through web-based platforms that allow patient access both in the clinic at a kiosk or tablet or in their home prior to the visit. Sites that achieve high rates of PROM capture consistently report that a combination of integrating the collection in the visit workflow and human support encourages patient participation and complete data capture. In addition, the most successful sites had electronic PROM capture with real-time scoring so that data capture was meaningful to the clinic visit and could secondarily serve quality monitoring.

For quality monitoring programs, consistent PROM capture should focus on consistent time intervals when patients are likely to achieve peak outcomes to assure fair comparisons across settings. In contrast, clinicians may value PROM collection at varied time points to coincide with clinic visits to guide patient care decisions. Implementation should address these differing goals, and define compromise time intervals. Finally PROM capture at home will be a useful supplement when patients do not return to the office for physical visits. Defining stakeholders’ multiple goals in advance will guide the necessary timing and location for PROM implementation.

4. What? PROM selection

The selection of PROMs requires a balance among five PROM attributes and pragmatic considerations for implementation and use. Many quality programs collect both a brief global and disease-specific PRO. In addition, PROMs may be translated to multiple languages for use among diverse patients, may require use fees and licenses, and have varied literacy requirements. The availability of external comparisons, or national benchmarks, for legacy scores may also influence the selection. Each of these considerations must be addressed when selecting PROMs for implementation.

Table 2 includes a brief checklist to guide PROM instrument selection with key considerations [36].

Table 2

Checklist for PROM Selection


1. PROM content Items specific to single condition and/or treatment; or assess global health status
2. Patient acceptance Language and cultural appropriateness for users; literacy level appropriate for users
3. Costs, licensing Proprietary with use fees or publicly available
4. Ease of clinic integration Number of items, administration time, and patient burden; valid across multiple modes of administration (electronic, paper, oral)
5. External considerations Legacy or concurrent benchmarks available to guide interpretation; mandates for specific measures; legacy inclusion in disease-specific registries.

PROM Content

Both disease-specific and global PROMs have value. Clinicians value disease-specific PROMs when treating patients, whereas policy makers may value global health measures that capture general emotional and physical health over time, and researchers use global measures across patients with different clinical conditions to identify which care interventions are associated with the greatest improvements in global health. In addition, global PROMs retain the ability to extract utilities, important to conducting cost effectiveness analyses.

Today, patients often complete one of each type of PROM. For example, a patient with knee arthritis may complete both the knee injury and osteoarthritis score (KOOS) and the SF12 (global) before and after TJR [33, 34, 37]. Although many validated condition-specific and global tools exist, they must be evaluated for appropriateness with specific patient populations and distinct conditions. The Patient-Reported Outcome Measurement Information System (PROMIS) initiative is a federally funded effort to address this need through integration and standardization of PROM measures across diverse legacy measures [38].

Patient acceptance of PROMs

Patient attributes may influence PROM selection. For example, visual impairment, dexterity issues or other functional limitations, or lack of web access may preclude people from using specific PROM tools. In Sweden, web-based response rates after TJR were one-third the response compared with paper and pencil collection. As technology evolves and becomes more common-place across socioeconomic and age groups, administrators must stay abreast of changes in patient preferences and behaviors to ensure provision of the most appropriate interfaces that can be completed without barriers.

Cost, Licensing, Ease of Clinic Integration

Some PROMs are proprietary while others remain in the public domain with published norms and scoring protocols. Lengthy PROMs may reduce item completeness as patients become fatigued and may impact patient flow through the health system. These pragmatic considerations are important to assure successful PROM integration and sustained use. User fees should be taken into consideration as part of sustainability planning for long-term projects for which there is a desire to collect repeated measures over time. A need to change measures due to cost considerations can significantly disrupt the continuity of measurement.

External Considerations

Clinicians, quality leaders, and payers need population norms and benchmarks against which to compare PROMs in specific clinical groups. Specialty-specific registries may already collect PROMs and new users may choose to adopt the same measures to assure comparability when benchmarking. Payer mandates may require specific PROMs in order to assemble norms and benchmarks to guide comparisons across hospitals or clinicians. In addition, co-existing clinical factors that influence outcomes will vary across patients and clinical settings must be identified to inform risk-adjustment or case-mix adjustment analytic strategies. For example, a patient with a successful knee replacement surgery who has co-existing low back pain may report excellent pain relief in a disease-specific measure (e.g., knee pain), but limited global function because of low back pain. Before individual or aggregate outcomes can be compared, refined risk-adjustment methods are needed to guide meaningful interpretation of the PROMs.

5. How? Factors in PROM collection

Ideally, PROM collection systems are integrated into the clinical workflow and meet the needs of providers and patients in terms of scored, available information. While primary PROM use may be in clinic, PROMs may be collected in a variety of settings (e.g., medical office, at home, or in the hospital)andmulti-modally (e.g., via paper and pen, phone texting, or a web-based form) so the PROM collection infrastructure must be flexible to meet the demands of varied patients and diverse settings. The choice of PROM collection mode often depends on the proposed use of the data, and availability of technology and resources for establishing collection systems. Consideration must be given to regulatory expectations, data security and ownership, and ease of access and data interoperability, all of which are impacted by planned uses of the data.

Modes of Data Collection

A variety of technical tools are available for PROM capture via paper, web portals, tablets, personal computers, and mobile phones. Automated voice surveys are also available as well as scannable paper forms. While paper assures accessibility to all patients, the results are not immediately scored and available in clinic. As computer literacy becomes more universal, web- or phone-based PROM capture will become the norm.

PROM users may elect EHR web-portal administration so PROM data are collected and stored in the EHR, similar to the availability of standard lab test results. While today’s EHR portals allow PROM collection from home, the EHR rarely offers real-time scoring, risk-adjustment, and trended data limiting the ability to leverage interpretable PROM data in routine patient care. In addition, while the EHR is a reliable source of patient demographic, treatment, and comorbidity data, it may not be optimal for longitudinal PROM collection as patients change providers regularly resulting in PROMs being stored in multiple EHRs.

Direct-to-patient data collection can be a successful alternative for PROM collection. Both the United States FORCE-TJR registry and the Swedish Hip Arthroplasty Register have shown high response rates when collecting data directly from patients at home and aggregating post-operative PROMs centrally. While pre-operative PROMs are collected in the clinic, post-operative surveys (paper or web) are sent to patients in their homes. Sweden reports a 90 percent post-TJR response rate and FORCE-TJR has maintained response rates over 80 percent. Direct-to-patient post-TJR collection assures consistent PROM timing as collection is not linked to an office visit.

Regulatory Considerations

Regulatory considerations in PROM collection are based on data use. PROMs collected for use in the treatment and routine course of care, or for quality improvement, may not require consent depending on the local Institutional Review Board interpretation and the goals of the project. If the purpose of collection is limited to research, then obtaining patient consent is likely necessary When consent is required, the consent process may be seen as an opportunity to engage patients and enhance their understanding of the value of the data to the patient’s care as well as in research. In FORCE-TJR, the consent process likely contributed to the high follow-up PROMs completion rates.

Data Access and Secure Storage

Data storage and access is best planned prior to implementation. If the EHR portal will be used, it must be able to receive the PROM items, calculate scores, link these to both the patient and the date of collection, and store the measures so that each question’s value can be independently extracted.

Stored PROMs should include both the item-level responses and the summary scores. DARTNet uses a distributed data network model where each center retains its own data while also uploading de-identified data to a shared, secure web-portal for QI and research. While this allows for sharing population-level data it does not facilitate patient-level data tracking between the centers. Alternately FORCE-TJR quality improvement sites assign a unique FORCE-TJR identifier to each patient so that when the de-identified data are uploaded to the registry for risk-adjustment and benchmarking, longitudinal PROMs for individuals can be tracked.

Interoperability

Disease-specific PROMs are often valued by specialists (e.g., after surgery or cancer care), but general health status measures are of value to all clinicians, including the patient’s primary care provider However, if the primary care physician does not practice in the same health system where the PROMs were collected, s/he may not have access to the data. Future PROM implementation may allow the patient to control PROM access so the patient can share PROMs across clinicians and health systems. In addition, if PROMs are stored using Cloud technology, multiple entities can access the data with permission from the patient. For example, the CERTAIN hub invites patients to directly enter their baseline data using an online form in the clinic or at home, and patients receive the survey responses to share with their physician. FORCE-TJR’s solution is to collect the PROM data on a third-party web-based platform, and interface with any EHR providing routine transfer of the PROM data or uploading a pdf as a “lab report” for the patient.

Federal standards for the transfer of clinical and administrative data provide guidelines to enhance interoperability DARTNet uses the Health Level-7 (HL7) standards to allow for transfer of the data by mimicking a lab result. The Office of the National Coordinator for Health Information Technology issues guidelines that may be useful in setting up a system that meets standards for sharing of data across electronic health systems.

6. PROMs to inform practice

Once collected, PROMs are an important piece of data to support patient care decisions, monitor population outcomes for quality improvement initiatives, meet external payer monitoring in bundled payments and accountable care arrangements, as well as to serve research. Stakeholders made several recommendations regarding strategies for providing rapidly available and actionable data.

PROMs for individual patient care

In general, trended PROMs require scoring, risk-adjustment, and clear visual displays to assure actionable information to inform individual level care decisions. While PROM scoring does not require statistical modeling, not all EHRs currently have the capability to provide real-time scored individual PROMs. Several health systems have invested in data visualization tools. For example, many diabetes clinics, Duke’s cancer clinics, FORCE-TJR and the Cincinnati Children’s Hospital developed displays that can be used in the office and on handheld devices for patients to track trends and report symptoms and responses to treatment.

Similarly, defining appropriate clinician alerts is an important part of making PROM data useful for clinical care. At Kaiser Permanente, the patient health questionnaire (PHQ)-9 for depression screening has been incorporated into the EHR. A PHQ-9 score below a pre-programmed threshold triggers an email to providers for urgent follow-up of depressive symptoms. Also at Kaiser Permanente, results from the Functional Outcomes of Cancer Treatment (FACT)-G7 are automatically scored and if the patient reports a high symptom burden, the oncology nurse sends a note to the physician with this information.

At the University of Massachusetts Medical Center Arthritis and Joint Center, surgeons use FORCE-TJR PROMs to allow surgeons to discuss the timing and need for TJR if the patient reports pain and function scores below national surgical norms. After TJR, patients and surgeons compare improved scores to the expected norms [39]. Figure 2 and 3 graphs patient-reported physical function and emotional health scores before and after surgery Colors are based on national norms (green-healthy red-below average TJR patient).

Figure 2 

Clinic: Individual Care Decisions: Trended Individual Patient PROM Scores (lab test) Before and After TJR Compared to Benchmark in FORCE-TJR Reporting System

Figure 3 

Quality Improvement: Comparing pre-TJR Function Across Sites Using FORCE-TJR Reporting System

PROMs for quality monitoring

PROMs can be aggregated at intervals to compare risk factors and outcomes for quality monitoring. As mentioned above in orthopedics, health systems use PROM results at internal meetings to monitor quality and evaluate needed changes in practice.

A FORCE-TJR quarterly benchmarking report allows clinicians or hospital leaders to compare the pre-TJR function scores. Median scores (32) represent significant disability and are consistent across sites. However, 5 percent of patients report mild functional deficits (scores 45–50) at the time of surgery.

In general, payer mandates expect one aggregate, risk-adjusted outcome metric for comparison with national benchmarks. FORCE-TJR also reports risk-adjusted functional outcome per site as compared to the national cohort and communitydwelling “normal” patients without arthritis and TJR. These metrics guide quality assessment within health systems and prepares the system for payer evaluations.

Finally, population health research can benefit from the integration of clinical data and PROMs. FORCE-TJR investigators combined ICD-coded medical comorbidities with pre-operative PROMs to enhance the 30-day readmission prediction model [40]. Additional research on optimal ways to include patient-reported outcomes in comparative effectiveness research is needed. For example, PCORI recently funded a pragmatic trial of diverse anti-thrombotic prophylaxis in TJR and the endpoints include both clinical measures (e.g., prevalence of bleeding, clotting) and PROs (e.g., painful, stiff joints).

Conclusion and Next Steps

Based on extensive dialogue with leading national experts, we propose a PROM implementation model to guide learning health care systems in implementing PROMs to meet the needs of diverse stakeholders. While consistent best practices to guide PROM collection and use are emerging, many questions remain unanswered. The solutions to these challenges will influence future PROM adoption success. Innovative methods are needed to define solutions for four persistent challenges.

First, limited EHR interoperability across health care systems is a significant barrier to longitudinal PROM use. Because PROMs capture global health status, we should consider how PROM data can move with the patient to inform care across health care settings, EHRs, and time. Until EHRs are interoperable and accessible to diverse clinicians, cloud solutions, regional health information exchanges, or personal health records may be preferred for accessible PROM storage and optimal use.

Second, longitudinal collection of PROMs (e.g., knee pain at one or two years after TJR) must be flexible enough to reach the patient when clinic visits are not routinely scheduled or required. In our TJR example, following successful surgery, the patient may no longer receive any care from the surgeon or the surgeon’s hospital. Thus, longitudinal PROM assessments distributed from the surgical hospital portal will not serve the patient’s current health care needs. To overcome this limitation, again, users are relying on cloud and health information exchanges or personal health records to assure that the PROM data follow the patient over time.

Third, to engage patients and maximize value in their care, PROM tools must assure real-time scored, consistent measures are trended over time and available to clinicians at the time of treatment decisions. Creative PROM storage strategies are needed to make all PROMs and predictive analytics available to the patient and his or her caregivers. Efficient and effective solutions to these questions are needed to scale PROM collection across the health care system.

Web-based technology and patient computer literacy are both rapidly evolving and current barriers to PROM collection, processing, and storage will be minimized in the future. In parallel, research to define the full potential for PROMs to inform clinical decisions—both for individual patient care and national best practices—is critical before patients, clinicians, health care leaders, and researchers view PROMs as an integral component of the learning health system.

Before the learning health care system can fully embrace patient-reported outcome measures as a valuable, consistent information source to guide care decisions, consensus on best PROMs, efficient collection and analytic methods, and widely available storage solutions are needed. In the interim, our proposed PROM implementation model can guide health care systems adopting PROMs to meet the needs of diverse stakeholders. To expand PROM use and value, research is needed to refine PROM score interpretation for patients, clinicians, health leaders, and policy makers. With these answers, patient-report of change in symptoms or health status will guide future resource allocation and priorities in the learning health care system.