Skip to main content
Journal of Patient-Reported Outcomes
Download PDF
Download ePub

Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature

Journal of Patient-Reported Outcomes volume 2, Article number: 64 (2018) Cite this article

Abstract

Background

To explore the existing evidence of the real-world implementation of patient-reported outcomes (PROs) in oncology clinical practice and address two aims: (1) summarize available evidence of PRO use in clinical practice using a framework based on the International Society for Quality of Life Research (ISOQOL) PRO Implementation Guide; and (2) describe reports of real-world, standardized PRO administration in oncology conducted outside of scope of a research study.

Methods

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol was developed to guide the systematic literature review (SLR) that was conducted in MEDLINE and Embase databases. A two step search strategy was implemented including two searches based on previously completed reviews. Studies published from 2006 to 2017 were synthesized using a framework based on the ISOQOL PRO Implementation Guide.

Results

After screening 4427 abstracts, 36 studies met the eligibility criteria. Most elements of the ISOQOL PRO Implementation Guide were followed. Two notable exceptions were found: 1) providing PRO score interpretation guidelines (39% of studies); and 2) providing patient-management guidance for addressing issues identified by PROs (25% of studies). Of the 22 studies with an intervention component, 19 (86%) reported intervention effects on study outcomes. The European Organisation for Research and Treatment of Cancer Quality-of-Life Questionnaire-Core 30 (EORTC QLQ-C30) was the most commonly used PRO (n = 10, 28%); use of 38 other PRO measures was also reported. Only three studies (8%) reported real-world PRO implementation.

Conclusion

Reports of real-world PRO implementation are limited. Reports from studies conducted in clinical settings suggest gaps in information on PRO score interpretation and the use of PRO results to inform patient management. Before the promise of practice-based PRO assessment in oncology can be truly realized, investigators need to advance the state-of-the-art of real-time PRO score interpretation as well as developing guidance on how to use PRO insights to drive clinically-meaningful patient-management strategies.

Introduction

There has been growing interest in the assessment of patient-reported outcomes (PROs) over the last 40 years, and the use of PROs in clinical and health services research is common [1]. PROs have been defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.” [2] The incorporation of PROs in clinical practice can serve numerous purposes [3] including: (1) describing a patient’s overall state; (2) screening for incipient disease and undetected disability [4, 5]; (3) monitoring disease progression and response to treatment; (4) assessing patient-centered needs; (5) formulating treatment plans consistent with patient preferences [6,7,8]; (6) improving physician-patient communication [9,10,11]; (7) providing patient-based data for quality initiatives [12,13,14]; and (8) standardizing interactions between healthcare providers and patients [3]. While the use of PROs in clinical practice can help in all of these areas [1, 15], critical questions remain about how patient outcome data should be collected, shared, and used to improve the quality of care and patient health outcomes [16]. Some reports have emerged regarding the use of PROs in routine clinical practice across different conditions [1, 17,18,19,20,21,22], but the incorporation of these tools in oncology clinical practice has been slower than adoption in research [23,24,25,26].

More recently, routine use of PROs in oncology practice has been identified as a priority area by the President’s Cancer Panel [27] as well as by national oncology societies such as the American Society of Clinical Oncology (ASCO) [28]. There is increasing interest in bringing “the patient’s perspective” to cancer decision making which is demonstrated by a number of key initiatives of PRO application in oncology research and regulatory decisions [29]. However, little evidence has been generated with regards to clinical-practice implementation [30].

The interest in implementation of PROs specifically in oncology care is exemplified by the number of recent reviews on PRO clinical applications and their impact on health outcomes [31,32,33,34]. All of the recent oncology reviews provide some insights on gaps in existing evidence of PRO use in clinical practice related to both challenges in implementation and PRO use impact. For example, Howell identified that more attention needs to be paid to complexity of implementation and interpretation [32]. King and colleagues [33] found a scarcity of studies reporting data on actions and medical decisions [33]. Two others—Chen [31] and Kotronoulas [34]—examined PRO intervention evidence and identified weak signals specific to changes in patient management and improved health outcomes [31, 34]. While these reviews identified important evidence gaps, none of them used an existing implementation framework to organize findings or focused on a review of publications reporting on the actual implementation of PRO in real-world settings beyond the context of a feasibility study or intervention trial. The current review makes a unique contribution to the field, by summarizing currently existing evidence using an implementation framework based on the user’s guide for the implementation of PROs in clinical practice recently developed by the International Society for Quality of Life Research (ISOQOL) [35]. The guide includes recommendations for the following implementation elements: (1) identifying the goals for collecting PROs in clinical practice and which key patient outcomes or barriers need attention; (2) considering group of patients and the care settings; (3) determining which questionnaire(s) to use (e.g., whether to use generic or disease-specific questionnaires, profile or preference-based measures, single or multi-item scales, and static or dynamic questionnaires); (4) choosing how often a patient should complete the questionnaires and whether it should be one-time completion or repeated, tied to clinic visits, or a way to monitor patients between visits; (5) deciding how the PRO will be administered and scored; (6) identifying interpretation benchmarks for the PRO score and how scores requiring follow-up will be determined; (7) developing strategies for when the PRO results will be presented and discussed with the patient (such as during or after the visit), how the results will be presented (e.g., numeric, graphical, one-time results or trends over time), and who will see the PRO score reports; (8) determining what will be done to respond to issues identified by the PROs and follow-up; and (9) evaluating the impact and value of the PRO interventions on the practice and patient [35]. While previous publications have discussed various considerations and potential applications of PROs in clinical practice [36,37,38], the ISOQOL PRO Implementation Guide is most recent and provides specific implementation guidance developed by subject matter experts and endorsed by a professional organization.

The objective of this systematic literature review (SLR) was to explore and summarize the existing evidence of PRO use in oncology clinical practice. We address two key aims in the review: (1) summarize available evidence of PRO use in clinical practice using a framework based on the ISOQOL PRO Implementation Guide [35]; (2) describe reports of real-world implementation of PRO measures with oncology patients. Real-world implementation of PROs can provide evidence regarding the usage and potential benefits of PRO adoption derived from real-world clinical settings. For the purposes of this review, real-world implementation studies were defined as those reporting the process of ongoing standardized PRO administration and related clinical actions to manage patient care conducted in a routine clinical practice beyond the scope of a specific research study.

Methods

Study design

We conducted a SLR in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [39, 40]. The protocol was developed following PRISMA guidelines and the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews [41].

Data sources

The literature search was conducted in two databases: MEDLINE and Embase. Blocks of medical subject heading (MeSH) terms were used to identify the most relevant articles and conference papers that describe PRO implementation in oncology clinical practice.

Search strategy

The SLR search strategy was developed in consultation with a professional librarian and used a two-step approach for identifying studies. The first step included a search strategy and approach based on a previously published systematic review of use of PROs in oncology care [31]. As part of the first step, two additional, closely-related systematic reviews were identified [32, 33]. References from these oncology literature reviews were examined, and studies meeting the selection criteria were incorporated into the review [32, 33]. As time has elapsed since the publication of these reviews, they had a narrower focus and used different search strategies and search terms, a second search was conducted to replicate and update the earlier Howell [32] and King [33] reviews to further ensure a comprehensive review of recent publications from the end date of published reviews. MeSH terms and free-text keyword groups (e.g., “neoplasm,” “PRO measure,” “clinical practice,” and “treatment”) were used in different combinations. These updated searches also included specific PROs as search terms, minimizing the risk of missing relevant articles that used these measure, but may have resulted in overrepresentation of these specific measures in the final results. Terminology adjustments were made according to the requirements of each database. Both searches were supplemented by a hand search of references of relevant articles. Appendix A shows the full search strategy.

Selection criteria

Articles were included in the review if they met the following inclusion criteria: (1) cancer focus; (2) articles published in the past 10 years from 2006 to 2016 (inclusive) and abstracts from meetings held in 2015–2016 to ensure review of the current state of the field; (3) published in English; (4) title, abstract, or article contained information pertaining to the measurement of treatment satisfaction, process of care, treatment adherence, treatment decision-making, patient activation, PROs of health-related quality of life (HRQoL), symptoms, or function; and (5) the study design was a randomized controlled trial or an observational study in a clinical-practice setting or a report of PRO implementation in clinical practice.

Exclusion criteria were: (1) articles focused on non-cancer populations; (2) measures did not pertain to clinical outcomes or PROs associated with cancer treatment; (3) basic science studies (e.g., molecular biomarkers, neuroimaging drug formulation); (4) study designs not relevant including study protocols, case studies, case reports, case series, editorials, reviews, commentary, news, or study protocols; and (5) non-English language.

Data screening and abstraction

All abstracts were reviewed using DistillerSR® [42]—a systematic literature review reporting software—to assist with the organization, extraction, and categorization of all literature. Abstract and article screening was performed by three trained reviewers in a two-step process. During the Level 1 review, in order to standardize the review process, a calibration exercise was conducted by the reviewers for all abstracts and titles to assess eligibility for inclusion in the full-text review. Full-text articles that met the inclusion criteria were retrieved. If a determination of eligibility was not possible from the abstract, the full-text article was reviewed. During Level 2, full-text articles were reviewed again for eligibility. Disagreements on eligibility of screened publications at both levels were resolved through discussion with reviewers and final adjudication of unresolved disagreements by the first author of this paper (M.A). For eligible articles, the data was abstracted into a detailed source table that included data fields on study country, study type, cancer type, study objectives, sample size, study duration, study inclusion/exclusion criteria, PRO intervention characteristics, PRO reporting characteristics, study endpoints, assessment timepoints, PRO study results, and limitations/contextualization. The data abstracted into the detailed source table was validated by a second independent senior reviewer to ensure the accuracy of data abstraction. The detailed source table was used to organize information in summary tables that were developed during data analysis and based on the ISOQOL PRO Implementation Guide Framework.

Data analysis

We used the ISOQOL PRO Implementation Guide [35] as the basis for developing a framework to address our first research aim (Table 1). Categories corresponding to each of the ISOQOL PRO Implementation Guide recommendations were created, and information from the articles was extracted into summary tables from the original detailed literature source table. This was done to explore relationships in the data and to establish if all recommended information was included in reports of PRO use in oncology clinical practice. The framework was used to explore the use of PROs in clinical care and their relationship to outcomes in the context of the ISOQOL PRO Implementation Guide. To inform the second aim of this review—to describe reports of real-world implementation of PRO measures settings with oncology patients—we examined the characteristics of all real-world implementation reports.

Table 1 ISOQOL PRO Implementation Guide Framework for SLR Data Synthesis
Full size table

Results

As shown in the PRISMA diagram (Fig. 1), a total of 5754 records were identified, and they yielded 4427 unique publications (following removal of duplicates). A total of 117 full-text articles were retrieved for a second round of screening. Of these, 36 articles met the inclusion criteria and are presented herein (Table 2). The largest number of studies were from the United States (US) (n = 16, 44%) followed by the United Kingdom (UK) (n = 6, 17%), Germany (n = 3, 8%), Canada (n = 2, 6%), The Netherlands (n = 2, 6%), other individual European Union (EU) countries (n = 6, 17%), and Australia (n = 1, 3%).

Fig. 1
figure 1

PRISMA Flow Chart

Full size image
Table 2 Summary Table with Key Elements of Included Studies
Full size table

Results: Available evidence on reporting of ISOQOL PRO implementation guide categories

Data from all 36 reports of PRO measures used in clinical settings were summarized according to the ISOQOL PRO Implementation Guide Framework [35]. .Publications included research studies (intervention research (n = 19, 58%), feasibility research (n = 10, 28%), combination intervention and feasibility (n = 3, 8%), real-world implementation reports (n = 3, 8%); and an intervention for quality improvement (n = 1, 3%) We initially summarized findings related to design considerations of PRO integration in clinical practice within the specified ISOQOL categories (goals for collecting PROs, assessment details, PRO selection, and mode of administration) followed by evidence in categories related to reporting and use of PRO results (reporting of PRO results, PRO score interpretation, plans for addressing issues identified by the PRO, and evaluation of PRO impact on clinical practice). Table 2 presents a summary of key elements from each paper included in the review.

Goals for collecting PROs

As part of this synthesis step, we examined the stated clinical-practice goals for the PRO data collection. Planned use of PROs included monitoring symptoms (n = 13, 36%), improving quality of care (n = 10, 28%), enhancing patient-provider communication (n = 9, 25%), and delivering patient-centered care (n = 9, 25%) (Table 3).

Table 3 Reported Goals of PRO Inclusion in Clinical Care
Full size table

Patients, setting, and timing of assessments

All studies provided relevant details on the type of patients included, study setting, and timing of assessment (before visit, during visit, after visit, at home). Most studies were conducted with adult populations (94%) in outpatient settings (92%). PROs were administered most often at the clinic immediately before seeing the doctor (36%) or during a visit (33%). The type of cancer patient varied with a majority of studies including three or more cancer types (69%), and only seven studies (19%) with a single cancer type.

PROs selected for use in the studies

A total of 46 PRO measures were used across the 36 studies; 33 of the PRO measures were rarely used and were included in only one or two studies suggesting wide variability in measures used. Studies predominantly reported measuring symptoms (n = 15, 42%) or cancer-specific HRQoL (n = 13, 33%) outcomes. The most widely-used measure was the EORTC QLQ-C30 (n = 10,28%) followed by the Hamilton Anxiety and Depression Scale (HADS) (n = 5, 14%).

PRO mode of administration

All but one of the studies reported the mode of PRO administration. Electronic administration was the most common mode (n = 31, 69%) followed by paper-and-pencil (n = 12, 27%). Two studies used interactive voice response (IVR) (n = 2, 5%).

Reporting of PRO results

The summary of information on PRO results indicated that the preferred format of results presentation was an electronic summary report (n = 19, 56%) or a printed copy of PRO results (n = 7, 19%) while e-mails/telephones (n = 5, 14%) were used less often. Results were most often presented only to the clinical team (n = 30, 83%); only three studies (11%) presented the PRO results to both the clinician and the patient, and one study presented results to patients only.

PRO score interpretation

A large proportion of studies (n = 17, 47%) failed to report information on how to interpret PRO scores. Eleven studies (31%) provided PRO scores alone with no interpretation guidance, and six articles (17%) did not report any information on PRO score interpretation. Fifteen studies (42%) provided scores in the form of a graphical display which may aid in score interpretation. About half of the papers reviewed (n = 19, 53%) reported PRO scores along with some information on threshold values, cut-off scores, or severity levels. Only three studies (8%) provided reference groups or norms information for the selected PRO.

Plans for addressing issues identified by the PRO

The majority of studies did not provide any instructions on follow-up steps when PRO scores raised areas of concern. In 13 studies (36%), no instructions were given on next steps or patient-management action items based on PRO results; eight studies (22%) did not include sufficient information on whether PRO results were addressed. The plans for addressing issues identified by the PROs were often related to discussing the identified issues with the provider (n = 10, 28%) with single studies also suggesting specialist referrals, reporting adverse events (AEs), and/or providing educational materials to patients.

Evaluation of PRO impact on clinical practice

Only 19 studies (53%) included results of PRO intervention on patient outcomes. The outcomes for which most studies reported evidence of PRO intervention effect included patient reported symptoms, functioining or quality of life scores (n = 13) and patient-provider communication (n = 8) (Table 4). Eleven of the 19 studies (58%) reported significant PRO intervention effects for all reported endpoints, and five studies (26%) had mixed results and reported significant PRO intervention effects for some—but not all—of the assessed outcomes. Only three of the 19 studies (16%) reported no intervention effect (Table 2).

Table 4 Intervention Study Outcomes and Evidence of PRO Intervention Effect
Full size table

Results: Real-world implementation of PRO measures with oncology patients

Our review identified only three reports of real-world implementation of PRO measures in clinical practice which we defined as the ongoing administration of a standardized PRO and related clinical actions to manage patient care in routine clinical practice beyond the scope of a specific research study.

The first study [43] used retrospective chart review to investigate the relationship between standardized symptom screening and clinical actions to manage symptoms using the Edmonton Symptom Assessment Scale (ESAS) [44]. The ESAS was included in routine clinic visits though self-reporting via an electronic touch-screen kiosk. The ESAS measures the severity (scale of 0–10; 0 = none, 10 = worst) of nine common cancer physical and psychological symptoms (pain, shortness of breath, nausea, anxiety, depression, tiredness, drowsiness, appetite, and well-being). ESAS symptoms were categorized into four severity categories: none (0 score), mild (1–3 score), moderate (4–6 score), and severe (7–10 score) where scores of > 4 indicate clinically-significant symptom issues. Symptom-related actions included relevant drugs being prescribed, medication dosage titration, or a test, treatment, or referral being made. Pain and shortness of breath were documented in 52% and 30% of charts; a related action occurred in 17% and 4% of charts, respectively. However, the frequency of relevant clinical actions was not proportionate to the documented symptom severity [43].

Trautmann and colleagues [45] described the development, implementation, completeness, and first results of an electronic, real-time assessment program for the collection of PROs in a tertiary referral cancer center in Germany. The EORTC QLQ-C30 [46], National Comprehensive Cancer Network Distress Thermometer (DT) [47], and the Hornheider Screening Instrument (HIS) of need for psycho-oncological support [48] were measured. Nutritional status was assessed using the Short-Form Mini Nutritional Assessment (MNA) [49], and pain was assessed using the Brief Pain Inventory (BPI) [50, 51]. A traffic-light system was applied for visualized score interpretation using published cutoff values or means/standard deviations (SDs). A green light indicated scores below a critical clinical importance thresholds based on means and standard deviations of reference population of cancer patients thereby indicating no need for clinical action; a red light indicated scores above a critical cut-off that indicated need for further action. Overall, 67% of patients provided complete information on 12 PROs. Rates of approach and participation varied between the different departments with the highest completion rates in patients presenting for oncological surgical consultation. The number of patients approached to complete PROs increased from 17% to 56% over three months. The percentage of patients completing the PROs increased from 70% to 92% over three months. The majority of patients (62%) reported a score of five or higher on the NCCN Distress scale indicating moderate to high burden; 53% of the patients had a score of four or higher on the HSI indicating a need for psycho-oncological support. Very few participants reported on pain outcomes as EORTC QLQ-C30 pain intensity, and impairment scores were only documented in patients reporting moderate to severe pain. Findings revealed that physician usage of PRO during the clinical consultation was limited. Limiting factors reported by physicians were the lack of knowledge of the PRO reporting system and perceived irrelevancy of some of the assessed PRO data. Rates of clinical action were not reported in the study.

The authors acknowledged a number of obstacles in the study—even though there was an increased number of patients recruited, the usage of PROs in the patient-physician interaction was limited due to physician turnover and lack of completion time provided to patient prior to consultation. The authors concluded that PRO assessments should be more carefully selected to be more clearly of benefit to the health care provider and patient. Additionally, sustaining the implementation and interpretation of PROs should be constantly reinforced with clinicians [45].

Wagner and colleagues [52] assessed cancer-related symptoms with electronic health record (EHR) integration to communicate assessment results to clinical teams in real time. PROMIS computer adaptive tests (CATs) use a computer algorithm developed with item response theory to administer the items. The psychosocial assessment was adapted from the National Comprehensive Cancer Network Distress Thermometer and Problem Checklist [53]. Over the course of three years, 636 patients completed a total of 1493 assessments with 636 patients completing the assessment at least once (301 twice, 184 three times, and 129 four times). Most patients (90.1%) completed the assessment at home rather than at the clinic (9.3%). Severe PROMIS symptom scores (≥70 or 75 depending on symptom) triggered a message to the oncology team. PROMIS T-score clinical severity thresholds (normal, mild, moderate, or severe) have been previously determined with a standard setting exercise that converged clinician expert ratings and patient self-reported severity scores [54]. Overall, one-third of the patients reported current psychosocial health needs. The authors consider that this study demonstrates that precise measurement of symptoms can be implemented while maintaining the brevity required for clinical implementation. EEHR integration also facilitated automated triage for psychosocial and supportive care [52].

Discussion

Evidence on reporting of ISOQOL PRO implementation guide categories

In order to be able to follow the ISOQOL PRO Implementation Guide, researchers need to have a body of evidence to guide choices on recommended categories.The first aim of our review was to examine existing information on recommended implementation based on published information from oncology clinical settings. While no studies in our review directly referenced the ISOQOL PRO Implementation Guide [35], publications on PRO use in oncology clinical practices were well aligned in their reporting of most recommended implementation elements. However, a gap exists in the description of PRO interpretation guidelines and attendant patient-management recommendations necessary to improve PRO outcomes.

Most studies adequately described the planned goal for PRO data collection, study setting, and selection of PRO and mode of administration. Electronic administration, which allows for flexible integration of PROs in clinical care, was used by the majority of studies, but most were not formally integrated into the electronic health record. In addition, formal integration into the EHR may require intensive resources and stakeholder buy-in that have been lacking perhaps due to limited evidence of the improvement in patient outcomes or lack of financial alignment (or incentives).

While the EORTC QLQ-C30 was the most commonly-used measure across studies, a wide variety of PRO measures was reported suggesting there is little consensus on core domains or the “best” PRO to use or consideration whether a measure is developed for clinical trial or clinical practice use. Such variability may be an implementation barrier for PROs in everyday oncology practice—the burden to individual organizations associated with selecting PRO measures, developing assessment guidelines for the selected PROs, and interpreting PRO results may discourage adoption in routine clinical care.

The main gap in evidence identified by this review was the sparsity of interpretation guidelines for PRO results provided to care providers. While most of the reviewed studies provided PRO scores, fewer added interpretation guidelines to these scores or provided follow-up instructions or procedures in case a problem was identified by the PRO. This is an important gap. Without clarity on the meaning, significance, and interpretation of collected PRO data, how can clinical actions be effected to result in improved health care processes and outcomes?

Real-world implementation of PRO measures with oncology patients

Based on the review of the published literature, the use of PRO measures in routine cancer clinical practice outside the context of feasibility or research intervention studies is seldom reported. Only three reports of routine implementation of PROs in clinical settings were identified by the current review and provided limited information for our first key aim The multi-stage process that is required for developing, introducing, testing, integrating, and monitoring PROs in EHR systems has been achieved in only a few US medical centers [55]. Numerous barriers to implementation have been discussed in the literature including: (1) perception among clinicians that PRO completion consumes valuable time during the patient visit; (2) EHR systems have limited abiity to deliver PROs in user-friendly formats for patients; and (3) clinical ecosystem workflow demands challenge full implementation and integration into clinical practice [56].

While our review suggests that PRO implementation in real-world settings outside of research context is scarce, it is possible that PRO implementation in real-world oncology clinical practice may be underreported in the research literature. Implementation efforts may be viewed more as quality-improvement efforts that are building on existing evidence but not always viewed as generating evidence warranting dissemination [23]. Therefore, it is plausible that PRO implementation may be more widespread than indicated by existing peer-reviewed publications. In a recent article, Basch and colleagues [55] noted that a handful of institutions have successfully integrated systematic PRO collection into routine clinical practice; however, no published data have been generated from these institutions related to real-world PRO implementation in oncology clinical care.

Comparison to other systematic literature reviews

Some of our findings are consistent with the results from earlier literature reviews evaluating different aspects of PROs in the context of oncology care (Table 5). We confirmed earlier findings that there is evidence for the effectiveness of PROs on improving provider-patient communication and increased discussion of mental health issues [23, 31,32,33,34]. The EORTC QLQ-C30 was also found to be the most commonly-evaluated PRO in oncology clinical practice settings [32]. The wide variability of PRO measures used has also been noted [55] and continues to be a challenge as confirmed by our findings. Several of the earlier reviews also pointed out the need for increased attention in providing guidance for PRO implementation in oncology clinical practice [33, 34, 57]. Since the completion of our review window, several additional reviews have appeared that focus on some aspect of PRO use in oncology clinical care such as mechanisms through which PROs facilitate increase in patient-physician communication [58] and use of PROs specifically in treating lung cancer [59]. The unique contributions of the current review remains, as no other review focused on separately examining PRO real-world implementation reports or used the ISOQOL PRO Implementation Guide as the framework in analyzing the identified articles. The use of this framework has allowed us to identify specific gaps in the PRO implementation cycle that need to be addressed to encourage use in clinical practice―mainly, the insufficient focus on developing and providing clear PRO score interpretation guidelines and patient-management action plans related to PRO results.

Table 5 Summary of Earlier Relevant SLR of Implementation of PRO in Cancer Clinical Care
Full size table

Strengths and limitations

The strengths of this review include the compliance with PRISMA guidelines, development of a comprehensive two-step search strategy, and review of results in the context of a framework built on the ISOQOL PRO Implementation Guide [35]. The results of the review help further the conversation on PRO implementation in oncology clinical practice by identifying gaps in guidance on interpretation of PRO results and action-oriented patient management based on PRO results. The review also has some limitations including the relatively small number of databases included in the review.

Conclusion

The existing evidence of PRO implementation in real-world clinical care in the published literature is very limited. It is unclear whether implementation efforts are not being studied, not being reported in peer-reviewed journals, simply being published in the grey literature, or not taking place at all.

While publication on PRO real-world implementation is uncommon, a good number of publications on PRO feasibility and/or PRO use in research in oncology clinical care exists as evidenced by our work and earlier literature reviews. This paper also aimed to organize findings of published studies in a framework informed by the ISOQOL PRO Implementation Guide [35]. Results suggested that, with the notable exception of PRO score interpretation and action strategies for PRO-identified problems, most studies report information suggested by the ISOQOL PRO Implementation Guide.

Based on the findings from our review, we offer two insights to help enable more widespread PRO implementation in routine clinical practice. First, adequate interpretation guidelines are needed for PRO results to be acted upon in clinical practice. Second, exploration should be conducted into how to best address issues raised by PRO results—particularly when the identified needs of patients extend beyond the expertise or training found in a routine oncology clinical practice such as depression or lack of social support. In the absence of available information on these key elements, implementation of PROs in clinical practice is unlikely to bridge the gap between perceived usefulness by researchers and routine uptake in oncology practice by clinicians.

Abbreviations

ASCO:

American Society of Clinical Oncology

BPI:

Brief Pain Inventory

DT:

Distress Thermometer

EHR:

Electronic health record

EORTC QLQ-C30:

European Organisation for Research and Treatment of Cancer Quality-of-Life Questionnaire-Core 30

ESAS:

Edmonton Symptom Assessment Scale

HIS:

Hornheider Screening Instrument

HRQoL:

Health-related quality of life

ISOQOL:

International Society for Quality of Life Research

MNA:

Mini Nutritional Assessment

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PRO:

Patient-reported outcome

SD:

Standard deviation

SLR:

Systematic literature review

US:

United States

References

  1. Lohr, K. N., & Zebrack, B. J. (2009). Using patient-reported outcomes in clinical practice: Challenges and opportunities. Qual Life Res, 18(1), 99–107. https://doi.org/10.1007/s11136-008-9413-7.

    Article  PubMed  Google Scholar 

  2. Food and Drug Administration. (2009). Guidance for industry on patient-reported outcome measures: Use in medical product development to support labeling claims. Fed Regist, 74(235), 65132–65133.

    Google Scholar 

  3. McHorney, C. A., & Tarlov, A. R. (1995). Individual-patient monitoring in clinical practice: Are available health status surveys adequate? Qual Life Res, 4(4), 293–307.

    Article  CAS  Google Scholar 

  4. Almy, T. P. (1988). Comprehensive functional assessment for elderly patients. Ann Intern Med, 109(1), 70–72.

    Article  Google Scholar 

  5. Fried, L., Herdman, S., Kuhn, K., Rubin, G., & Turano, K. (1991). Preclinical disability: Hypotheses about the bottom of the iceberg. J Aging Health, 3(2), 285–300.

    Article  Google Scholar 

  6. Detmar, S. B., Aaronson, N. K., Wever, L. D., Muller, M., & Schornagel, J. H. (2000). How are you feeling? Who wants to know? Patients' and oncologists' preferences for discussing health-related quality-of-life issues. J Clin Oncol, 18(18), 3295–3301. https://doi.org/10.1200/JCO.2000.18.18.3295.

    Article  CAS  PubMed  Google Scholar 

  7. Levine, M. N., Gafni, A., Markham, B., & MacFarlane, D. (1992). A bedside decision instrument to elicit a patient's preference concerning adjuvant chemotherapy for breast cancer. Ann Intern Med, 117(1), 53–58.

    Article  CAS  Google Scholar 

  8. Street, R. L., Jr., Gold, W. R., & McDowell, T. (1994). Using health status surveys in medical consultations. Med Care, 32(7), 732–744.

    Article  Google Scholar 

  9. Branch, W. T., & Malik, T. K. (1993). Using ‘windows of opportunities’ in brief interviews to understand patients’ concerns. JAMA, 269(13), 1667–1668.

    Article  CAS  Google Scholar 

  10. Delbanco, T. L. (1992). Enriching the doctor-patient relationship by inviting the patient's perspective. Ann Intern Med, 116(5), 414–418.

    Article  CAS  Google Scholar 

  11. Taylor, K. M., Macdonald, K. G., Bezjak, A., Ng, P., & DePetrillo, A. D. (1996). Physicians' perspective on quality of life: An exploratory study of oncologists. Qual Life Res, 5(1), 5–14.

    Article  CAS  Google Scholar 

  12. Avalere. (2014) A multi-stakeholder vision for patient-centered measurement in new payment and delivery models. http://avalere.com/expertise/life-sciences/insights/avalere-white-paper-facilitating-a-transition-to-using-pros-to-measure-perf.

  13. Lansky, D., Butler, J. B., & Waller, F. T. (1992). Using health status measures in the hospital setting from acute care to ‘outcomes management’. Med Care, 30(5), MS57–MS73.

    CAS  PubMed  Google Scholar 

  14. Stucki, G., & Sangha, O. (1996). Clinical quality management: Putting the pieces together. Arthritis Care Res, 9(5), 405–412.

    Article  CAS  Google Scholar 

  15. Deyo, R. A., & Carter, W. B. (1992). Strategies for improving and expanding the application of health status measures in clinical settings. A researcher-developer viewpoint. Med Care, 30(5), MS176–MS186 discussion MS196–209.

    CAS  PubMed  Google Scholar 

  16. Baumhauer, J. F. (2017). Patient-reported outcomes - are they living up to their potential? N Engl J Med, 377(1), 6–9. https://doi.org/10.1056/NEJMp1702978.

    Article  PubMed  Google Scholar 

  17. Jensen, R. E., Rothrock, N. E., DeWitt, E. M., Spiegel, B., Tucker, C. A., Crane, H. M., Forrest, C. B., Patrick, D. L., Fredericksen, R., Shulman, L. M., Cella, D., & Crane, P. K. (2015). The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care, 53(2), 153–159. https://doi.org/10.1097/MLR.0000000000000289.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Wintner LM, Sztankay M, Aaronson N, Bottomley A, Giesinger JM, Groenvold M, Petersen MA, van de Poll-Franse L, Velikova G, Verdonck-de Leeuw I, Holzner B, Group EQoL. (2016). The use of eortc measures in daily clinical practice-a synopsis of a newly developed manual. Eur J Cancer, 68, 73–81. https://doi.org/10.1016/j.ejca.2016.08.024.

    Article  Google Scholar 

  19. Detmar, S. B., Muller, M. J., Schornagel, J. H., Wever, L. D., & Aaronson, N. K. (2002). Health-related quality-of-life assessments and patient-physician communication: A randomized controlled trial. JAMA, 288(23), 3027–3034.

    Article  Google Scholar 

  20. Rose, M., & Bezjak, A. (2009). Logistics of collecting patient-reported outcomes (pros) in clinical practice: An overview and practical examples. Qual Life Res, 18(1), 125–136. https://doi.org/10.1007/s11136-008-9436-0.

    Article  PubMed  Google Scholar 

  21. Santana, M. J., Feeny, D., Johnson, J. A., McAlister, F. A., Kim, D., Weinkauf, J., & Lien, D. C. (2010). Assessing the use of health-related quality of life measures in the routine clinical care of lung-transplant patients. Qual Life Res, 19(3), 371–379. https://doi.org/10.1007/s11136-010-9599-3.

    Article  PubMed  Google Scholar 

  22. Velikova, G. (2004). Use of electronic quality of life applications in cancer research and clinical practice. Expert Rev Pharmacoecon Outcomes Res, 4(4), 403–411. https://doi.org/10.1586/14737167.4.4.403.

    Article  PubMed  Google Scholar 

  23. Alsaleh, K. (2013). Routine administration of standardized questionnaires that assess aspects of patients' quality of life in medical oncology clinics: A systematic review. J Egypt Natl Canc Inst, 25(2), 63–70. https://doi.org/10.1016/j.jnci.2013.03.001.

    Article  PubMed  Google Scholar 

  24. Basch, E., Deal, A. M., Dueck, A. C., Scher, H. I., Kris, M. G., Hudis, C., & Schrag, D. (2017). Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. https://doi.org/10.1001/jama.2017.7156.

  25. Lipscomb, J., Gotay, C. C., & Snyder, C. F. (2007). Patient-reported outcomes in cancer: A review of recent research and policy initiatives. CA Cancer J Clin, 57(5), 278–300. https://doi.org/10.3322/CA.57.5.278.

    Article  PubMed  Google Scholar 

  26. Stover, A. M., & Basch, E. M. (2016). Implementation of sumptom questionnaires into oncology workflow. J Clin Oncol, 12(10), 859–862.

    Google Scholar 

  27. President’s Cancer Panel Connected health: Improving patients’ engagement and activation for cancer-related outcomes: President’s cancer panel 2014–2015 series. https://deainfo.nci.nih.gov/advisory/pcp/glance/ConnectedHealth14-15.pdf. Accessed 27 Feb 2018.

  28. Stover, A. M., Chiang, A. C., & Basch, E. M. (2016). Asco pro workgroup update: Patient-reported outcome measures as a quality indicator. J Clin Oncol, 34(7), 276.

    Article  Google Scholar 

  29. Basch, E., Snyder, C., McNiff, K., Brown, R., Maddux, S., Smith, M. L., Atkinson, T. M., Howell, D., Chiang, A., Wood, W., Levitan, N., Wu, A. W., & Krzyzanowska, M. (2014). Patient-reported outcome performance measures in oncology. J Oncol Pract, 10(3), 209–211. https://doi.org/10.1200/JOP.2014.001423.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Lipscomb, J., Reeve, B. B., Clauser, S. B., Abrams, J. S., Bruner, D. W., Burke, L. B., Denicoff, A. M., Ganz, P. A., Gondek, K., Minasian, L. M., O'Mara, A. M., Revicki, D. A., Rock, E. P., Rowland, J. H., Sgambati, M., & Trimble, E. L. (2007). Patient-reported outcomes assessment in cancer trials: Taking stock, moving forward. J Clin Oncol, 25(32), 5133–5140. https://doi.org/10.1200/JCO.2007.12.4644.

    Article  PubMed  Google Scholar 

  31. Chen, J., Ou, L., & Hollis, S. J. (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res, 13, 211. https://doi.org/10.1186/1472-6963-13-211.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Howell, D., Molloy, S., Wilkinson, K., Green, E., Orchard, K., Wang, K., & Liberty, J. (2015). Patient-reported outcomes in routine cancer clinical practice: A scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol, 26(9), 1846–1858. https://doi.org/10.1093/annonc/mdv181.

    Article  CAS  PubMed  Google Scholar 

  33. King, S., Exley, J., Parks, S., Ball, S., Bienkowska-Gibbs, T., MacLure, C., Harte, E., Stewart, K., Larkin, J., Bottomley, A., & Marjanovic, S. (2016). The use and impact of quality of life assessment tools in clinical care settings for cancer patients, with a particular emphasis on brain cancer: Insights from a systematic review and stakeholder consultations. Qual Life Res, 25(9), 2245–2256. https://doi.org/10.1007/s11136-016-1278-6.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Kotronoulas, G., Kearney, N., Maguire, R., Harrow, A., Di Domenico, D., Croy, S., & MacGillivray, S. (2014). What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol, 32(14), 1480–1501. https://doi.org/10.1200/JCO.2013.53.5948.

    Article  PubMed  Google Scholar 

  35. International Society for Quality of Life Research prepared by Aaronson N ET, Greenhalgh J, Halyard M, Hess R, Miller D, Reeve B, Santana M, Snyder C User’s guide to implementing patient-reported outcomes assessment in clinical practice, Version: 2015.

    Google Scholar 

  36. McHorney, C. A. (1999). Health status assessment methods for adults: Past accomplishments and future challenges. Annu Rev Public Health, 20, 309–335. https://doi.org/10.1146/annurev.publhealth.20.1.309.

    Article  CAS  PubMed  Google Scholar 

  37. Higginson, I. J., & Carr, A. J. (2001). Measuring quality of life: Using quality of life measures in the clinical setting. BMJ, 322(7297), 1297–1300.

    Article  CAS  Google Scholar 

  38. Santana, M. J., & Feeny, D. (2014). Framework to assess the effects of using patient-reported outcome measures in chronic care management. Qual Life Res, 23(5), 1505–1513. https://doi.org/10.1007/s11136-013-0596-1.

    Article  PubMed  Google Scholar 

  39. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & Group P-P. (2015). Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015 statement. Syst Rev, 4(1). https://doi.org/10.1186/2046-4053-4-1.

  40. Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & Group P-P. (2015). Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015: Elaboration and explanation. BMJ, 349, g7647. https://doi.org/10.1136/bmj.g7647.

    Article  Google Scholar 

  41. Popay J, Roberts H, Sowden A, et al. (2006) Guidance on the conduct of narrative synthesis in systematic reviews: A product from the esrc methods programme. http://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications/NS_Synthesis_Guidance_v1.pdf.

  42. Borstlap, M., van de Laar, M., Zant, J., & van der Korst, J. (1993). Components of health: An analysis in rheumatoid arthritis using quality of life questionnaires and clinical and laboratory variables. Ann Rheum Dis, 52(9), 650–654.

    Article  CAS  Google Scholar 

  43. Seow, H., Sussman, J., Martelli-Reid, L., Pond, G., & Bainbridge, D. (2012). Do high symptom scores trigger clinical actions? An audit after implementing electronic symptom screening. J Oncol Pract, 8(6), e142–e148. https://doi.org/10.1200/JOP.2011.000525.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Bruera, E., Kuehn, N., Miller, M. J., Selmser, P., & Macmillan, K. (1991). The Edmonton symptom assessment system (esas): A simple method for the assessment of palliative care patients. J Palliat Care, 7(2), 6–9.

    CAS  PubMed  Google Scholar 

  45. Trautmann, F., Hentschel, L., Hornemann, B., Rentsch, A., Baumann, M., Ehninger, G., Schmitt, J., & Schuler, M. (2016). Electronic real-time assessment of patient-reported outcomes in routine care-first findings and experiences from the implementation in a comprehensive cancer center. Support Care Cancer, 24(7), 3047–3056. https://doi.org/10.1007/s00520-016-3127-0.

    Article  PubMed  Google Scholar 

  46. Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., Filiberti, A., Flechtner, H., Fleishman, S. B., de Haes, J. C., et al. (1993). The european organization for research and treatment of cancer qlq-c30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst, 85(5), 365–376.

    Article  CAS  Google Scholar 

  47. Roth, A. J., Kornblith, A. B., Batel-Copel, L., Peabody, E., Scher, H. I., & Holland, J. C. (1998). Rapid screening for psychologic distress in men with prostate carcinoma: A pilot study. Cancer, 82(10), 1904–1908.

    Article  CAS  Google Scholar 

  48. Strittmatter G, Mawick R, Tilkorn M (2000) Entwicklung und klinischer einsatz von screening-instrumenten zur identifikation betreuungsbedürftiger tumorpatienten. In: Leb. Aus med. –soziol. Perspekt. Hogrefe, Göttingen, Bern, Toronto, Seattle, pp 59–75.

  49. Rubenstein, L. Z., Harker, J. O., Salva, A., Guigoz, Y., & Vellas, B. (2001). Screening for undernutrition in geriatric practice: Developing the short-form mini-nutritional assessment (mna-sf). J Gerontol A Biol Sci Med Sci, 56(6), M366–M372.

    Article  CAS  Google Scholar 

  50. Cleeland, C. S., & Ryan, K. M. (1994). Pain assessment: Global use of the brief pain inventory. Ann Acad Med Singap, 23(2), 129–138.

    CAS  PubMed  Google Scholar 

  51. Cleeland, C. S. (2009). The brief pain inventory user guide. Houston: M.D. Anderson Cancer Center.

    Google Scholar 

  52. Wagner, L. I., Schink, J., Bass, M., Patel, S., Diaz, M. V., Rothrock, N., Pearman, T., Gershon, R., Penedo, F. J., Rosen, S., & Cella, D. (2015). Bringing promis to practice: Brief and precise symptom screening in ambulatory cancer care. Cancer, 121(6), 927–934. https://doi.org/10.1002/cncr.29104.

    Article  PubMed  Google Scholar 

  53. Holland, J. C., Andersen, B., Breitbart, W. S., Compas, B., Dudley, M. M., Fleishman, S., Fulcher, C. D., Greenberg, D. B., Greiner, C. B., Handzo, G. F., Hoofring, L., Jacobsen, P. B., Knight, S. J., Learson, K., Levy, M. H., Loscalzo, M. J., Manne, S., McAllister-Black, R., Riba, M. B., Roper, K., Valentine, A. D., Wagner, L. I., Zevon, M. A., & Panel, N. D. M. (2010). Distress management. J Natl Compr Cancer Netw, 8(4), 448–485.

    Article  CAS  Google Scholar 

  54. Cella, D., Choi, S., Rosenbloom, S. K., et al. (2008). A novel irt-based caseranking approach to derive expert standards for symptom severity. Qual Life Res, 17(1), A32.

    Google Scholar 

  55. Basch, E. (2017). Patient-reported outcomes - harnessing patients' voices to improve clinical care. N Engl J Med, 376(2), 105–108. https://doi.org/10.1056/NEJMp1611252.

    Article  PubMed  Google Scholar 

  56. Jensen, R. E., Potosky, A. L., Moinpour, C. M., Lobo, T., Cella, D., Hahn, E. A., Thissen, D., Smith, A. W., Ahn, J., Luta, G., & Reeve, B. B. (2017). United States population-based estimates of patient-reported outcomes measurement information system symptom and functional status reference values for individuals with cancer. J Clin Oncol, 35(17), 1913–1920. https://doi.org/10.1200/JCO.2016.71.4410.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Antunes B, Harding R, Higginson IJ, Euroimpact (2014) Implementing patient-reported outcome measures in palliative care clinical practice: A systematic review of facilitators and barriers. Palliat Med 28 (2):158–175. doi:https://doi.org/10.1177/0269216313491619.

  58. Yang, L. Y., Manhas, D. S., Howard, A. F., & Olson, R. A. (2018). Patient-reported outcome use in oncology: A systematic review of the impact on patient-clinician communication. Support Care Cancer, 26(1), 41–60. https://doi.org/10.1007/s00520-017-3865-7.

    Article  CAS  PubMed  Google Scholar 

  59. Bouazza, Y. B., Chiairi, I., El Kharbouchi, O., De Backer, L., Vanhoutte, G., Janssens, A., & Van Meerbeeck, J. P. (2017). Patient-reported outcome measures (proms) in the management of lung cancer: A systematic review. Lung Cancer, 113, 140–151. https://doi.org/10.1016/j.lungcan.2017.09.011.

    Article  PubMed  Google Scholar 

  60. Alvarez-Maestro, M., Viladoms, J. M., Fernandez, A., & De la Cruz, G. (2014). Evaluation of the clinical usefulness of a health-related quality of life questionnaire in patients with prostate cancer. Actas Urol Esp, 38(10), 669–677. https://doi.org/10.1016/j.acuro.2014.03.007.

    Article  CAS  PubMed  Google Scholar 

  61. Basch, E., Deal, A. M., Kris, M. G., Scher, H. I., Hudis, C. A., Sabbatini, P., Rogak, L., Bennett, A. V., Dueck, A. C., Atkinson, T. M., Chou, J. F., Dulko, D., Sit, L., Barz, A., Novotny, P., Fruscione, M., Sloan, J. A., & Schrag, D. (2016). Symptom monitoring with patient-reported outcomes during routine cancer treatment: A randomized controlled trial. J Clin Oncol, 34(6), 557–565. https://doi.org/10.1200/JCO.2015.63.0830.

    Article  CAS  PubMed  Google Scholar 

  62. Basch, E., Artz, D., Iasonos, A., Speakman, J., Shannon, K., Lin, K., Pun, C., Yong, H., Fearn, P., Barz, A., Scher, H. I., McCabe, M., & Schrag, D. (2007). Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. J Am Med Inform Assoc, 14(3), 264–268. https://doi.org/10.1197/jamia.M2177.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Berry, D. L., Hong, F., Halpenny, B., Partridge, A. H., Fann, J. R., Wolpin, S., Lober, W. B., Bush, N. E., Parvathaneni, U., Back, A. L., Amtmann, D., & Ford, R. (2014). Electronic self-report assessment for cancer and self-care support: Results of a multicenter randomized trial. J Clin Oncol, 32(3), 199–205. https://doi.org/10.1200/JCO.2013.48.6662.

    Article  PubMed  Google Scholar 

  64. Berry, D. L., Blonquist, T. M., Patel, R. A., Halpenny, B., & McReynolds, J. (2015). Exposure to a patient-centered, web-based intervention for managing cancer symptom and quality of life issues: Impact on symptom distress. J Med Internet Res, 17(6), e136. https://doi.org/10.2196/jmir.4190.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Berry, D. L., Blumenstein, B. A., Halpenny, B., Wolpin, S., Fann, J. R., Austin-Seymour, M., Bush, N., Karras, B. T., Lober, W. B., & McCorkle, R. (2011). Enhancing patient-provider communication with the electronic self-report assessment for cancer: A randomized trial. J Clin Oncol, 29(8), 1029–1035. https://doi.org/10.1200/JCO.2010.30.3909.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Blum, D., Koeberle, D., Omlin, A., Walker, J., Von Moos, R., Mingrone, W., deWolf-Linder, S., Hayoz, S., Kaasa, S., Strasser, F., & Ribi, K. (2014). Feasibility and acceptance of electronic monitoring of symptoms and syndromes using a handheld computer in patients with advanced cancer in daily oncology practice. Support Care Cancer, 22(9), 2425–2434. https://doi.org/10.1007/s00520-014-2201-8.

    Article  CAS  PubMed  Google Scholar 

  67. Boyes, A., Newell, S., Girgis, A., McElduff, P., & Sanson-Fisher, R. (2006). Does routine assessment and real-time feedback improve cancer patients' psychosocial well-being? Eur J Cancer Care (Engl), 15(2), 163–171. https://doi.org/10.1111/j.1365-2354.2005.00633.x.

    Article  CAS  Google Scholar 

  68. Chiang, A. C., Buia Amport, S., Corjulo, D., Harvey, K. L., & McCorkle, R. (2015). Incorporating patient-reported outcomes to improve emotional distress screening and assessment in an ambulatory oncology clinic. J Oncol Pract, 11(3), 219–222. https://doi.org/10.1200/JOP.2015.003954.

    Article  PubMed  Google Scholar 

  69. Compaci, G., Rueter, M., Lamy, S., Oberic, L., Recher, C., Lapeyre-Mestre, M., Laurent, G., & Despas, F. (2015). Ambulatory medical assistance--after cancer (ama-ac): A model for an early trajectory survivorship survey of lymphoma patients treated with anthracycline-based chemotherapy. BMC Cancer, 15, 781. https://doi.org/10.1186/s12885-015-1815-7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Cox, A., Illsley, M., Knibb, W., Lucas, C., O'Driscoll, M., Potter, C., Flowerday, A., & Faithfull, S. (2011). The acceptability of e-technology to monitor and assess patient symptoms following palliative radiotherapy for lung cancer. Palliat Med, 25(7), 675–681. https://doi.org/10.1177/0269216311399489.

    Article  PubMed  Google Scholar 

  71. Engelen, V., Detmar, S., Koopman, H., Maurice-Stam, H., Caron, H., Hoogerbrugge, P., Egeler, R. M., Kaspers, G., & Grootenhuis, M. (2012). Reporting health-related quality of life scores to physicians during routine follow-up visits of pediatric oncology patients: Is it effective? Pediatr Blood Cancer, 58(5), 766–774. https://doi.org/10.1002/pbc.23158.

    Article  PubMed  Google Scholar 

  72. Epstein, R. M., Duberstein, P. R., Fenton, J. J., Fiscella, K., Hoerger, M., Tancredi, D. J., Xing, G., Gramling, R., Mohile, S., Franks, P., Kaesberg, P., Plumb, S., Cipri, C. S., Street, R. L., Jr., Shields, C. G., Back, A. L., Butow, P., Walczak, A., Tattersall, M., Venuti, A., Sullivan, P., Robinson, M., Hoh, B., Lewis, L., & Kravitz, R. L. (2017). Effect of a patient-centered communication intervention on oncologist-patient communication, quality of life, and health care utilization in advanced cancer: The voice randomized clinical trial. JAMA Oncol, 3(1), 92–100. https://doi.org/10.1001/jamaoncol.2016.4373.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Erharter, A., Giesinger, J., Kemmler, G., Schauer-Maurer, G., Stockhammer, G., Muigg, A., Hutterer, M., Rumpold, G., Sperner-Unterweger, B., & Holzner, B. (2010). Implementation of computer-based quality-of-life monitoring in brain tumor outpatients in routine clinical practice. J Pain Symptom Manag, 39(2), 219–229. https://doi.org/10.1016/j.jpainsymman.2009.06.015.

    Article  Google Scholar 

  74. Hilarius, D. L., Kloeg, P. H., Gundy, C. M., & Aaronson, N. K. (2008). Use of health-related quality-of-life assessments in daily clinical oncology nursing practice: A community hospital-based intervention study. Cancer, 113(3), 628–637. https://doi.org/10.1002/cncr.23623.

    Article  PubMed  Google Scholar 

  75. Izard, J., Hartzler, A., Avery, D. I., Shih, C., Dalkin, B. L., & Gore, J. L. (2014). User-centered design of quality of life reports for clinical care of patients with prostate cancer. Surgery, 155(5), 789–796. https://doi.org/10.1016/j.surg.2013.12.007.

    Article  PubMed  Google Scholar 

  76. Kallen, M. A., Yang, D., & Haas, N. (2012). A technical solution to improving palliative and hospice care. Support Care Cancer, 20(1), 167–174. https://doi.org/10.1007/s00520-011-1086-z.

    Article  PubMed  Google Scholar 

  77. Mooney, K. H., Beck, S. L., Friedman, R. H., Farzanfar, R., & Wong, B. (2014). Automated monitoring of symptoms during ambulatory chemotherapy and oncology providers' use of the information: A randomized controlled clinical trial. Support Care Cancer, 22(9), 2343–2350. https://doi.org/10.1007/s00520-014-2216-1.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Nicklasson, M., Elfstrom, M. L., Olofson, J., & Bergman, B. (2013). The impact of individual quality of life assessment on psychosocial attention in patients with chest malignancies: A randomized study. Support Care Cancer, 21(1), 87–95. https://doi.org/10.1007/s00520-012-1496-6.

    Article  PubMed  Google Scholar 

  79. Rogers, S. N., Lowe, D., & Kanatas, A. (2016). Suitability of the patient concerns inventory as a holistic screening tool in routine head and neck cancer follow-up clinics. Br J Oral Maxillofac Surg, 54(4), 415–421. https://doi.org/10.1016/j.bjoms.2016.01.018.

    Article  CAS  PubMed  Google Scholar 

  80. Rosenbloom, S. K., Victorson, D. E., Hahn, E. A., Peterman, A. H., & Cella, D. (2007). Assessment is not enough: A randomized controlled trial of the effects of hrql assessment on quality of life and satisfaction in oncology clinical practice. Psychooncology, 16(12), 1069–1079. https://doi.org/10.1002/pon.1184.

    Article  PubMed  Google Scholar 

  81. Ruland, C. M., Holte, H. H., Roislien, J., Heaven, C., Hamilton, G. A., Kristiansen, J., Sandbaek, H., Kvaloy, S. O., Hasund, L., & Ellison, M. C. (2010). Effects of a computer-supported interactive tailored patient assessment tool on patient care, symptom distress, and patients' need for symptom management support: A randomized clinical trial. J Am Med Inform Assoc, 17(4), 403–410. https://doi.org/10.1136/jamia.2010.005660.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Schuler, M. K., Trautmann, F., Radloff, M., Schmadig, R., Hentschel, L., Eberlein-Gonska, M., Petzold, T., Vetter, H., Oberlack, S., Ehninger, G., & Schmitt, J. (2016). Implementation of a mobile inpatient quality of life (qol) assessment for oncology nursing. Support Care Cancer, 24(8), 3391–3399. https://doi.org/10.1007/s00520-016-3163-9.

    Article  PubMed  Google Scholar 

  83. Siekkinen, M., Pyrhonen, S., Ryhanen, A., Vahlberg, T., & Leino-Kilpi, H. (2015). Psychosocial outcomes of e-feedback of radiotherapy for breast cancer patients: A randomized controlled trial. Psychooncology, 24(5), 515–522. https://doi.org/10.1002/pon.3684.

    Article  PubMed  Google Scholar 

  84. Snyder, C. F., Jensen, R. E., Geller, G., Carducci, M. A., & Wu, A. W. (2010). Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practice: Putting doctors and patients on the same page. Qual Life Res, 19(7), 1045–1055. https://doi.org/10.1007/s11136-010-9655-z.

    Article  PubMed  Google Scholar 

  85. Snyder, C. F., Herman, J. M., White, S. M., Luber, B. S., Blackford, A. L., Carducci, M. A., & Wu, A. W. (2014). When using patient-reported outcomes in clinical practice, the measure matters: A randomized controlled trial. J Oncol Pract, 10(5), e299–e306. https://doi.org/10.1200/JOP.2014.001413.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Taenzer, P., Bultz, B. D., Carlson, L. E., Speca, M., DeGagne, T., Olson, K., Doll, R., & Rosberger, Z. (2000). Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psychooncology, 9(3), 203–213.

    Article  CAS  Google Scholar 

  87. Takeuchi, E. E., Keding, A., Awad, N., Hofmann, U., Campbell, L. J., Selby, P. J., Brown, J. M., & Velikova, G. (2011). Impact of patient-reported outcomes in oncology: A longitudinal analysis of patient-physician communication. J Clin Oncol, 29(21), 2910–2917. https://doi.org/10.1200/JCO.2010.32.2453.

    Article  PubMed  Google Scholar 

  88. Velikova, G., Keding, A., Harley, C., Cocks, K., Booth, L., Smith, A. B., Wright, P., Selby, P. J., & Brown, J. M. (2010). Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: Secondary outcomes of a randomised controlled trial. Eur J Cancer, 46(13), 2381–2388. https://doi.org/10.1016/j.ejca.2010.04.030.

    Article  PubMed  Google Scholar 

  89. Whittle, A. K., Kalsi, T., Babic-Illman, G., Wang, Y., Fields, P., Ross, P. J., Maisey, N. R., Hughes, S., Kwan, W., & Harari, D. (2016). A comprehensive geriatric assessment screening questionnaire (cga-gold) for older people undergoing treatment for cancer. Eur J Cancer Care (Engl). https://doi.org/10.1111/ecc.12509.

  90. Wolfe, J., Orellana, L., Cook, E. F., Ullrich, C., Kang, T., Geyer, J. R., Feudtner, C., Weeks, J. C., & Dussel, V. (2014). Improving the care of children with advanced cancer by using an electronic patient-reported feedback intervention: Results from the pediquest randomized controlled trial. J Clin Oncol, 32(11), 1119–1126. https://doi.org/10.1200/JCO.2013.51.5981.

    Article  PubMed  PubMed Central  Google Scholar 

  91. Wolpin, S., Berry, D., Austin-Seymour, M., Bush, N., Fann, J. R., Halpenny, B., Lober, W. B., & McCorkle, R. (2008). Acceptability of an electronic self-report assessment program for patients with cancer. Comput Inform Nurs, 26(6), 332–338. https://doi.org/10.1097/01.NCN.0000336464.79692.6a.

    Article  PubMed  PubMed Central  Google Scholar 

  92. Wright, P., Smith, A., Roberts, K., Selby, P., & Velikova, G. (2007). Screening for social difficulties in cancer patients: Clinical utility of the social difficulties inventory. Br J Cancer, 97(8), 1063–1070. https://doi.org/10.1038/sj.bjc.6604006.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  93. Jensen, R. E., Snyder, C. F., Abernethy, A. P., Basch, E., Potosky, A. L., Roberts, A. C., Loeffler, D. R., & Reeve, B. B. (2014). Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract, 10(4), e215–e222. https://doi.org/10.1200/JOP.2013.001067.

    Article  PubMed  Google Scholar 

  94. Luckett, T., Butow, P. N., & King, M. T. (2009). Improving patient outcomes through the routine use of patient-reported data in cancer clinics: Future directions. Psychooncology, 18(11), 1129–1138. https://doi.org/10.1002/pon.1545.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by funding from Genentech, South San Francisco, CA. The authors thank Michael Celone, Owen Cooper, Ashley Duenas, and David Hengerer for their assistance with data collection, and Fritz Hamme for editorial assistance.

Funding

This study was funded by Genentech, Inc.

Availability of data and materials

The data collected during this study may be considered for release in response to reasonable requests to review the data for scientific and/or research purposes.

Author information

Authors and Affiliations

  1. Patient-Centered Research, Evidera, Bethesda, MD, USA

    Milena Anatchkova, Anne M. Skalicky & Colleen A. McHorney

  2. Genentech Inc., South San Francisco, CA, USA

    Jennifer Whiteley

  3. San Francisco, California, USA

    Sarah M. Donelson

  4. Tampa, Florida, USA

    Dayo Jagun

Authors
  1. Milena Anatchkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah M. Donelson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anne M. Skalicky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Colleen A. McHorney
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dayo Jagun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jennifer Whiteley
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Research project: A. Conception, B. Organization, C. Execution; A, B, C: MA, SD, DJ, CAM, AS, JW. Statistical Analysis: A. Design, B. Execution, C. Review and Critique; A: B: MA, CAM, AS, B: MA, AS, C: MA, SD, DJ, CAM, AS, JW. Manuscript: A. Writing of the first draft, B. Review and Critique. A: MA, CAM, AS, B: MA, SD, DJ, CAM, AS, JW. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Milena Anatchkova.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Milena Anatchkova, Anne M. Skalicky, and Colleen A. McHorney are employees of Evidera and received funding from Genentech to conduct the systematic literature review. Jennifer Whiteley is an employee of Genentech, Inc., a member of the Roche Group, and receives stock/options as part of her standard employment compensation.

Dayo Jagun and Sarah Donelson were full-time Genentech employees when this study was initiated and completed. Dayo Jagun and Sarah Donelson reviewed and approved the first and all subsequent drafts of the manuscript.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anatchkova, M., Donelson, S.M., Skalicky, A.M. et al. Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature. J Patient Rep Outcomes 2, 64 (2018). https://doi.org/10.1186/s41687-018-0091-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-018-0091-0

Keywords