Feasibility of Integrating Standardized Patient-Reported Outcomes in Orthopedic Care

ABSTRACT

Objectives: Osteoarthritis of the knee is a chronic disease associated with pain and reduced quality of life. The ability to reliably measure patient-reported symptoms is important for clinical decision making and evaluation of outcomes. Electronic and web-based tools can eliminate much of the labor-intensive aspects of questionnaire administration and enables both real-time evaluation of responses by physicians and integration of data from multiple sites. This article describes the results of implementing a single integrated electronic questionnaire system into routine orthopedic practice at 2 diverse institutions.

Study Design: Case study.

Methods: A web-based version of a general quality-of-life questionnaire (EuroQol 5-dimension [EQ-5D]) and the pain domain of a disease-specific questionnaire (Knee Osteoarthritis Outcome Score [KOOS]) were administered in the office waiting room to (n = 666) patients at 2 centers over a 9-month period using touchscreen devices. Data were analyzed and descriptive statistics were calculated to assess feasibility of integration into the distinct work flows and to assess the agreement of the results.

Results: The electronic questionnaire had a completion rate of 93% to 95%. Average questionnaire completion times were 3 to 5 minutes at each institution. Mean EQ-5D and KOOS scores for patients pre- and postsurgery were also consistent with prior literature studies.

Conclusions: Lessons learned for future adoption of questionnaire systems elsewhere include the need for baseline assessment of clinic work flows to identify the optimal point of administration and the need for IT support. This study demonstrates the feasibility of routinely collecting patient-reported data as part of standard care, which will become increasingly important as the nationwide emphasis on tracking quality and cost-effectiveness of treatments in orthopedics grows.

Am J Manag Care. 2015;21(8):e494-e500

Take-Away Points

Patient-reported outcomes (PROs), through the use of new technologic advances, can be successfully integrated into routine orthopedic practice and networked across distinct institutions.

• PRO data can be collected electronically and scored instantaneously, allowing for real-time feedback to providers.
• PRO data can be used to track patient progress over time and for quality assurance purposes.
• Important lessons learned include the need for baseline assessment of clinic work flows to identify the optimal point of administration and need for IT support.
• PRO collection will become increasingly important as the nationwide emphasis on tracking quality and cost-effectiveness of treatments in orthopedics grows.

Osteoarthritis (OA) is a chronic, disabling disease associated with aging, and it is the predominant reason that most total joint replacements are performed.

The ability to reliably measure and track pain symptoms and quality of life are critical for measuring the success of various therapeutic strategies used to manage OA symptoms, and can also help inform decisions concerning the appropriate timing of surgery.

Along with providing clinicians with an additional tool to guide treatment decisions, patient-reported outcome (PRO) data can also be used to compare the outcome associated with new techniques or procedures and to benchmark best practices, enabling ongoing quality improvement and identifying areas needing additional clinical research.

Consequently, routine clinical practice can strongly benefit from the capture of PRO data to monitor the outcome of care for patients with OA. However, the infrastructure needed to achieve this goal is generally lacking, and there is an ongoing perception that collecting PRO data will disrupt practice flow and increase workload.

^12,34,56

Numerous challenges and obstacles confront institutions attempting to incorporate PRO data collection into routine practice. Traditionally, patient data capture has relied on the distribution of paper questionnaires that are filled out by patients and stored in electronic format—a time-intensive process fraught with risk of information loss or error. Moreover, paper-based systems do not allow for scoring in real time, making it difficult to incorporate the information into the immediate patient—physician encounter. Technology solutions can improve upon the paper-based questionnaire process: specifically, touch-screen technology offers the potential to facilitate collection of data, save time on administration, scoring, and data entry, and increase utility by allowing physicians immediate access to results. In addition, appropriate data security controls allow for secure PRO data collection and transmission across institutions for multi-site research studies. Although previous studies have shown high-quality agreement between data collected from paper-based and electronic-based instruments and demonstrated patients’ willingness to use electronic tools for data collection,

more work is needed to demonstrate the usability of such tools as part of routine orthopedic practice.

^6-9

This article describes the results and experience of a pilot study to incorporate and integrate an electronic PRO data capture system into routine orthopedic practice at 2 large, geographically separated and distinct orthopedic centers. A unified data capture system was developed and implemented, and our objective was to evaluate the feasibility of routine coordinated PRO collection between these 2 institutions and to describe the challenges and innovations that made routine collection possible.

METHODS

We conducted a prospective cohort study to evaluate the implementation of an electronic PRO process into routine orthopedic practice at 2 institutions. A secondary objective was to compare the patient-reported data with previously reported values to validate the use of the data for research.

Study Population

The study was conducted in the orthopedic practices of 2 large academic medical centers: 1 urban (referred to as Center A) and 1 rural (referred to as Center B). The inclusion criteria included patients be aged at least 18 years with a knee-related complaint, seeing any of 5 participating orthopedic surgeons at Center A or 2 participating orthopedic surgeons at Center B, between August 30, 2010, and June 27, 2011. Patients were required to be able to read English or have a translator present. The institutional review board approved this study at both institutions.

Data Collection

Two PRO instruments were chosen, including 1 general quality of life instrument and 1 disease-specific instrument. The 5-item EuroQol (EQ-5D [3L]) was chosen as the general preference-based quality-of-life instrument because it can be used for economic evaluations such as cost per quality-adjusted life-year (QALY) and comparative effectiveness research.

The EQ-5D index is scored on a scale of 0 to 1, with 1 being the best score, and includes 1 additional visual analogue-style question (EQ-VAS) asking the patient to rate their quality of life on a scale from 0 to 100, with 100 being the best score. The pain subscale of the Knee Osteoarthritis Outcome Score (KOOS) was chosen as the disease-specific instrument. This instrument uses the pain questions adapted from the widely used Western Ontario and McMaster Universities Arthritis Index (WOMAC) and is validated for both arthroplasty and non-arthroplasty patients.

The KOOS pain index is scored from 0 to 100, with 100 as the best score indicating “no pain.”

^10,1112,13

These 2 questionnaires were combined into a 16-question instrument and converted into an electronic, Web-based touchscreen version using commercial software (DatStat Illume, Seattle, Washington). The questionnaire was hosted on a Web server at Center B, and a user management system was set up to restrict access to study personnel only. Scoring of the PRO instruments was performed instantly in real time by the questionnaire software, and scores along with individual question responses, patient identifiers, and completion times were saved to an SQL database. A separate, Web-based visual interface (VI) was also developed to electronically view the patient’s scores. In addition to summary scores, the VI allowed physicians to view responses to individual questions, trends over multiple visits, and normative literature-based ranges of values in order to enhance the physician’s understanding of the patient’s condition and any change from prior visits. This individual patient information was also viewable by the physician in real time, enabling physicians to use the PRO data during the visit at which the patient had completed the questionnaires.

The questionnaire completion work flow differed at each institution (

Figure

), so each site’s work flow was studied to determine how to integrate uniform PRO collection with minimal disruption to existing practice flow. At Center A, patients used a touchscreen tablet (iPad) in the waiting room to complete the questionnaires. This electronic version was configured so that there was only 1 question per screen with a large font size (the subject could also zoom in if needed). After registration at the front desk, research assistants obtained informed consent from each patient (described below) and distributed the tablets. Once completed, the research assistant collected the tablet devices, accessed the patient scores via the web application, printed the score summary, and placed this summary into the medical chart for the physician to review during the exam. In addition, the PRO score summary was also scanned to the patient’s electronic health record (EHR) chart after the visit. At Center B, patients used a PC with an LCD touch-screen monitor in the exam room to complete the questionnaires. The clinic nurse responsible for escorting the patient to the exam room opened the questionnaire via an embedded link in the patient’s EHR chart, and instructed the patient on how to use the touchscreen before leaving them to wait for the physician. After patients completed the questionnaire, the physician would access the VI using another embedded link in the EHR chart in the exam room and view the scores with the patient. Although the VI display of scores was not a part of the EHR itself, a copy-and-paste function allowed the physician to easily copy the scores from the VI into the EHR’s visit note without additional typing.

Informed Consent Process

At Center A, because the electronic questionnaire responses were being transmitted to a server at GMC, patients in the study were required to sign an institutional review board-approved informed consent form. All information transmitted between institutions was HIPAA compliant as unique study identification numbers were used in place of any personally identifying information. At Center B, no data was being transmitted outside the institution, and completion of the questionnaires was considered a standard practice of care for the participating physicians.

Statistical Analysis

The statistical analysis for this study was descriptive. In addition to the patient scores on the EQ-5D Index, EQ-VAS, and KOOS pain subscale, we also analyzed the time needed to obtain patients’ informed consent (at Center A) and the time needed for patients to complete the PRO instruments. For the subset of patients who had multiple visits during the study period, we also compared the completion times between the first and second visit to see whether patients were able to reduce their completion times after becoming familiar with the questionnaire. Lastly, for the subset of patients who had completed a questionnaire both before and after total knee replacement (TKR) surgery, we analyzed the change in scores within this patient subgroup.

RESULTS

Center A

A total of 519 questionnaires were completed by 410 unique patients at Center A during the study period. An additional 31 patients that met the inclusion criteria did not complete the questionnaire, giving an overall completion rate of 93% (as shown in

Table 1

). Females composed 64% (263) of the patients, and 34% (143) of patients were male; 2% did not disclosure their gender

The median age was 61 years (range = 18-96). Patients took an average of 3 minutes and 17 seconds (range = 1:05-18:42) to grant informed consent. Once consent was obtained, it took an average of 3 minutes and 19 seconds (range = 0:55-15:00) for patients to complete the questionnaire using the iPad. Eighty-two patients completed the PRO questionnaire on multiple visits, and in this subset of patients, the mean time to complete the questionnaire decreased by 39 seconds, from 3 minutes and 37 seconds at the first visit to 2 minutes and 58 seconds at the second visit. For preoperative visits, the mean EQ-5D Index was 0.60 (SD = 0.20), the mean EQ-VAS score was 71 (SD = 18), and the mean KOOS pain score was 41.8 (SD = 20.2). For visits at 6 months or greater following TKR surgery, the mean EQ-5D Index increased to 0.74 (SD = 0.22), the mean EQ-VAS score increased to 73 (SD = 23), and the mean KOOS pain score increased to 70.4 (SD = 18.4).

.

Center B

A total of 316 questionnaires were completed by 256 unique patients at Center B during the study period. An additional 14 patients that met the inclusion criteria did not complete a questionnaire, and 8 of the 256 patients who did complete 1 questionnaire opted not to fill out additional questionnaires at a subsequent visit during the period. Therefore, the overall completion rates were 95% of patients and 93% of visits. Females composed 64% (162) of the patients, and 36% (85) were male; the median age was 62 years (range = 21-92). The median time to complete the questionnaire was 4 minutes (range = 1-24). Forty patients completed the PRO questionnaire on multiple visits, and in this subset of patients, the mean time to complete the questionnaire decreased by 18 seconds, from 4 minutes, 57 seconds at the first visit to 4 minutes, 39 seconds at the second visit. For preoperative visits, the mean EQ-5D score was 0.61 (SD = 0.25), the mean EQ-VAS score was 63 (SD = 21), and the mean KOOS pain score was 50.6 (SD = 22.8). For visits at 6 months or greater following TKR surgery, the mean EQ-5D Index increased to 0.76 (SD = 0.20), the mean EQ-VAS score increased to 69 (SD = 27), and the mean KOOS pain score increased to 68.9 (SD = 27.1).

In addition, we compared the scores obtained from this study with previously reported EQ-5D and KOOS pain scores in the literature (as shown in

Table 2

) to evaluate whether the data obtained were similar to that obtained from previous research studies, and found that scores at both institutions were well within the expected ranges. Brazier et al,

Fransen et al,

and Lygre et al

reported mean EQ-5D and EQ-VAS scores in various samples of knee osteoarthritis patients and found mean preoperative scores to be in the range of 0.45 to 0.58 (SD = 0.18-0.22). For the KOOS pain scale, Roos et al and Nerhus et al reported mean preoperative scores of 38 to 42 (SD = 18-21) and 6-month postoperative mean scores of 74 to 79 (SD = 20-35). All are not statistically significantly different to the scores we report here, providing reassurance regarding the integrity of the data collected and of data collected during routine practice and its suitability for patient care.

¹⁴¹⁵¹⁶

DISCUSSION

The results of this study demonstrate that PRO data, through the use of new technology advances, can be successfully integrated into routine orthopedic practice and networked across distinct institutions. Patients from both the urban and the other rural areas were able to successfully complete the PRO questionnaire without disrupting clinical work flow, and the resulting data were consistent with scores reported in previously published validation studies. An initial concern was whether patients—especially elderly patients who were not technologically savvy or experienced with touch-screen technology—would be able to complete the PRO questionnaire on a touch-screen device. However, we found that patients from these very different populations were able to successfully complete the questionnaire. The fact that there was a 93% to 95% completion rate gives credence to the fact that the clinical work flow was not disrupted. Acknowledging the fact that surgical decisions are generally made based on a combination of both clinical and radiographic findings, the real-time scoring was seen as beneficial and informative to clinicians because it provided an additional tool to gain insight into the patient’s experience and symptoms, thereby allowing physicians to track patient progress over time and generally improve the quality of patient-centered care.

It should be noted that a critical step in designing a system for routine PRO data capture is a thorough understanding of clinic work flow, physical space, and staffing. In an effort to ensure minimal disruption to the clinical work flow, patient work flow for each participating physician was studied to identify the key personnel involved and the appropriate point of administration of the questionnaire. In addition, because the data from both institutions was stored on a single database housed on Center B’s server, informed consent was only required from Center A’s patients whose protected health information was transmitted outside Center A’s firewall. The need for informed consent was likely a major barrier to routine collection of PRO data because it is a personnel-intensive process as it takes time to go through the informed consent document. Consequently, office staff had to spend time with patients to explain the study and answer questions.

CONCLUSIONS

With a national commitment by both the American Recovery and Reinvestment Act of 2009 and Affordable Care Act of 2010 to both invest in EHRs and to advance the evidence base around areas such as cost effectiveness, safety, and quality through registries, finding ways to combine PRO registry databases with EHRs will become very important. This study illustrated that despite differences in clinic environment, logistical work flow, staffing, and physical devices used for data collection, a uniform, Web-based software system could be deployed at 2 sites to create a multi-site registry of secure PRO data that was usable in real time by both patients and physicians. Collecting and reviewing PROs can serve a critical function in improving patient care, and, therefore, developing simple processes for their routine collection is important for widespread adoption. Our study is generalizable to other sites interested in collecting patient reported outcomes and our results demonstrate that routine collection and sharing data across multiple sites is feasible with the proper technology platforms and trained personnel.

The feasibility of PRO collection demonstrated in this study resulted in the routine collection of PRO data as part of standard care within our department. Streamlining general health and disease-specific questionnaires will allow for more efficient data capture and the ability to form multi-institutional registries. Combining pragmatic data from multiple institutions is important to understanding the outcomes associated with orthopedic procedures in the real world, and therefore essential to rational health policy decisions.

The data collection system we describe here can be generalizable to many other conditions and disciplines as healthcare becomes more focused on value and outcomes. This study demonstrates that even in 2 separate busy orthopedic clinical practices, an electronic system to capture PRO in real time is feasible without any major disruption to the clinical work flow.

Acknowledgments

The authors thank Drs Charles Nelson, Thomas Bowen, Joseph Zuckerman, Laith Jazrawi, Scott Marwin, and Eric Strauss for their participation in the study.

New York University School of Medicine (RK, PB, ZG, JS), New York, NY; Geisinger Health System (CH, JG), Danville, PA.

Author Affiliations:

Source of funding was provided by internal grants from NYU Medical Center and Geisinger Medical Systems.

Source of Funding:

The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Author Disclosures:

Concept and design (RK, PB, ZG, JS, CH, JG); acquisition of data (RK, PB, ZG, JS, CH, JG); analysis and interpretation of data (RK, PB, ZG, JS, JG); drafting of the manuscript (RK, PB, JS, CH, JG); critical revision of the manuscript for important intellectual content (RK, PB, JS, JG); statistical analysis (RK, PB, ZG, JG); provision of study materials or patients (CH); obtaining funding (PB, JG); administrative, technical, or logistic support (ZG, CH); and supervision (PB, JS).

Authorship Information:

Raj Karia, MPH, Hospital for Joint Diseases, NYU Langone, Medical Center, 301 2nd Ave, New York City, NY 10003. E-mail: raj.karia@nyumc.org.

REFERENCES

1. Petty W.

. Philadelphia: W.B. Saunders Company; 1991.

Address correspondence to: Total Joint Replacement

2. Gossec L, Paternotte S, Maillefert JF, et al; OARSI-OMERACT Task Force. The role of pain and functional impairment in the decision to recommend total joint replacement in hip and knee osteoarthritis: an international cross-sectional study of 1909 patients. Report of the OARSI-OMERACT Task Force on total joint replacement.

. 2011;19(2):147-154.

Osteoarthritis Cartilage

3. Hamel MB, Toth M, Legedza A, Rosen MP. Joint replacement surgery in elderly patients with severe osteoarthritis of the hip or knee: decision making, postoperative recovery, and clinical outcomes.

. 2008;168(13):1430-1440.

Arch Intern Med

4. Hibbard JH. Engaging health care consumers to improve the quality of care.

. 2003;41(suppl 1):I61-I70.

Med Care

5. Stewart M, Brown JB, Donner A, et al. The impact of patient-centered care on outcomes.

. 2000;49(9):796-804.

J Fam Pract

6. Roizen MF, Coalson D, Hayward RS, et al. Can patients use an automated questionnaire to define their current health status?

. 1992;30(suppl 5):MS74-MS84.

Med Care

7. VanDenKerkhof EG, Goldstein DH, Blaine WC, Rimmer MJ. A comparison of paper with electronic patient-completed questionnaires in a preoperative clinic.

. 2005;101(4):1075-1080, table of contents.

Anesth Analg

8. VanDenKerkhof EG, Goldstein DH, Lane J, Rimmer MJ, Van Dijk JP. Using a personal digital assistant enhances gathering of patient data on an acute pain management service: a pilot study.

. 2003;50(4):368-375.

Can J Anaesth

9. Galliher JM, Stewart TV, Pathak PK, Werner JJ, Dickinson LM, Hickner JM. Data collection outcomes comparing paper forms with PDA forms in an office-based patient survey.

. 2008;6(2):154-160.

Ann Fam Med

10. Szende A, Williams A (eds). Measuring self-reported population health: an international perspective based on EQ-5D. EuroQol website. http://www.euroqol.org/fileadmin/user_upload/Documenten/PDF/Books/Measuring_Self-Reported_Population_Health_-_An_International_Perspective_based_on_EQ-5D.pdf. Published 2004.

11. Barton GR, Sach TH, Avery AJ, Doherty M, Jenkinson C, Muir KR. Comparing the performance of the EQ-5D and SF-6D when measuring the benefits of alleviating knee pain.

. 2009;7:12.

Cost Eff Resour Alloc

12. Roos EM, Toksvig-Larsen S. Knee injury and Osteoarthritis Outcome Score (KOOS) - validation and comparison to the WOMAC in total knee replacement.

. 2003;1:17.

Health Qual Life Outcomes

13. Heir S, Nerhus TK, Røtterud JH, et al. Focal cartilage defects in the knee impair quality of life as much as severe osteoarthritis: a comparison of Knee Injury and Osteoarthritis Outcome Score in 4 patient categories scheduled for knee surgery.

. 2010;38(2):231-237.

Am J Sports Med

14. Brazier JE, Harper R, Munro J, Walters SJ, Snaith ML. Generic and condition-specific outcome measures for people with osteoarthritis of the knee.

. 1999;38(9):870-877.

Rheumatology (Oxford)

15. Fransen M, Edmonds J. Reliability and validity of the EuroQol in patients with osteoarthritis of the knee.

. 1999;38(9):807-813.

Rheumatology (Oxford)

16. Lygre SH, Espehaug B, Havelin LI, Vollset SE, Furnes O. Does patella resurfacing really matter? Pain and function in 972 patients after primary total knee arthroplasty.

. 2010;81(1):99-107.

Acta Orthop

Download PDF: Feasibility of Integrating Standardized Patient-Reported Outcomes in Orthopedic Care