Beyond Satisfaction Scores: Exploring Emotionally Adverse Patient Experiences

ABSTRACT

Objectives: Although improving the average patient experience is at the center of recent efforts to make cancer care more patient centered, extreme experiences may be more informative for quality improvement. Little is known about the most deeply dissatisfying experiences that predispose disengagement and negatively influence patient outcomes. We sought to establish a framework for emotionally adverse patient experiences and identify the range of common causes.

Study Design: Qualitative study including in-depth interviews and free-text survey comments.

Methods: Thematic analysis of 20 open-ended patient interviews and 2389 free-text survey comments collected in a medical center’s cancer clinics.

Results: Emotionally adverse experiences were rarely reported in survey comments (96; 4.0%) but more frequently discussed in interviews (12 interview participants). Such experiences were identified through explicit statements of negative emotion, language, syntax, and tone. Among these rare comments, hostility as an indicator was easiest to identify, whereas passive expressions of fear or hopelessness were less reliably identified. We identified 3 mutually inclusive high-level domains of triggers of negative emotion—system issues, technical processes, and interpersonal processes—and 10 themes within those domains. There was wide variation in the causes of emotionally adverse experiences and evidence of a complex interplay of patient expectations and preconditions that influenced the perception of negative experiences.

Conclusions: This study presents a taxonomy for classifying emotionally adverse patient experiences expressed in free-text format. Further research should test how perceptions of adverse experiences correspond to recorded ratings of patient satisfaction and subsequent enrollment or utilization.

Am J Manag Care. 2019;25(5):e145-e152Takeaway Points

Extreme dissatisfaction with care can have negative consequences for patients with cancer, such as nonadherence to treatment and disengagement. Understanding and identifying the causes of negative experiences could help focus quality improvement efforts.

• Although emotionally adverse experiences were extremely rare, their causes were diverse, including coordination, technical skills, communication, bad provider and staff behavior, wait times, scheduling, finance and insurance, physical symptoms, travel, and education and information.
• Perception of adverse experiences was influenced by patient priorities, past experiences, clinical needs, and expectations.
• We present a taxonomy that could be used to meaningfully analyze free-text patient data.

Improving patients’ experiences as they face serious illness is a worthy goal, and it correlates strongly with retention in care.¹ Evidence has accumulated that better patient experience has important ancillary benefits, including better treatment adherence and self-reported quality of life.^2-5 Although characteristics of care that lead to positive ratings of patient experience are becoming better understood,^6-9 less is known about the correlates of extreme ratings. Deeply dissatisfying experiences may not have the same correlates as positive ones, and their consequences may be more severe.

In consumer behavior research, events eliciting the strongest negative emotional responses that drive consumers away are sometimes known as disgusters.^10-12 Disgusters are issues that are both very important and very negative for consumers, as opposed to annoyances (negative but less important).¹² Patients are not simply consumers, so we must be cautious in applying marketing theory to healthcare. Nonetheless, the concept of discretely classifying the severity of adverse experiences may help increase understanding of the relationship of patient experience to subsequent adherence, utilization, and outcomes (Podtschaske et al, unpublished data, 2015). Negative experiences, although themselves consequential, also correlate to additional negative consequences, such as avoidance or withdrawal from care,^13,14 lack of participation in decision making,^15,16 nondisclosure of concerns to doctors,^17,18 nonadherence to treatment,^19,20 increased use of emergency services,²¹ and seeking care elsewhere.^22,23 Such actions have negative health consequences for patients and implications for retention in a patient-centered healthcare system.

Understanding patient experience in cancer care is particularly important. Beyond its inherent value, changing providers or poor participation in shared decision making may have worse consequences in cancer than in many other chronic diseases. Previous qualitative investigations of negative experiences have identified some causes, including perception of disparity and exclusion from resources,²³ wait times resulting in delayed treatment,²⁴ unmet information needs,²⁴ having the severity of symptoms dismissed or minimized by oncologists,²⁵ and excessive self-coordination of care.^23,24 Although these studies provide useful clues, there is limited evidence of the roots of extreme adverse experiences in cancer care. As part of an ongoing effort to transform cancer care quality, we aimed to develop a better understanding of emotionally adverse experiences that are egregious to patients and harmful to engagement among oncology patients, which we see as a parallel to the well-known concept of serious adverse events in healthcare.

METHODS

We conducted a secondary analysis of patient interviews and free-text survey comments collected as part of an evaluation of a transformation effort in the cancer clinics of an academic medical institution in the United States using the concept of “emotionally adverse experiences” as a lens. We drew on Fortini-Campbell’s marketing framework¹² to define our concept of emotionally adverse experiences. Her framework proposes that consumers make decisions along 2 axes: importance and good-positive/bad-negative valence. She argues that understanding how consumers experience a product or brand on different issues can help target areas for brand improvement. Issues perceived as important and negative are termed disgusters, which drive consumers away and thus should be prioritized. Although this framework is derived from marketing theory, we see it as applicable to quality improvement in healthcare. However, we broadened our definition to reflect that, unlike consumers in other markets, patients might be unable to switch providers. Drawing on Fortini-Campbell’s framework and directly from the data, we defined such experiences as being indicated by changing providers, filing a complaint, nonadherence to treatment, disengagement from care, or consideration of the aforementioned options; bad word of mouth; and expressions of affront.

Twenty patients were recruited for interviews by flyers at clinic visits. Participants had to be adult (≥18 years) patients with cancer treated at the center. Interviews were semistructured, asking open-ended questions about patients’ experiences with cancer care; they took place in person (in cancer center conference rooms or patients’ homes) or by telephone. Additionally, we analyzed 2389 free-text comments written by patients or their caregivers on surveys that asked structured questions regarding 1 of 4 topic areas: access, communication, coordination, or information and shared decision making.²⁶ Front-desk staff distributed the surveys to patients at check-in for clinic appointments. We included both in-depth interviews and free-text survey comments to minimize methodological bias.

Handwritten comments were transcribed into a database and imported along with interview transcripts into QSR International’s NVivo 11 Pro for analysis. The analytic process followed guidance by Miles and Huberman.²⁷ Two coders independently analyzed data inductively looking for identifiers of affect in language, syntax, and tone and for content that described adverse experiences. Transcripts were first read and then reread while listening to audio (if it was available) to see whether additional negative emotions could be detected in verbal data. We created 3 coding structures for which data were coded at all 3 levels: (1) affective identifiers, (2) triggers (content of issues) that related to an adverse experience, and (3) a 3-tiered subjective rating for the level of adversity (extreme, annoyance, would have been nice to have). Data had to contain 1 or more of the affective identifiers and be rated by the coder in the extreme to constitute an extreme negative experience.

We used the Pleasure—Arousal–Dominance (PAD) framework²⁸ to facilitate interpretation of the data and refine our coding structure of affective identifiers. The PAD framework characterizes emotional states along 3 dimensions: pleasure (+P)/displeasure (—P), aroused (+A)/not aroused (–A), and dominance (+D)/submissiveness (–D). In the data, we found that patients expressed negative emotion in a range of ways, which was difficult to interpret. We therefore used the model to define the emotions expressed by patients and focus analysis on the displeasure axis (ie, bored, disdainful, anxious, hostile). Through discussion, we refined our coding structure to agree on the identifiers of negative emotion and extreme negativity. After coding all data, we looked for patterns across codes to create higher-level, explanatory categories, and we examined whether there were differences in the content of emotionally adverse experiences between interviews and survey comments.

As an additional check of our understanding, we presented sample data, identifiers, and triggers to volunteers from the cancer patient and family advisory council for their opinion as to the completeness of the categories. This process confirmed our list and yielded the additional criterion of description specificity of the event as an identifier (ie, greater detail meant an adverse experience was more impactful).

This study received a nonresearch determination from the Stanford Institutional Review Board in July 2014 because the primary purpose was to evaluate quality improvement efforts.

RESULTS

We analyzed 20 oncology patient interview transcripts and free-text survey comments collected between December 2014 and April 2016. Comments were written on 2389 (8.3%) of 28,912 completed surveys. Cancer tumor groups of interviewed patients included breast, gynecological, and blood. Twelve interview participants were women, and ages reported by 10 participants ranged between 31 and 76 years (median, 60.5 years). Tumor groups of patients who completed surveys included those of interviewed patients plus gastrointestinal, head and neck, neurological, sarcoma, skin, thoracic, and urological. Demographic information was not collected on survey participants. Although it may be possible that some interview participants completed a survey at some point, surveys and interviews could not be linked due to privacy concerns.

Defining and Identifying Extreme Negative Experiences

Emotionally adverse experiences were extremely rare, especially in survey comments, with just 96 (4.0%) comments rated in the extreme. Positive comments were 11 times as common, with 1117 (46.8%) comments coded as happy patients, which was used as a default to categorize comments that lacked substance for content coding. Examples of typical positive survey comments included: “I receive excellent treatments and care here” (A0308) and “We love Dr [X]!” (CM6774). However, these comments were often vague, with little insight as to what actions produced a favorable experience. Patients who had negative experiences typically gave detailed information about what contributed to their negative experience; therefore, we explored this area for targeted quality improvement. Twelve interview participants described at least 1 experience rated as emotionally adverse. The higher frequency of reporting negative experiences in interviews compared with survey comments likely reflects the conversational aspect of interviews, which allows for probing of experiences.

Table 1 shows the index created for affective identifiers of extreme negative feelings. We also identified a range of intensity of negative feelings, from patients suggesting improvements that “would be nice” to a patient wishing for death rather than having to deal with the health system. Our analysis focused on triggers of the extreme end of negativity (—P) related to our definition of emotionally adverse experiences as described in the Methods section. We have included our qualitative readings of the arousal and dominance dimensions of the PAD framework in relation to the affective identifiers. We found that affective identifiers that related to comments that seemed to express low arousal (–A) or submissiveness (–D) were more difficult to judge as adverse experiences than those expressing hostility (+A, +D). Submissiveness appeared in passive statements of problems, words expressing fear or hopelessness, and wishes rather than demands for improvement (ie, suggestions vs complaints). These data felt inherently negative but were less obviously expressive.

Triggers: Causes of Emotionally Adverse Experiences

We identified 3 high-level domains of triggers of adverse experiences: system issues (features of the local health system, including location and insurance authorization processes), technical processes (execution of clinical care by providers, including skills and care planning), and interpersonal processes (how providers/staff relate to patients, such as empathy and understanding patient needs). Ten themes were identified within these domains, but the domains were not mutually exclusive (eg, appointment scheduling related to system problems with policies around appointment booking and peculiarities of individual providers). The relationship between the domains and themes is presented in the Figure. Triggers of extreme negative emotion related to patient expectations and priorities for care, and frequently consequences resulting from their experiences, were mentioned (described in following sections). Table 2 [part A and part B] presents the 10 trigger themes with descriptions, examples from the data, and relevant affective identifiers.

Predictors of Emotionally Adverse Experiences: The Role of Prior Annoyances and Expectations

My last 2 or 3 appointments have been cancelled and rescheduled to a more inconvenient time, because the doctors are so busy. Add that to the parking issues—not enough spaces and slow shuttle buses (or not enough), I am not sure I want to continue at [the cancer center] in the future. (CM4078)

Emerging from our analysis was how prior experiences and expectations preconditioned responses to negative triggers. Whereas certain experiences that threatened a patient’s health, financial stability, or trust in their provider usually rated as emotionally adverse experiences (eg, technical skills, communication, finance and insurance), other themes more frequently rated as annoyances and arose less frequently as extreme negatives (eg, scheduling, travel, wait times). Several factors escalated the intensity of negative experiences. First, repeated exposure to the same annoyance, such as regularly having to wait more than 2 hours for appointments, was interpreted by some as a lack of consideration by the provider, particularly if the patient was in pain or had advanced disease. The occasional long wait time was usually tolerable, but repeatedly delayed appointments created stress for patients, which did accumulate into an adverse experience. Prior experiences served as preconditions for the next encounter so that experiences were cumulative and not viewed in isolation.The proximity of issues also seemed to be a factor, such that waiting and scheduling problems were more frequently reported in survey comments. It may be that patients were situationally attuned to scheduling when completing a survey, as they had just checked in for an appointment.

I had a great relationship with my oncologist and she filled in the gaps. Else my responses would be very different. (CR1274)

First, the most important aspect of treatment is the surgery and hospital care. I personally don’t mind if sometimes the office is hectic or I have long waits. I feel I was given the best treatment possible for my condition. That, by far, makes me satisfied with my care. (CR6347)

Patient-Stated Consequences of Emotionally Adverse Experiences

On the other hand, prior positive experiences could reduce the impact of negative events such that experiences that seemed extremely negative did not evoke much emotion from some patients or the patients made allowances for them. For instance, long wait times were accepted by some patients because they felt their provider gave them full attention and thus recognized that similar attention given to another patient might be the cause of their wait.Patients also talked about how receiving lifesaving treatment was their priority and so they tolerated issues that others perceived as extremely negative.

Patients and family caregivers who related an emotionally adverse experience frequently indicated some additional consequence of that experience. These included emotional consequences, such as experiencing stress, fear, anxiety, and loss of hope, and actions following those emotions, such as filing a complaint or switching care providers. These stated consequences also provided a clear indication of adverse experiences.

DISCUSSION

This study has identified that although emotionally adverse experiences are infrequent, variability exists in how patients express these experiences, as well as their causes and predictors and their consequences. Studying oncology patients was a strength, as this population generally experiences more complex, long-term care requiring greater coordination with high emotional valence, especially compared with episodic care. Although the vast majority of patient survey comments were positive or neutral, patients expressed strong negative emotion in a range of ways, such as through sarcasm, hyperbole, and rhetorical statements. Methodologically, the intensity was variable across issues between interview and survey responses. For example, when patients talked about scheduling problems in interviews, they often spoke with great intensity, and these problems were rated as emotionally adverse experiences. Additionally, the interactive nature of the interview meant that interviewers could probe for information about negative experiences. In survey comments, some wording implied negativity (eg, “later,” “not”), but actual feelings of negativity were not expressed. It may be that these patients were concerned about potential negative impacts to their care as a result of voicing displeasure, as they completed the surveys in the clinic setting and therefore may have perceived that their responses might be identifiable.²⁹

Some patients expressed their feelings in nuanced ways that might be difficult for health systems to identify using typical methods of patient satisfaction surveys or complaint records. Few patients talked about making a formal complaint. Health systems could use existing data sources, such as the Hospital Consumer Assessment of Healthcare Providers and Systems survey comments, to identify negative experiences, but analyzing these data sensitively is time intensive and cumbersome. It should be noted that the cancer center in which this study took place had a high Likelihood to Recommend score at the same time as the study (87.2%, in the 91st percentile nationally; C. Montalvo, BA, written communication, April 2018) and high scores for the survey on which the patient comments were written,²⁶ indicating that overall there is a high level of satisfaction among patients treated at this institution. However, as this study indicates, focusing on survey scores alone may miss critiques that afford opportunities for improvement even in a highly rated system. We believe that health systems would benefit from analyzing textual data to ensure that responses to quality issues are congruent with patients’ priorities in care.² Our system of identifiers could be used as a categorization system for such data.

Although identifying and defining emotionally adverse experiences was more challenging than expected, the range of triggers was less surprising. There was wide variation in the triggers, but threats to well-being and trust were almost universally an affront to patients. This aligns with existing research that has found that causes of acute disgust have in common dehumanizing experiences and breaches of trust^13,30 but that thresholds for tolerating such feelings and coping are variable within and across individuals. In contrast to the literature, some themes we found, such as wait times and travel issues, do not appear to be related to well-being or trust but might reflect other underlying issues that precondition patients’ sensitivity to such annoyances over time.²⁵ For example, if confidence in the competence of the care provider is undermined, patients might be inclined to look for or detect other lapses in care. We found the inverse to be true: An excellent care provider could reduce the burden of annoyances. In this way, emotionally adverse experiences are formed within a context of priorities; the overall priority of surviving cancer might make patients tolerate more than they would otherwise. Alternatively, patients with a serious illness may feel that they are already under stress and perceive typical low-level annoyances as a serious threat. The relative and temporal nature of emotionally adverse experiences that evolve with time in a healthcare context is not a feature of consumer disgust as described in the marketing literature,¹² although the accumulation of experience has been acknowledged as a feature of patient satisfaction.³¹ The conceptual map presented in the Figure may help further understanding of the components of patient satisfaction and particularly the “process” aspect of patient satisfaction with care.³¹

Although some approaches to quality improvement might focus on enhancing positive attributes of care, such as through appreciative inquiry,³² our framework identified important domains that have typically been absent from predictors of patient satisfaction,⁸ including travel or transportation, education and information, scheduling, and finance/insurance. Likewise, typical indicators found in patient satisfaction models that emphasize positive attributes, such as the environment and physical setting, were not present in our framework.^8,33 There may not be complete congruence between issues that positively and negatively influence satisfaction. More mundane features of care, like travel to appointments and scheduling, might be noticed only when they fail to go smoothly, and they therefore might be overlooked by quality improvement that focuses on positive aspects only. Dissatisfaction with care may be more telling than satisfaction,³¹ particularly if patient outcomes are adversely affected. Our aim is not to enumerate absolute triggers of disgust but, rather, to describe the range of patient-specific issues that can trigger such feelings and find ways to recognize them, as we perceive that this may help health systems identify opportunities for quality improvement. Indeed, the medical center in this study responded positively to identifying adverse patient experiences and used it as an opportunity to target improvements in care.

Limitations

We developed our concept of emotionally adverse experiences using Fortini-Campbell’s framework¹² as those that are both important and negative, but within negative could be a range of emotions that were difficult to differentiate. Emotions such as disgust, anger, and fear are universal in the cancer experience and not always related to dissatisfaction with care. To protect patient privacy, we were unable to link surveys to data in the electronic health record for collecting demographic information. Although we have limited demographic information on interview participants, we perceived that the age and gender mix of our sample was broadly reflective of the patient population of the cancer center. For both surveys and interviews, non-English speakers were likely underrepresented. We did not have access to all the interview audio files, as this was a secondary analysis of data. Spoken data may provide more clues to patients’ emotional states, although listening to the audio did not change our interpretation of transcripts. The survey distributed to patients covered 5 theme areas, which may have constrained patients’ comments to these content areas, therefore potentially missing other adverse experiences. However, the themes covered a range of experiences and patients were not instructed to limit their comments. Indeed, many chose to write about topics not specifically queried in the survey. Our findings are limited to the experience of 1 institution.

CONCLUSIONS

We present a categorization system for adverse patient experiences that can be applied to qualitative data, like free-text survey comments, even when satisfaction ratings are high. The 10 domains demonstrate a wide range of issues that can lead to emotionally adverse experiences, which could be difficult for health systems to tackle at once. Drawing on the specificity found in routinely collected qualitative data, such as survey comments, can help target quality improvement efforts to those domains in greatest need of improvement. Further research should be conducted to test the congruence of extreme dissatisfiers with extreme delighters in healthcare. In the meantime, listening to the dissatisfied patient voice in survey comments can help providers and managers alike improve care, even in high-performing systems.

Acknowledgments

The authors thank Gurpreet Ishpuniani, BS, of Stanford University, and the patients, family caregivers, staff, and administrators for their contributions to this study.Author Affiliations: Division of Primary Care and Population Health, Stanford University School of Medicine (LMH, DLZ, MDW, SMA), Stanford, CA; Betty Irene Moore School of Nursing, University of California, Davis (KMDS-S), Sacramento, CA; Department of Management Science and Engineering, School of Engineering, Stanford University (MV), Stanford, CA.

Source of Funding: Stanford Health Care.

Author Disclosures: Dr De Sola-Smith received a modest payment for completion of the background and discussion sections of the manuscript. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (LMH, MDW, SMA); acquisition of data (LMH, KMDS-S, MV); analysis and interpretation of data (LMH, DLZ, MDW); drafting of the manuscript (LMH, DLZ, KMDS-S, MDW, SMA); critical revision of the manuscript for important intellectual content (LMH, DLZ, KMDS-S, MV, MDW, SMA); statistical analysis (DLZ, MDW); provision of patients or study materials (MV, MDW); obtaining funding (MDW, SMA); administrative, technical, or logistic support (DLZ); and supervision (MDW).

Address Correspondence to: Laura M. Holdsworth, PhD, Division of Primary Care and Population Health, Stanford University School of Medicine, 1265 Welch Rd, Stanford, CA 94305. Email: l.holdsworth@stanford.edu.REFERENCES

1. Jones TO, Sasser WE Jr. Why satisfied customers defect. Harv Bus Rev. 1995;73(6):88-99. hbr.org/1995/11/why-satisfied-customers-defect. Accessed June 21, 2018.

2. Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. N Engl J Med. 2013;368(3):201-203. doi: 10.1056/NEJMp1211775.

3. Arora NK. Interacting with cancer patients: the significance of physicians’ communication behavior. Soc Sci Med. 2003;57(5):791-806. doi: 10.1016/S0277-9536(02)00449-5.

4. Street RL Jr, Makoul G, Arora NK, Epstein RM. How does communication heal? pathways linking clinician-patient communication to health outcomes. Patient Educ Couns. 2009;74(3):295-301. doi: 10.1016/j.pec.2008.11.015.

5. Street RL Jr, Mazor KM, Arora NK. Assessing patient-centered communication in cancer care: measures for surveillance of communication outcomes. J Oncol Pract. 2016;12(12):1198-1202. doi: 10.1200/JOP.2016.013334.

6. Attree M. Patients’ and relatives’ experiences and perspectives of “good” and “not so good” quality care. J Adv Nurs. 2001;33(4):456-466. doi: 10.1046/j.1365-2648.2001.01689.x.

7. Kvåle K, Bondevik M. What is important for patient centered care? a qualitative study about the perceptions of patients with cancer. Scand J Caring Sci. 2008;22(4):582-589. doi: 10.1111/j.1471-6712.2007.00579.x.

8. Shirley ED, Sanders JO. Patient satisfaction: implications and predictors of success. J Bone Joint Surg Am. 2013;95(10):e69. doi: 10.2106/JBJS.L.01048.

9. Heerdegen ACS, Petersen GS, Jervelund SS. Determinants of patient satisfaction with cancer care delivered by the Danish healthcare system. Cancer. 2017;123(15):2918-2926. doi: 10.1002/cncr.30673.

10. Malcolm E, Milstein A. Achieving higher quality and lower costs via innovations in healthcare delivery design. In: Phillips RA, ed. America’s Healthcare Transformation: Strategies and Innovations. New Brunswick, NJ: Rutgers University Press Medicine; 2016:105-112.

11. O’Toole K. How do you come up with good ideas? Stanford Bus. 2013;81(1):8-10. gsb.stanford.edu/sites/default/files/stanford-business-magazine-spring-2013.pdf. Accessed June 21, 2018.

12. Fortini-Campbell L. Integrated marketing and the consumer experience. In: Iacobucci D, Calder B, eds. Kellogg on Integrated Marketing. Hoboken, NJ: John Wiley & Sons Inc; 2003:54-89.

13. Reynolds LM, Bissett IP, Porter D, Consedine NS. The “ick” factor matters: disgust prospectively predicts avoidance in chemotherapy patients. Ann Behav Med. 2016;50(6):935-945. doi: 10.1007/s12160-016-9820-x.

14. Kannan VD, Veazie PJ. Predictors of avoiding medical care and reasons for avoidance behavior. Med Care. 2014;52(4):336-345. doi: 10.1097/MLR.0000000000000100.

15. Barry MJ, Edgeman-Levitan S. Shared decision making—the pinnacle of patient-centered care. N Engl J Med. 2012;366(9):780-781. doi: 10.1056/NEJMp1109283.

16. Institute of Medicine. Partnering With Patients to Drive Shared Decisions, Better Value, and Care Improvement. Washington, DC: The National Academies Press; 2014.

17. Bar-Sela G, Yochpaz S, Gruber R, Lulav-Grinwald D, Mitnik I, Koren D. The association between the strength of the working alliance and sharing concerns by advanced cancer patients: a pilot study. Support Care Cancer. 2016;24(1):319-325. doi: 10.1007/s00520-015-2794-6.

18. Dwamena F, Holmes-Rovner M, Gaulden CM, et al. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev. 2012;12:CD003267. doi: 10.1002/14651858.CD003267.pub2.

19. Jerant A, Fenton JJ, Bertakis KD, Franks P. Satisfaction with health care providers and preventive care adherence. Med Care. 2014;52(1):78-85. doi: 10.1097/MLR.0000000000000021.

20. Zolnierek KB, Dimatteo MR. Physician communication and patient adherence to treatment: a meta-analysis. Med Care. 2009;47(8):826-834. doi: 10.1097/MLR.0b013e31819a5acc.

21. Bertakis KD, Azari R. Patient-centered care is associated with decreased health care utilization. J Am Board Fam Med. 2011;24(3):229-239. doi: 10.3122/jabfm.2011.03.100170.

22. Roberson Barnard S. Is it OK to fire my oncologist? J Oncol Pract. 2014;10(2):151-153. doi: 10.1200/JOP.2013.001243.

23. Gould J, Sinding C, Mitchell T, et al. “Below their notice”: exploring women’s subjective experiences of cancer system exclusion. J Cancer Educ. 2009;24(4):308-314. doi: 10.1080/08858190902997324.

24. Mathews M, Ryan D, Bulman D. What does satisfaction with wait times mean to cancer patients? BMC Cancer. 2015;15:1017. doi: 10.1186/s12885-015-2041-z.

25. Izugami S, Takase K. Consumer perception of inpatient medical services. PLoS One. 2016;11(11):e0166117. doi: 10.1371/journal.pone.0166117.

26. Winget M, Haji-Sheikhi F, Asch SM. Development of a tailored survey to evaluate a patient-centered initiative. Am J Manag Care. 2018;24(2):e294-e301.

27. Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. 2nd ed. London, United Kingdom: Sage; 1994.

28. Russell JA, Mehrabian A. Evidence for a three-factor theory of emotions. J Res Pers. 1977;11(3):273-294. doi: 10.1016/0092-6566(77)90037-X.

29. Hawkins JB, Brownstein JS, Tuli G, et al. Measuring patient-perceived quality of care in US hospitals using Twitter. BMJ Qual Saf. 2016;25(6):404-413. doi: 10.1136/bmjqs-2015-004309.

30. Hack TF, Degner LF, Parker PA; SCRN Communication Team. The communication goals and needs of cancer patients: a review. Psychooncology. 2005;14(10):831-845; discussion 846-847. doi: 10.1002/pon.949.

31. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc Sci Med. 1997;45(12):1829-1843. doi: 10.1016/S0277-9536(97)00128-7.

32. Moorer K, Kunupakaphun S, Delgado E, et al. Using appreciative inquiry as a framework to enhance the patient experience. Patient Exp J. 2017;4(3):128-135. pxjournal.org/journal/vol4/iss3/18. Accessed June 21, 2018.

33. Naidu A. Factors affecting patient satisfaction and healthcare quality. Int J Health Care Qual Assur. 2009;22(4):366-381. doi: 10.1108/09526860910964834.

Download PDF: Beyond Satisfaction Scores: Exploring Emotionally Adverse Patient Experiences