Skip to main content

Development and psychometric evaluation of a questionnaire to measure cancer patients’ perception of care coordination

Abstract

Background

Although the importance of care coordination (CC) is well-recognized, cancer patients often receive poorly coordinated care across varied care settings and different oncology providers. Efforts to improve cancer care are hampered by lack of adequate measures. In this two-part, mixed-method study, we describe the development, refinement, and validation of a new care coordination instrument (CCI) designed to assess cancer patients’ perception of CC.

Methods

In Study 1, an initial CCI was developed incorporating questions based on literature review. The items were then modified following four field tests conducted in a large academic hospital with oncology nurses (n = 20) and cancer patients (n = 120). This modified instrument was used to determine whether the CCI was able to distinguish CC between two practices (30 GI and 30 myeloma patients) within the same hospital setting. In Study 2, 68 patients receiving community-based care participated in seven focus groups. Based on these discussions, the CCI items were again refined, and psychometric evaluation was conducted to assess the quality of the instrument.

Results

Based on field tests, 3 domains of the CCI, Communication, Navigation, and Operational, were defined as critical components of CC. The Operational domain evaluates efficiency of care and is unique to this CCI. The field test demonstrated that GI patients reported significantly better CC Overall and for the Communication and Navigation domains (all p < .05). In Study 2, patients expressed concordance with the CCI items and their CC experiences, establishing validity of the CCI. Qualitative analysis of the focus group discussions indicated that the items with the highest frequencies of participants’ comments were related to the concepts of Navigator, Team, Survey, and Communication. Quantitative analysis identified items with a limited response range or high rates of “neutral” responses; accordingly, those items were removed. The final CCI survey is a 29 item, multiple-choice questionnaire with excellent reliability, Cronbach’s α = .922.

Conclusions

We developed a novel, patient-centered tool with excellent psychometric properties that can be utilized across varied practice settings to assess patients’ perception of cancer care coordination.

Trial registration

Not required; retrospectively registered ClinicalTrials.gov NCT03594006 20 July 2018.

Peer Review reports

Background

Care coordination (CC) is a complex concept that greatly influences a patient’s perception of his/her interaction with a healthcare delivery system and can directly and indirectly affect the quality of patient care. Although the importance of CC is well-recognized, existing evidence demonstrates that cancer patients often receive poor, fragmented care across multiple settings and providers [1,2,3,4]. Poorly coordinated care has been linked to numerous adverse outcomes including medical errors, patient dissatisfaction, higher health care costs, excessive use of health services, and increased morbidity and mortality from the disease [5,6,7,8]. Further, the critical need to improve the quality and value of health care delivery has been highlighted by a recent report that demonstrated that failure in care coordination costs $27.2 to $78.2 billion annually in the U.S. health care system [9]. To date, efforts to improve cancer care delivery have been hampered by the lack of adequate patient-centered measures to assess cancer care coordination. Given the substantial costs and potentially detrimental impacts of poorly coordinated care for cancer patients [5], it is critical to address these measurement gaps in order to further our understanding of current CC approaches and to identify factors that contribute to effective versus ineffective cancer care coordination.

The Agency for Healthcare Research and Quality (AHRQ) defines CC as “deliberate organization of patient care activities between two or more participants (including the patient) involved in a patient’s care to facilitate the appropriate delivery of health care services” [1]. Although care coordination involves multiple participants, cancer patients and their families often serve as care coordinators, arranging appointments, seeking and furnishing information, and navigating the many steps in care [10]. Important for all patients, care coordination is especially critical for cancer patients as therapeutic interventions involve multiple episodes of care, numerous healthcare providers, varied health care settings, and a high overall symptom burden [11]. Effective CC is a key component of cancer care delivery, and provisions of well-coordinated care increase the efficiency of health care delivery and improve patient outcomes [5]. Accordingly, a coordinated, team-based, patient-centered approach to care has been highlighted by the Institute of Medicine (IOM) as a hallmark of a high-quality cancer care delivery system [3].

In response to efforts to improve the quality of care, research on cancer care delivery has rapidly expanded in recent years. A growing body of evidence has provided some insights on cancer CC, mostly based on health care systems or providers’ perspectives on CC. According to research conducted within closed systems such as the Veterans Administration Health System and Kaiser, patients have generally reported favorable cancer care coordination experiences [10, 12,13,14]. It is possible that positive reports of cancer care coordination experiences in these closed systems are due to limited numbers of providers involved and centralized systems that allow easier access to non-provider based personnel. Other systems-level research that examined the effects of specific CC interventions such as patient navigation suggests that there are promising approaches to CC intervention [15,16,17]. In contrast, studies on health care providers’ reports of CC have documented that there are some potential areas of improvement such as greater availability of resources, use of multidisciplinary teams and interventions, and enhanced communication between primary care physicians and oncologists [15,16,17,18,19].

Despite emerging research on cancer care delivery, there is relatively little empirical research on patients’ perception of cancer CC. Limited research on patient-reports of CC generally indicates that patients identify lack of coordinated care as a substantial barrier to cancer care [20,21,22]. However, prior research has yielded mixed findings. It is possible that sampling and methodological differences across prior studies partly account for discrepant results. On the one hand, research focused on patients with breast cancer has generally reported positive patients’ views of CC [23, 24]. Given that breast cancer is the most commonly diagnosed cancer in women [25] and that there are more breast cancer patients participating in social support groups [26], it is likely that the availabiltiy of various support resources mitigates some of the CC-related challenges in these patients. On the other hand, studies on patients with complex cancers such as pancreatic, lung, GI, and head and neck have generally reported poorer perception of CC [27,28,29]. Other research suggests that poorer perception of CC may be associated with patient characteristics such as having comorbid conditions [27, 28], greater levels of depression [13], and lower levels of health literary [23, 28]. Further, poor care coordination often manifests in specific racial and ethnic population groups with disparities in cancer health outcomes [30, 31]. Taken together, the available evidence suggests that there are some patient-level characteristics that influence patients’ perception of CC; however, the processes and specific aspects of cancer CC associated with optimal versus poor CC remain relatively unknown.

Another critical barrier to improving care delivery is the gap in measures for cancer care coordination [3]. According to the AHRQ Care Coordination Measures Database and the Measures for Person Centred Coordinated Care [32, 33], many validated measures for primary and in-patient care exist, but very few are designed to evaluate patients’ perception of CC specifically in cancer patients. Moreover, prior research on CC has primarily focused on health systems or providers’ perspectives, and many are designed to measure satisfaction with care rather than CC. A patient-centered approach to care requires a standardized measurement system that captures experiences from the patient perspective as well as providing a model for implementing in clinical research studies [34, 35]. Patient-reports of care coordination may not accurately identify the nature of information exchange within their care team, but patients are able to identify specific coordination processes or activities associated with poor CC experiences such as lack of adequate information regarding next steps in care delivery or confusion during the transitions in care [10]. Although a small number of prior studies have evaluated the measurement of patients’ perspectives of CC [36,37,38], findings are inconsistent. In the CCCQ-P survey developed and validated in Australia, two domains of care coordination are measured [38]; in contrast, CAHPS, which is developed in the US, assesses six dimensions of cancer care that includes 3-item care coordination measure [37]. Existing CC measures have many methodological and application challenges such as lack of rigorous psychometric evaluation, limited range of applicable settings, lengthy questionnaires, and limited generalizability of the survey instrument [4, 35]. Further, these measures were generally developed and geared for use at large institutions and by specialty providers, and only a few studies have examined optimal methods to assess patients’ perspectives of CC in community settings [10]. In the United States, over 75% of patients receive community-based care, which refers to private practice or hospital settings in or near the communities where cancer patients reside [39, 40]. In contrast, smaller proportions of cancer patients receive care in academic institutions or major cancer centers that are located in largely urban areas [40]. Moreover, as indicated by a systematic review by Gorin and colleagues [5], there are substantial variations in the validity and reliability of cancer CC measures.

Taken together, findings from prior work suggest the critical need for adequate, patient-centered CC measures or tools that can be used across diverse practice settings and patient populations. Such a tool may further our understanding of current CC approaches and identify specific CC processes and activities associated with both optimal and poorly coordinated care. A validated, patient CC measure with adequate psychometric properties may facilitate quality improvement efforts by informing strategies for CC improvement and identifying aspects of care that may be specific targets for intervention to improve care delivery.

Present study

In order to address these knowledge gaps, a new survey instrument, a Care Coordination Instrument (CCI), was developed to measure patients’ perception of cancer care coordination across varied healthcare settings and populations. Given the emphasis toward a patient-centered model of care that integrates patients’ perspectives and preferences, a measure that assesses patients’ perspectives of cancer care coordination that can be easily incorporated into existing clinic workflows and patient portals is needed to further efforts to improve cancer care delivery. To that end, we employed a comprehensive approach to develop, evaluate, and refine the CCI as a self-report questionnaire with robust psychometric properties. This instrument was derived from and expands on existing care coordination framework by the AHRQ [32] and includes three domains of care coordination conceptualized as central to patient-centered CC: Communication, Navigation, and Operational. The CCI items were designed to expand the two-main AHRQ framework in order to encompass patients’ preferences and perspectives of CC processes and goals.

The present study describes the development and validation of the CCI conducted in two phases; 1) development and field testing using mixed-methods, and 2) validation and refinement of the CCI survey questionnaire based on focus group interviews with cancer patients. A better understanding of patients’ perception of CC can inform ways to improve cancer care delivery and may benefit oncology providers, hospital administrators, and patients as we work toward developing effective and efficient care delivery models.

Methods

Study 1: instrument development and field testing

The draft CCI items were prepared based on a review of the literature by the research team #1. First, a systematic PubMed search was performed to identify prior work on cancer care coordination measures. Search terms were entered in the following order: 1) “care coordination”; 2) “care coordination AND cancer”, and 3) “care coordination AND cancer AND tool/measure”. Of the 1918 publications identified in the first step using the search term “care coordination”, 151 focused on cancer; among those publications, only 10 publications included the key terms “cancer care coordination”, “tool”, or “measure”. Of the 10 articles, only three referenced a specific, patient-focused care coordination measurement process or instrument. This review confirmed a major gap in this literature, as relatively little research has specifically addressed cancer care coordination measures.

Next, existing CC framework and instruments were reviewed to identify relevant terminology, concepts, and care coordination processes. Information derived from theoretical models and existing care coordination instruments designed for oncology and primary care served as a foundation for the CCI items. Existing measures that were used as starting points for the development of the CCI include the Adapted Picker Institute Cancer Survey [41], Consumer Assessment of Healthcare Providers and Systems (CAHPS) [37], and the Cancer Care Coordination Questionnaire for Patients (CCCP-Q) [36, 38]. The Adapted Picker Institute Cancer Survey is a 34-item questionnaire covering coordination of care as well as confidence in providers, treatment information, and access to care, and is designed to assess patients’ experiences with cancer care and health-related quality of life [41]. The CC section of this survey addresses multidisciplinary communication but does not include transitions in care, goals of care, navigator functions, or coordination with ancillary services. A widely used patient measure, the CAHPS, was originally developed to assess patient satisfaction, and the current CAHPS Cancer Care Survey consists of six core composite measures with 56 questions [37]. Although the CAHPS Cancer Care Survey assesses patient-reports of clinician behaviors, its 3-item Care Coordination scale has weak internal consistency reliability and the lowest reliability among the six domains assessed in this survey [37]. Given that many cancer patients receive a combination of two or more interventions, the CAHPS for Cancer Survey will not capture important components of CC related to communication between different disciplines and across varied health care settings. The CCCQ-P is an Australian measure designed to assess cancer patients’ perception of care coordination [38]. Although many constructs underlying the CCCQ-P are also relevant to cancer CC in the United States, given that there are many differences in cancer care between Australian and US health care systems including a universal health care system with a large parallel private health care system and greater provisions of cancer care by general practitioners and surgeons in Australia [42,43,44], generalizability of the CCCQ-P to US patients is uncertain. In light of these limitations above in existing measures, the CCI was designed to complement ongoing efforts to develop care coordination measurement as well as expand on the existing framework, with greater flexibility in application and utility across varied practice settings and diverse patient populations.

Following instrument development and IRB approval, a series of field tests were conducted at Mount Sinai Hospital, a large academic hospital in New York City. During field testing, focus group discussions and modified cognitive interviews were conducted to obtain feedback from patients and providers on the CCI items. Modified cognitive interviewing is an evidence-based, qualitative method specifically designed to test whether a survey fulfills its intended purpose, and it is commonly used before data collection for initial testing of a survey [45]. Field testing consisted of four independent samples; 1) Review by 20 oncology RNs and NPs, 2) 30 patients, 3) 30 patients, and 4) 60 patients. Each sample was tested sequentially, and between each testing the research team examined patients’ comments and modified the CCI if necessary. The fourth sample included 30 myeloma and 30 GI patients derived from the same hospital but with different specialty providers; thus, this sample served to determine whether the CCI is able to differentiate patients’ perception of CC across different practices. Adult patients (age 18 and over) with any type of cancer receiving active therapy were eligible to participate in the study. Active treatment was defined as having a minimum of three outpatient therapy or cancer care visits within a preceding 6-month period. Given that myeloma and GI patients were seen in the same hospital clinic where the overall infrastructure for clinical practices included common operational aspects such as electronic health record systems, clinic staff, and a call center, it was anticipated that only the Overall, Communication, and Navigation domains scores would differ between the two groups, and that no differences would be found for the Operational domain. Thus, this sample served as initial validation for the three CC domains of the CCI.

Study 2: validation and refinement

In order to increase the generalizability of the CCI to community-based practices including private and hospital-based settings, a focus group study was conducted in Hawai‘i by research team #2. Prior to the first focus group interview, the readability of the CCI was evaluated using the Fleisch-Kincaid Grade Level. Analysis showed that the reading grade level was 10.1, indicating that the CCI was at a high school (10th grade) reading level, and some items may be difficult to read and understand for a general adult population. Accordingly, the structure and wording of the CCI items were examined, and the length of several questions were shortened to improve readability. Re-assessment of the reading level showed that the Fleisch-Kincaid Grade Level was 8.2, which was deemed as acceptable for the target population in this study.

Focus groups were comprised of cancer patients recruited through study flyers posted in waiting rooms at private practice and hospital-based sites, and through presentations at community support groups. Paralleling Study 1 eligibility criteria, adult patients (age 18 and over) with any type of cancer receiving active therapy were eligible to participate in Study 2. Prior to each focus group, participants were asked to review and complete a written informed consent form, a brief demographic questionnaire, and the CCI. Each focus group discussion was facilitated by two co-facilitators and conducted in a private conference room at the cancer center or a hospital. Discussions were semi-structured, with facilitators asking pre-determined questions from the discussion guide developed by the research team (see Additional file 1 for the focus group discussion guide). Open-ended questions were used to assess face and content validity of the CCI and to probe for participants’ feedback on whether any CCI items were unclear, not relevant, or needed refinement. Some of the questions in the discussion guide included “What did you think about the survey?”, “How did this survey capture your experiences with care coordination?”, and “What information related to care coordination that you think might be missing from the current survey?”. In order to assess whether the CCI adequately captured patients’ CC experiences as well as to probe for any additional aspects of CC that were not in the earlier versions of CC that may be relevant to patients receiving community-based care, additional questions on patients’ CC experiences were included in the focus group discussions. Each discussion lasted about 60–90 min, and all discussions were transcribed verbatim for analysis, with participants’ names and other identifying information (e.g., physician or other provider names mentioned during the discussions) removed to ensure anonymity. Following completion of focus groups, the research team reviewed and analyzed the transcripts, and the CCI was modified based on focus group discussions. An additional informal focus group session was conducted following the revision to ensure that the final version of the CCI demonstrated face and content validity and had acceptable readability. The University of Hawai‘i Institutional Review Board approved all study procedures and documents.

Analyses

For Study 1, descriptive analyses (e.g., frequency distribution, means, standard deviations) were conducted at various stages of field testing to identify item(s) requiring modification or refinement. Visual inspections of response patterns served to describe mean differences and identify any atypical response patterns (e.g., skew, ceiling effect). In the fourth sample that compared patients’ responses between two oncology practices, independent-samples t-tests were conducted on overall and three domain scores. For quantitative analysis, the overall score was computed by summing responses across all the CCI items, with higher score indicating patient-reports of better CC.

For Study 2, a qualitative evaluation of the focus group transcripts was undertaken by the research team, using aspects of thematic analysis [46]. First, we openly coded narratives using an inductive approach to thematically characterize participants’ discussions that often ventured beyond the constraints of the existing CCI item domains. For initial analysis, 1–2 transcripts were coded independently by the members of the research team. After this phase, we hypothesized relationships, identified new items and reassigned existing CCI items to their respective domains as driven by the participant narratives. Each researcher identified and nominated common themes and preliminary codes, and all codes were reviewed and discussed by the team until a final consensus was reached. A codebook was subsequently developed and used to drive a subsequent thematic analysis of the additional 1–2 transcript data by each researcher. Final codes and themes were entered in NVivo v.11.4.3 for analysis. Following qualitative evaluation of focus group discussions, descriptive analyses (e.g., frequency distributions, means, standard deviations) were conducted to characterize the study sample and evaluate response patterns for each CCI item.

Results

Study 1: field testing

First, the draft version of the CCI was submitted to RN and NPs for review. Based on their feedback, the CCI was modified to include question boxes for respondents to provide any feedback. Next, the revised CCI was administered to 30 patients. Many patients expressed confusion with the tense of questions; thus, the questions were revised to reflect present tense. Additionally, one question that puzzled nearly all patients was removed. Results of the third field test indicated that two questions had over 50% N/A (not applicable) response rates, suggesting that these questions may not be relevant to many cancer patients and were candidates for removal. Further, participants suggested that several questions may be improved by modifying wording to make them less concrete and more conceptual. For example, based on the feedback, a question “I feel that my providers address my nausea and vomiting” was revised to “I feel that my providers will address any concerns such as nausea and vomiting should I experience this” to improve applicability of the question to a broader cancer patient population. Additionally, in order to improve interpretability of responses, the N/A response option was dropped as this response option does not distinguish optimal versus suboptimal CC according to patients’ perspectives [47]. The “neutral” response option was retained.

In the final field testing at Mount Sinai Hospital, the revised CCI was administered to 60 patients, 30 myeloma and 30 GI cancer patients from separate practices within the same academic hospital setting. Patients receiving care from the GI oncology practice reported significantly better perception of CC than those receiving care from the myeloma practice on Overall (p < .01), Communication (p < .01), and Navigation domain scores (p < .05). Consistent with expectations, there was no significant difference in the Operational domain scores between the two groups. Figure 1 illustrates the mean score comparison for overall and three domain scores between the two groups.

Fig. 1
figure1

Mean score comparisons for overall and 3 domain scores by groups, M = Myeloma, G = GI

The initial version of the CCI following field testing consisted of 27 multiple-choice items rated on a 5-point Likert-scale from Strongly Agree to Strongly Disagree, with a response option “Not Applicable”. The Communication domain assesses information regarding the quality of communication among the various components of the health care team(s) and between the healthcare team and patients. The Navigation domain provides an assessment of patients’ perceptions about global aspects of care beyond the physician’s office or practice site such as information about financial resources, insurance, and social and educational issues. The Operational domain is a unique subscale in the CCI that probes for information regarding how a patient perceives his/her care is coordinated related to access to care, scheduling issues, and efficiency of care delivery. This instrument was designed for use with patients with any cancer type receiving active treatment and across varied healthcare settings. The three domains were developed based on feedback obtained through focus group discussions and cognitive interviews with patients and providers during field testing. Specifically, the Operational domain that is unique to the CCI was deemed as a critical component of CC by cancer patient participants in focus groups and cognitive interviews.

Study 2: validation and refinement

Sixty-eight cancer patients on active therapy participated in seven focus group sessions held between February 2018 and August 2018. As can be seen in Table 1, the mean age of this sample was 61.8, and the majority of participants were female (78%). Participants were ethnically diverse with various cancer types, with a high proportion of women with breast cancer (see Table 1). The majority of participants indicated receiving care in a hospital-based facility (60%). Approximately one-fourth of the participants reported receiving care across multiple settings.

Table 1 Sample characteristics

Based on the final coding, thematic analysis of the focus group transcripts was conducted. Analysis demonstrated the following most prevalent themes: Navigator, Team, Survey, and Communication. Other themes included Access, Insurance, Providers, Support, Advocacy, Education & Knowledge, Advocate, Facility, and Timing. These themes were developed by the research team following independent review of the focus group transcripts. Table 2 presents descriptions/subthemes for each theme and the number of focus groups in which each theme was discussed. Overall, patients expressed concordance with the CCI items and their care coordination experiences, thereby establishing face and content validity of the CCI.

Table 2 Themes and descriptions/subthemes and the number of focus groups in which each theme was discussed

As can be seen in Table 2, the highest proportion of focus group comments was related to the navigator theme. The majority of participants indicated that they did not have access to a patient navigator or were unfamiliar with the term “patient navigator”; thus, a higher than expected proportion of participants’ comments were related to the navigator theme. Regarding the theme on team-based care, many participants expressed confusion with the terms that referred to their care teams such as “my oncology team” or “member of my oncology team,” as they either did not have an oncology team or did not perceive receiving care from a team of providers. Relatedly, many focus group comments were related to survey wording regarding health provider(s). For example, some patients indicated that they did not see an oncologist, as their cancer care was provided by a urologist or another physician. Thus, questions with the wording “team” and “my oncologist” were revised to refer to treating physician(s) as “my cancer doctor” to increase relevancy to actual care. Regarding specific CCI intent, some suggestions from participants included re-wording of some items, use of more concrete/direct language, and an inclusion of a global question to assess overall perceptions of CC. Discussions regarding communication varied widely; some patients expressed challenges with communication with providers, whereas others had favorable experiences, in particular with use of electronic health records.

Participants had unique care coordination experiences, and these experiences were reflected in the comments provided in focus group discussions. Such comments broadly addressed access to treatment, insurance, providers, support, advocacy, education and knowledge. These comments provided additional insights regarding content validity as well as potential areas of refinement of the CCI. For example, patients expressed concordance with questions that probed regarding the availability of appointments and access to providers or their staff during and outside of normal business hours for any questions or complications from treatment. Insurance-related discussions unveiled some unexpected CC challenges that were not included in the initial CCI. The CCI question regarding information on support groups was described by most participants as essential as they were deemed socially and emotionally beneficial, and increased knowledge about the disease and treatment process provided by peers positively impacted patients’ experiences. Regarding advocacy, some participants emphasized the importance of the CCI questions on seeking a second opinion and other treatment-related resources on their own or having to rely on a family member/friend to provide support. Other less commonly discussed themes included advocate, facility issues, and treatment timing, all of which appeared in less than half of the focus groups.

Following qualitative analysis, results of the CCI completed by focus group participants were examined. Visual inspection of the responses revealed that some questions had limited variability or response ranges. For example, responses to some items such as “In general, I know exactly where to go for my appointments” were nearly unanimously “Strongly Agree”, indicating that these items did not have sufficient variability in responses; thus, these items were removed from the final CCI. Further, some items including “I was told what paperwork I needed to bring to my first appointment” had high proportions of missing, neutral, or not applicable written responses. These items were discussed and modified or removed by the research team. The response option “Neutral” was also removed, as responses to this category presented with multiple potential interpretations (e.g., not sure, not applicable, unknown) thereby reducing validity of the survey results. Furthermore, based on focus group suggestions, additional items were incorporated into the questionnaire. These new additional questions were designed to address insurance and financial aspects of cancer care, availability of family member/relative/friend help with care coordination, information about clinical trials, and a global question to assess overall perceptions of CC.

The final CCI questionnaire from Study 2 consists of 29 items rated on a 4-point Likert scale from Strongly Disagree to Strongly Agree. These items load on the three domains as follows: 16 Communication, 14 Navigation, and 10 Operational. Item-domain loadings were nominated by three members of the research team and subsequently finalized by consensus. Some items were specified to load on more than one domain, as some aspects of these domains overlap conceptually. Psychometric evaluation of the CCI version tested in the focus groups demonstrated that it has excellent overall internal consistency reliability, Cronbach’s α = .922. Internal consistency reliability for the three domains were adequate to excellent; Communication α = .916, Navigation α = .793, and Operational α = .738. Reading level analysis of the final CCI demonstrated acceptable reading level, Flesch-Kincaid Grade Level = 8.2.

Discussion

Although the importance of coordinated care has been well-recognized, the paucity of measures to assess cancer care coordination limits efforts to improve cancer care delivery. As patient-centered care delivery research evolves, tools for assessment aimed at improving care delivery are needed. The present study used mixed-methods to develop, validate, and refine a new care coordination instrument, CCI, designed to assess cancer patients’ perception of care coordination across varied healthcare settings and populations. Overall, the results indicated that the CCI has excellent psychometric properties and is a useful tool to assess patients’ perception of cancer care coordination. Findings from qualitative analysis provided a rich, in-depth evaluation of patients’ care coordination experiences, with emerging themes that captured perceptions of cancer CC from diverse individual experiences. Knowledge gained from the use of this tool may provide important first-hand information from patients and identify specific CC processes and activities that may be targets for effective CC intervention efforts.

In the first part of this study, the instrument development process and initial field testing results were described. Questionnaire development is a complex process, and a number of iterative steps were used to test and further refine the instrument across multiple patient samples to improve clarity and increase relevance. Instrument development was informed by existing CC framework and measures, and the CCI was designed to include three domains conceptualized as critical components of CC: Communication, Navigation, and Operational. Expanding on the IOM model, the CCI’s Communication domain was conceptualized to assess communication among health providers as well as between patient and health providers, and it also includes assessment regarding whether patients’ preferences and opinions were incorporated as part of information provision. A departure from existing measures, the CCI includes a unique domain, an “Operational” subscale, that assesses patients’ views on access to care, scheduling, and efficiency. In the final field testing, preliminary findings of discriminant validity of the CCI domains were reported, as evidenced by expected and observed differences in mean scores between two groups receiving specialty care within the same hospital.

Consistent with prior studies, participants in the focus groups expressed various challenges associated with CC including communication and lack of informational resources [22, 28]. Focus group discussions revealed important themes regarding care coordination based on patients’ perspectives and some important distinctions between hypothesized CC based on the literature and actual CC. For example, although team-based approach to care is conceptualized as a gold standard for cancer care [3], many participants in this study did not perceive their care as delivered by a team of health care providers or a cancer care team. Further, despite the growing interest in a patient navigator approach, the majority of participants indicated that they did not have a patient navigator or were unfamiliar with patient navigator programs or resources. These contrasting views may be in part due to prior research focused on systems and providers’ perspectives rather than patients’ [15,16,17,18,19]. The current findings suggest the need to expand cancer CC research to include broader samples of cancer patients receiving care in private, community-based, and other hospital settings.

There are many strengths to the present study. First, this study contributes a new patient-centered instrument for assessment of cancer CC in varied care settings and populations. Expanding on existing CC framework and measures, the CCI has a unique domain, “Operational”, which has not been found in other CC measures. This new domain is conceptualized to assess patients’ views related to access to care and efficiency, and it provides an additional dimension to capture specific CC processes and activities important to patient-centered care. Second, based on the results, the CCI has excellent psychometric properties with internal consistency and good face and content validity. Given that rigorous psychometric evaluations of existing measures designed for US cancer patient population are currently lacking [35], the CCI presents as a promising new tool. Third, the CCI items were modified and refined according to feedback from patients receiving care in an academic setting as well as those receiving community-based care in varied health care settings, increasing generalizability of the CCI to “real-life” cancer care. Importantly, this study has illuminated notable differences between the existing literature and actual CC approaches used in cancer care. Although a multidisciplinary, team-based approach to cancer care in the US is extensively studied and recommended [3, 4, 8, 25], patient-reports of actual care indicated that many do not perceive receiving a team-based care and some patients receive oncology care from non-oncology providers (e.g., urologists, primary care). It is possible that these differences are in part due to that many prior studies on cancer CC were based on systems or providers’ perspectives of CC or focused on specific systems of care. Thus, these and other insights gained from this study provide important groundwork for further research in cancer care coordination.

Conversely, there are several limitations to the present study. First, the study samples were relatively small and comprised primarily of female patients and patients with breast cancer; thus, current findings are preliminary and need further replication in a larger sample. Second, given the nature of the focus group research, participants in Study 2 were limited to those who had volunteered to attend the focus group session at the research sites, were mobile, and had access to transportation. Cancer patients who are inpatient, have difficulty with transportation or mobility, or not well enough to participate in focus groups discussions were not part of this study. It is possible that patients with more advanced stage of the disease, receiving inpatient care, or in poorer health may have additional CC challenges beyond the scope of this study. Third, although the present study utilized patient samples receiving care in academic hospital and community-based settings, it is unknown how the CCI may generalize to closed systems such as Kaiser and Intermountatin systems [48, 49]. Future research will include validation of the CCI in closed systems.

Implications

The present work has many applied and research implications for improving cancer care delivery. As indicated by the IOM’s landmark report on cancer care [3], patient-centered care that incorporates patients’ preferences and experiences requires informed and participatory patients and families. To that end, an instrument with adequate psychometric properties that assesses patients’ perspectives of care coordination is needed to further our understanding of current CC approaches, assess quality of CC, and provide valuable information regarding variables associated with optimal and poorly coordinated care. Such a tool is critical for healthcare professionals and entities to identify potential targets for CC improvement, monitor variations over time, and develop and shape intervention efforts designed to enhance healthcare delivery. Given that lack of rigorous psychometric evaluation of existing measures is one of the current limitations in cancer measures [35], this mixed-methods research to develop and validate a new care coordination instrument is an important first step to establishing an evidence-based, psychometrically sound patient-centered CC measure. The present study expands on the existing literature on cancer measures and provides a novel tool that can be used across varied care settings and populations. Future research using a larger sample will confirm the factor structure and psychometric properties of the CCI and demonstrate the utility of the instrument in a variety of different practice settings and patient populations.

Interestingly, the results of focus group interviews suggest that the existing literature and patient-reports of “real-life” care offer contrasting views of some aspects of cancer CC. These findings underscore the need for further investigation of current CC approaches used in cancer care and incorporation of patients’ perspectives when developing interventions to improve cancer care coordination. Additionally, subgroups of cancer patient populations such as racial and ethnic minority groups experience poorer cancer health outcomes, in part due to poorly coordinated care [30, 31]. The CCI, which has been refined in a racially diverse sample of cancer patients, may be particularly useful in efforts to further understand cancer CC and reduce cancer health disparities across specific race/ethnic populations. An important direction for future research is to examine utility of the CCI in African American and Latino cancer patients.

Conclusions

Improving cancer care coordination is a priority. Many gaps in the literature including lack of adequate measures to assess patients’ perspectives of cancer care coordination hamper current efforts to improve cancer care delivery. A patient-centered approach to cancer care requires a psychometrically sound measure to assess care coordination from patients’ perspectives. Overall, the current findings are encouraging in that the new instrument, CCI, has demonstrated excellent reliability and validity in both academic and community-based samples. Use of this tool in applied clinical settings may provide health care providers and health care entities with capabilities to assess optimal CC strategies, monitor the effectiveness of CC intervention, and identify ways to improve cancer care delivery for all cancer patients.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Any information made available will be fully de-identified.

Abbreviations

AHRQ:

Agency for Healthcare Research and Quality

CAHPS:

Consumer Assessment of Healthcare Providers and Systems

CC:

Care Coordination

CCCP-Q:

Cancer Care Coordination Questionnaire for Patients

CCI:

Care Coordination Instrument

GI:

Gastrointestinal

IOM:

Institute of Medicine

NP:

Nurse Practitioner

RN:

Registered Nurse

References

  1. 1.

    McDonald KM, Sundaram V, Bravata DM, Lewis R, Lin N, Kraft SA, et al. Closing the quality gap: a critical analysis of quality improvement strategies (Vol. 7: Care Coordination). 2007.

  2. 2.

    Benito L, de la Cueva AL, Delgado-Hito P, Martinez Momblan MA, Romero Garcia M, Garcia M. Identifying coordination and continuity of care indicators for population-based Cancer screening programs: a Delphi study. Nurs Res. 2018;67(5):411–8.

    PubMed  Article  Google Scholar 

  3. 3.

    Committee on Improving the Quality of Cancer Care: Addressing the Challenges of an Aging P, Board on Health Care S, Institute of M. In: Levit L, Balogh E, Nass S, Ganz PA, editors. Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis. Washington (DC): National Academies Press (US). 2013.

  4. 4.

    Weaver SJ, Jacobsen PB. Cancer care coordination: opportunities for healthcare delivery research. Transl Behav Med. 2018;8(3):503–8.

    PubMed  PubMed Central  Article  Google Scholar 

  5. 5.

    Gorin SS, Haggstrom D, Han PKJ, Fairfield KM, Krebs P, Clauser SB. Cancer care coordination: a systematic review and meta-analysis of over 30 years of empirical studies. Ann Behav Med. 2017;51(4):532–46.

    PubMed  Article  Google Scholar 

  6. 6.

    Holmes JA, Anderson RF, Hoffman LG, Showalter TN, Kasibhatla M, Collins SP, et al. Cardiovascular preventive care and coordination of Care in Prostate Cancer Survivors: a multi-institutional prospective study. Int J Radiat Oncol Biol Phys. 2019;103(1):112–5.

    PubMed  Article  Google Scholar 

  7. 7.

    Hussain T, Chang HY, Veenstra CM, Pollack CE. Fragmentation in specialist care and stage III colon cancer. Cancer. 2015;121(18):3316–24.

    PubMed  PubMed Central  Article  Google Scholar 

  8. 8.

    Page JS, Lederman L, Kelly J, Barry MM, James TA. Teams and teamwork in Cancer care delivery: shared mental models to improve planning for discharge and coordination of follow-up care. J Oncol Pract. 2016;12(11):1053–8.

    PubMed  Article  Google Scholar 

  9. 9.

    Shrank WH, Rogstad TL, Parekh N. Waste in the US health care system: estimated costs and potential for savings. JAMA. 2019;322(15):1501–9.

    Article  Google Scholar 

  10. 10.

    Clauser SB, Wagner EH, Aiello Bowles EJ, Tuzzio L, Greene SM. Improving modern cancer care through information technology. Am J Prev Med. 2011;40(5 Suppl 2):S198–207.

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Holcombe RF, Cortese M, Zinghini I, Mahida R, Draluk D, Ru M, et al. Development of an oncology-specific instrument to measure care coordination. Proc. Am Soc Clin Oncol. 2017;35(8 suppl):148.

    Article  Google Scholar 

  12. 12.

    Alsamarai S, Yao X, Cain HC, Chang BW, Chao HH, Connery DM, et al. The effect of a lung cancer care coordination program on timeliness of care. Clin Lung Cancer. 2013;14(5):527–34.

    CAS  PubMed  Article  Google Scholar 

  13. 13.

    Chumbler NR, Kobb R, Harris L, Richardson LC, Darkins A, Sberna M, et al. Healthcare utilization among veterans undergoing chemotherapy: the impact of a cancer care coordination/home-telehealth program. J Ambul Care Manage. 2007;30(4):308–17.

    PubMed  Article  Google Scholar 

  14. 14.

    Jackson GL, Zullig LL, Phelan SM, Provenzale D, Griffin JM, Clauser SB, et al. Patient characteristics associated with the level of patient-reported care coordination among male patients with colorectal cancer in the veterans affairs health care system. Cancer. 2015;121(13):2207–13.

    PubMed  PubMed Central  Article  Google Scholar 

  15. 15.

    Bickell NA, Young GJ. Coordination of care for early-stage breast cancer patients. J Gen Intern Med. 2001;16(11):737–42.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  16. 16.

    Brazil K, Bainbridge D, Sussman J, Whelan T, O'Brien MA, Pyette N. Coordination of palliative cancer care in the community: "unfinished business". Support Care Cancer. 2009;17(7):819–28.

    PubMed  Article  Google Scholar 

  17. 17.

    Easley J, Miedema B, Carroll JC, Manca DP, O'Brien MA, Webster F, et al. Coordination of cancer care between family physicians and cancer specialists: importance of communication. Can Fam Physician. 2016;62(10):e608–e15.

    PubMed  PubMed Central  Google Scholar 

  18. 18.

    Jagsi R, Abrahamse P, Morrow M, Hamilton AS, Graff JJ, Katz SJ. Coordination of breast cancer care between radiation oncologists and surgeons: a survey study. Int J Radiat Oncol Biol Phys. 2012;82(5):2072–8.

    PubMed  Article  PubMed Central  Google Scholar 

  19. 19.

    Klabunde CN, Haggstrom D, Kahn KL, Gray SW, Kim B, Liu B, et al. Oncologists' perspectives on post-cancer treatment communication and care coordination with primary care physicians. Eur J Cancer Care (Engl). 2017;26(4):1–9.

  20. 20.

    Hannon B, Swami N, Krzyzanowska MK, Leighl N, Rodin G, Le LW, et al. Satisfaction with oncology care among patients with advanced cancer and their caregivers. Qual Life Res. 2013;22(9):2341–9.

    PubMed  Article  Google Scholar 

  21. 21.

    McAlearney AS, Murray K, Sieck C, Lin JJ, Bellacera B, Bickell NA. The challenge of improving breast Cancer care coordination in safety-net hospitals: barriers, facilitators, and opportunities. Med Care. 2016;54(2):147–54.

    PubMed  PubMed Central  Article  Google Scholar 

  22. 22.

    Patel MI, Periyakoil VS, Blayney DW, Moore D, Nevedal A, Asch S, et al. Redesigning Cancer care delivery: views from patients and caregivers. J Oncol Pract. 2017;13(4):e291–302.

    PubMed  PubMed Central  Article  Google Scholar 

  23. 23.

    Brennan ME, Butow P, Spillane AJ, Boyle F. Patient-reported quality of life, unmet needs and care coordination outcomes: moving toward targeted breast cancer survivorship care planning. Asia Pac J Clin Oncol. 2016;12(2):e323–31.

    PubMed  Article  Google Scholar 

  24. 24.

    Hawley ST, Janz NK, Lillie SE, Friese CR, Griggs JJ, Graff JJ, et al. Perceptions of care coordination in a population-based sample of diverse breast cancer patients. Patient Educ Couns. 2010;81(Suppl):S34–40.

    PubMed  Article  Google Scholar 

  25. 25.

    Ghoncheh M, Pournamdar Z, Salehiniya H. Incidence and mortality and epidemiology of breast Cancer in the world. Asian Pac J Cancer Prev. 2016;17(S3):43–6.

    PubMed  Article  Google Scholar 

  26. 26.

    Till JE. Evaluation of support groups for women with breast cancer: importance of the navigator role. Health Qual Life Outcomes. 2003;1(1):16.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Beesley VL, Janda M, Burmeister EA, Goldstein D, Gooden H, Merrett ND, et al. Association between pancreatic cancer patients' perception of their care coordination and patient-reported and survival outcomes. Palliat Support Care. 2018;16(5):534–43.

    PubMed  Article  Google Scholar 

  28. 28.

    Kedia SK, Ward KD, Digney SA, Jackson BM, Nellum AL, McHugh L, et al. 'One-stop shop': lung cancer patients' and caregivers' perceptions of multidisciplinary care in a community healthcare setting. Transl Lung Cancer Res. 2015;4(4):456–64.

    PubMed  PubMed Central  Google Scholar 

  29. 29.

    Lee SJC, Jetelina KK, Marks E, Shaw E, Oeffinger K, Cohen D, et al. Care coordination for complex cancer survivors in an integrated safety-net system: a study protocol. BMC Cancer. 2018;18(1):1204.

    PubMed  PubMed Central  Article  Google Scholar 

  30. 30.

    Elliott TE, Elliott BA, Regal RR, Renier CM, Haller IV, Crouse BJ, et al. Improving rural cancer patients' outcomes: a group-randomized trial. J Rural Health. 2004;20(1):26–35.

    PubMed  Article  Google Scholar 

  31. 31.

    Sprague Martinez L, Freeman ER, Winkfield KM. Perceptions of Cancer care and clinical trials in the black community: implications for care coordination between oncology and primary care teams. Oncologist. 2017;22(9):1094–101.

    PubMed  PubMed Central  Article  Google Scholar 

  32. 32.

    AHRQ. Care Coordination Measures Atlas Update. Chapter 3: Care Coordination Measurement Framework. 2014 [Available from: http://www.ahrq.gov/professionals/prevention-chronic-care/improve/coordination/atlas2014/chapter3.html. Accessed 1 July 2019.

  33. 33.

    Lloyd H, Wheat H, Horrell J, Sugavanam T, Fosh B, Valderas JM, et al. Patient-reported measures for person-centered coordinated care: a comparative domain map and web-based compendium for supporting policy development and implementation. J Med Internet Res. 2018;20(2):e54.

    PubMed  PubMed Central  Article  Google Scholar 

  34. 34.

    Basch E, Snyder C, McNiff K, Brown R, Maddux S, Smith ML, et al. Patient-reported outcome performance measures in oncology. J Oncol Pract. 2014;10(3):209–11.

    PubMed  PubMed Central  Article  Google Scholar 

  35. 35.

    Tzelepis F, Rose SK, Sanson-Fisher RW, Clinton-McHarg T, Carey ML, Paul CL. Are we missing the Institute of Medicine’s mark? A systematic review of patient-reported outcome measures assessing quality of patient-centred cancer care. BMC Cancer. 2014;14(1):41.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36.

    Collett GK, Durcinoska I, Rankin NM, Blinman P, Barnes DJ, Anderiesz C, et al. Patients' experience of lung cancer care coordination: a quantitative exploration. Support Care Cancer. 2019;27(2):485–93.

    PubMed  Article  Google Scholar 

  37. 37.

    Evensen CT, Yost KJ, Keller S, et al. Development, testing, and final content of a survey of patient experience with cancer care. J Clin Onc. 2017;35(8_suppl):227.

  38. 38.

    Young JM, Walsh J, Butow PN, Solomon MJ, Shaw J. Measuring cancer care coordination: development and validation of a questionnaire for patients. BMC Cancer. 2011;11:298.

    PubMed  PubMed Central  Article  Google Scholar 

  39. 39.

    American Society of Clinical Oncology. The state of cancer care in America, 2017: a report by the American Society of Clinical Oncology. J Onc Prac. 2017;13(4):e353–e94.

    Article  Google Scholar 

  40. 40.

    RA. O. Evolution of a community-based regional program of excellence in cancer care. J Lancaster Gen Hosp 2009;4(1):28–31.

  41. 41.

    Ayanian JZ, Zaslavsky AM, Guadagnoli E, Fuchs CS, Yost KJ, Creech CM, et al. Patients' perceptions of quality of care for colorectal cancer by race, ethnicity, and language. J Clin Oncol. 2005;23(27):6576–86.

    PubMed  Article  Google Scholar 

  42. 42.

    Durcinoska I, Young JM, Solomon MJ. Patterns and predictors of colorectal cancer care coordination: a population-based survey of Australian patients. Cancer. 2017;123(2):319–26.

    PubMed  Article  Google Scholar 

  43. 43.

    Alfano CM, Jefford M, Maher J, Birken SA, Mayer DK. Building personalized Cancer follow-up care pathways in the United States: lessons learned from implementation in England, Northern Ireland, and Australia. Am Soc Clin Oncol Educ Book. 2019;39:625–39.

    PubMed  Article  Google Scholar 

  44. 44.

    Morrell S, Young J, Roder D. The burden of cancer on primary and secondary health care services before and after cancer diagnosis in New South Wales. Australia BMC Health Serv Res. 2019;19(1):431.

    PubMed  Article  Google Scholar 

  45. 45.

    Miller K WS, Chepp V, Padilla JL. Cognitive Interviewing Methodology: A Sociological Approach for Survey Question Evaluation. 2014.

  46. 46.

    Patton MQ. Qualitative research and evaluation methods. 3rd ed. London: Sage Publications; 2002.

    Google Scholar 

  47. 47.

    Pedhazur EJ, Schmelkin LP. Measurement, design, and analysis: an integrated approach: psychology press; 2013.

    Google Scholar 

  48. 48.

    Shapiro J, Smith S. Lessons for the NHS from Kaiser Permanente. BMJ. 2003;327(7426):1241–2.

    PubMed  PubMed Central  Article  Google Scholar 

  49. 49.

    Institute of Medicine Roundtable on Evidence-Based M. The National Academies Collection: Reports funded by National Institutes of Health. In: Olsen LA, Aisner D, McGinnis JM, editors. The Learning Healthcare System: Workshop Summary. Washington (DC): National Academies Press (US). National Academy of Sciences; 2007.

Download references

Acknowledgements

Not applicable.

Funding

This study was supported in part by P30CA071789 and the University of Hawaii Cancer Center. Funding agencies were not involved in any aspect of the design, data collection, analysis, interpretation of the data, and writing of the manuscript.

Author information

Affiliations

Authors

Contributions

RFH designed and led Study 1 and 2 presented in the manuscript and conducted all aspects of Study 1. RFH, IO, and KC conducted focus group interviews, administered surveys, and analyzed the qualitative data for Study 2. IP and IO analyzed the quantitative data for Study 2. IO and RFH drafted the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Izumi Okado.

Ethics declarations

Ethics approval and consent to participate

All procedures performed in our studies involving human participants were in accordance with the ethical standards of the institutional research committees and with the 1964 Helskinki declaration and its later amendments or comparable ethical standards. The studies were reviewed and approved by the Icahn School of Medicine at Mount Sinai and the University of Hawaii Institutional Review Boards. Written informed consent was obtained from all individual participants included in the present research.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Focus group discussion guide.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Okado, I., Cassel, K., Pagano, I. et al. Development and psychometric evaluation of a questionnaire to measure cancer patients’ perception of care coordination. BMC Health Serv Res 20, 52 (2020). https://doi.org/10.1186/s12913-020-4905-4

Download citation

Keywords

  • Care coordination
  • Cancer
  • Instrument
  • Questionnaire
  • Psychometric evaluation