Skip to main content
  • Research article
  • Open access
  • Published:

Validation of the care providers version of the Rainbow Model of Integrated Care-measurement tool in Chinese primary care systems

Abstract

Background

The original Rainbow Model of Integrated Care Measurement Tool (RMIC-MT) is based on the Rainbow Model of Integrated Care (RMIC), which provides a comprehensive theoretical framework for integrated care. To translate and adapt the original care provider version of the RMIC-MT and evaluate its psychometric properties by a pilot study in Chinese primary care systems.

Methods

The translation and adaptation process were performed in four steps, forward and back-translation, experts review and pre-testing. We conducted a cross-sectional study with 1610 community care professionals in all 79 community health stations in the Nanshan district. We analyzed the distribution of responses to each item to study the psychometric sensitivity. Exploratory factor analysis with principal axis extraction method and promax rotation was used to assess the construct validity. Cronbach’s alpha was utilized to ascertain the internal consistency reliability. Lastly, confirmation factor analysis was used to evaluate the exploratory factor analysis model fit.

Results

During the translation and adaptation process, all 48 items were retained with some detailed modifications. No item was found to have psychometric sensitivity problems. Six factors (person- & community-centeredness, care integration, professional integration, organizational integration, cultural competence and technical competence) with 45 items were determined by exploratory factor analysis, accounting for 61.46% of the total variance. A standard Cronbach’s alpha of 0.940 and significant correlation among all items in the scale (> 0.4) showed good internal consistency reliability of the tool. And, the model passed the majority of goodness-to-fit test by confirmation factor analysis.

Conclusions

The results showed initial satisfactory psychometric properties for the validation of the Chinese RMIC-MT provider version. Its application in China will promote the development of people-centered integrated primary care. However, further psychometric testing is needed in multiple primary care settings with both public and private community institutes.

Peer Review reports

Background

Health systems in some low- and middle-income countries, as well as in most high-income countries, face the challenges of aging populations and rising chronic disease prevalence [1, 2]. These growing challenges call for more integrated approaches instead of the current single-disease and acute-care-focused health care systems. The World Health Organization (WHO) acknowledges integrated care in its vision and global strategy for health care delivery [3]. There has been a proliferation of integrated care initiatives in many different countries and settings [4]. Furthermore, some researches have demonstrated that integrated care contributes to improving the 3 components of Triple Aim, improved population health, improved individual experiences of care and reducing costs of per capita care [5,6,7]. While the aims are promising, integrated care remains a complex health intervention involving multiple levels of organizations with multiple care providers, multiple interventions, and multiple contextual factors that can influence processes and outcomes of care delivery [8, 9]. Measuring care providers’ experience and behavior in a consistent way is critical for evaluation of the implementation of integration interventions and advancement of the success of health care integration [10], since multiple care providers are the final driver of providing integrated care.

Several instruments have been developed for the assessment of integrated care [11,12,13]. For example, Suter et al., identified 114 instruments, over half of which were self-reporting questionnaires that measured care coordination and patient and family involvement, by a knowledge synthesis of indicators and measurement tools for health system integration [12]. Bautista et al. conducted a comprehensive systematic review based on the Rainbow Model of Integrated Care (RMIC) [11]. They also confirmed that patient-focused dimensions, such as patient-centered care and care integration, were the most common measuring constructs in majority of studies. Furthermore, they found that less than half of the instrument validation studies were of good quality for the measurement of properties. Only a few research reports assessed integrated care from the perspective of healthcare professionals [14,15,16]. Stephenson et al., performed a rapid review of all quantitative surveys and qualitative research studies to assess healthcare professionals’ experiences with integrated care [14]. They reported that the common dimensions of existing quantitative surveys were communication, agreement on clear roles and responsibilities, facilities, information systems, coordination of care and access, which missed some deeper aspects affecting teamwork such as trust between and among providers, and management in the institutes [17]. In summary, there are three deficiencies of the existing instruments. First, it remains an area for potential validated instruments to assess integrated care from the perspective of multiple care providers. Second, organizational and system integration dimensions and normative enabling factors are not addressed in current instruments. Third, weak psychometric validity and reliability measurement of current instruments.

The RMIC was developed as a result of literature reviews in primary care settings, and was validated by a series of Delphi panels and a panel of international experts [18]. Subsequently, the Triple Aim framework was synthesized into the model by Delphi studies with an interdisciplinary panel of experts [19]. The RMIC distinguishes four integrated care dimensions (clinical integration, professional integration, organizational integration, system integration), two enablers (functional integration, and normative integration) at micro- meso- and macro-levels, two guiding principles of integration (person-focused care and population-based care), and the three interrelated outcome dimensions (population health, experience of care and cost). The RMIC provides a comprehensive framework for integrated care from the perspectives of both patients and care providers, combining the implementation and outcome measurement of integrated care. Based on the RMIC and more than 300 integrated care instruments for healthcare providers and patients, the RMIC-MT (measurement tool) was developed. In addition to micro-level measurements in previous quantitative surveys, RMIC-MT care provider version adopted items at meso- and macro-levels, such as regional healthcare context and performance management, which may more deeply influence the implementation of healthcare integration [20].

The RMIC-MT was developed based on a literature review and two international Delphi studies. The preliminary version of the RMIC-MT provider version has been tested in the Netherlands, Australia, and Singapore [21,22,23,24]. These studies indicated that further research was needed to improve the psychometric properties of the professional, organizational, system, functional and population-based scales. Therefore, in international validation across 19 countries has been conducted in 2017 [25]. The results of this study indicated that the internal consistency, reliability, and construct validity of the RMIC-MT (36 items, 9 subscales) provider version was good. These results showed that the RMIC-MT provider version is a valuable psychometric tool for evaluating integrated care initiatives in various countries. This study aimed to validate the Chinese RMIC-MT care provider version in the context of the implementation of integrated primary care [18]. Application of this tool could promote regular evaluation of integrated care and further implementation of integrated health systems in China.

Methods

The English RMIC-MT care provider version as used in the international validation study was tested for its validation in Chinese primary health systems [25, 26]. The validation was conducted in two phases, the translation and adaptation process, and the assessment of psychometric properties (Fig. 1).

Fig. 1
figure 1

Study design

The structure and scoring of the original RMIC-MT care provider version

Based on the dimensions in the Rainbow Model, The original RMIC-MT care provider version consists of 48 items grouped into eight dimensions: (1) Person-centeredness (5 items), (2) Community-centeredness (4 items), (3) Clinical Integration (7 items), (4) Professional Integration (7 items), (5) Organizational Integration (4 items), (6) System Integration (3 items), (7) Technical competence (10 items), (8) Cultural competence (8 items). Items in dimensions 1, 2, 5, 6 are answered on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree”, with rating scores from 1 to 5. Items in the dimensions 3, 4, 7, and 8 are answered on a 5-point Likert scale ranging from “never” to “always”, with rating scores from 1 to 5. There are five reverse questions (items 17, 18, 19, 22, 23) in the dimension “Professional Integration”, which are also answered on a 5-point Likert scale ranging from “never” to “always” with rating scores from 5 to 1. The total RMIC-MT care provider score is computed by summing the scores on each item, with a maximum score of 240 points. Providers were also asked to rate the overall perceived ability to coordinate care internally and externally on a -point scale ranging from very poor (1) to excellent (10). The two questions were how did you rate coordination among CCPs inside and outside your CHS. In addition, the adaptive reserve of the CHS was assessed using the resource and culture subscale on a five-point Likert scale derived from the work of Helfrich et al. [27].

The translation and adaptation process

According to Harkness and Streiner [28, 29], we utilized a four-step systematic approach for the translation and adaptation process: forward translation, back-translation, expert reviews, and pre-testing of content validity.

Forward translation

Two postgraduate students with Chinese as their first language, and majoring in Health Economics and Policy (HEP), independently translated the RMIC-MT-Staff into Chinese independently. By comparing the two forward translated versions, the author (X.W.) reviewed the differences and discussed them with the two translators until they all agreed on a reconciled Chinese version 1.0 (CV1.0).

Back-translation

The CV1.0 was translated back into English by two PhD candidates majoring in HEP, who had never read the original English version. Both translators are bilingual translators with English as their first language. The back-translated and original English versions were compared and discussed with the author (X.W.) and the two translators with the aim of reaching satisfactory equivalence between EV2.0 and EV1.0. CV2.0 was agreed based on modification after the discussion.

Expert review

Semi-structured interviews were conducted independently with four Chinese experts working on integrated care or primary care to obtain their reflections on the suitability of the CV2.0 of the RMIC-MT care provider version for use and the reasons underpinning the responses. The CV3.0 was obtained based on the experts’ reflections.

Pre-testing for assessment of face and content validity

Pre-testing was conducted with two policy makers and 12 community care professionals (CCPs). Through a face-to-face interview, each participant was asked to review each item on the CV3.0, and comment on wording, the length of the survey, and the relevance of the items in the Chinese setting. The relevant items were rated as 1 and the unnecessary items rated as 0, and a content validation index was calculated to test content validity of the CV3.0. After the pre-testing phase, two researchers (X.W. and P.V.) discussed the final modification C-RMIC-MT care provider version.

The pilot test of the C-RMIC-MT care provider version

Sampling and data collection

After the translation and adaptation process, the final version was used in a pilot study that launched the establishment of a medical consortium. In April 2017, the General Office of the State Council issued a guideline for constructing medical consortia [30], making them a main means for achieving people-centered integrated care. As a response to the new policy, policymakers in Nanshan began construction of a medical consortium with the purpose of strengthening cooperation between hospital care and community care, especially enhancing community care. There are 79 community health stations (CHSs) in the Nanshan district under the unified management of the medical consortium. In these 79 stations are 1784 working CCPs, including general practitioners (GP), nurses, public health physicians and the others. CCPs of the 79 CHSs, who were at work in the last three months and able to complete online questionnaire independently, were invited to participant in the survey. Data were collected by using an online data collection tool “SO JUMP [31]”. In July 2018, links to the online questionnaire were sent to all potential participants’ mailboxes through an automatic office system. Once the online questionnaire was completed, it was re-submitted to “SO JUMP”. After logging in, authors could immediately download all original data.

Data analysis

Scales with more than 30% missing data were excluded from the analysis. After the cleaned data were entered, psychometric sensitivity, construct validity, concurrent validity and internal consistency were analyzed for the assessment of psychometric properties of the C-RMIC-MT care provider version by SPSS 20 and Amos.

Psychometric sensitivity

Distribution of responses to each item was analyzed for the study of psychometric sensitivity. Items with skewness value > 3 and kurtosis value > 7 [32], or items with a floor or ceiling effects of > 75% of respondents, were considered for deleting as sensitivity problems [33].

Validity

The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity were used first to determine if exploratory factor analysis (EFA) could be conducted [34, 35]. With a KMO value over 0.8 and a significant Bartlett’s test, EFA analysis could be conducted to assess construct validity [36]. EFA with principal axis factoring extraction method and promax rotation were used to assess the underlying structure. EFA and promax rotation in this study followed the description by Brown [37, 38]. The number of factors was determined by consideration of the eigenvalue (> 1), scree plot, and interpretability of the factor. More importantly, the factors retained had to be guided theoretically. Names were given for each identified factor based on the dimensions of the RMIC. Items that cross-loaded on more than one factors were placed with the factor that was most closely related conceptually. Items with poor factor loading (< 0.4) were removed from the questionnaires.

Construct validity was assessed by the calculation of Pearson’s correlations between the scale scores and two overall perceived coordination questions within the questionnaires. Moderately positive associations (≥ 0.4) between the score of the scale and these correlations would indicate good construct validity [39]. The following hypotheses were tested: 1) care providers who indicate a better care coordination ability are more satisfied with the internal adaptive reserve and external care coordination ability; and 2) each subscale aims to measure the CHS coordinated care ability and are therefore positively and significantly correlated with other subscales. P-value < 0.05 were considered statistically significant.

Internal consistency reliability

Based on potential modification in the above two phases, internal consistency reliability was assessed by items-total correlations and Cronbach’s alpha. Items-total correlations coefficients between items within a scale should be ≥ 0.4 [40]. If Cronbach’s alpha ranged between 0.70 and 0.95, the scale was considered reliable for use in the sample population [41]. Moreover, Pearson correlation coefficients were calculated to assess whether each item was correlated the highest on an assigned subscale by correlation of items with the subscale means. If no correlation was achieved, the item would be eliminated [42].

Goodness to fit factor model

Confirmation factor analysis was used to evaluate the explorative factor analysis model fit by using the standard fit indices: root-mean-square error of approximation RMSEA (≤0.06), standardized root-mean-square residual (SRMR) (≤0.08), comparative fit index (CFI) (≥0.80), Tucker-Lewis index (TLI) (≥0.80).

Results

The translation and adaptation process

The back-translation and the original version were basically identical. Only a few words were modified because it was difficult to find words in Chinese conveying the same meaning. For example, the word “coordinate” was replaced by “borrow or rent” in item 26, and by “expert consultation” in item 27. ‘Coordinate’ is a word not commonly used in daily life and has general meaning in Chinese. The distinction between “discipline” and “professional” is not familiar in Chinese. During the expert review, four experts gave some suggestions fitting Chinese primary care systems. For example, they suggested taking GPs and public health physicians as members of a multidisciplinary team, rather than psychologist and dietitian in item 15 of the original version. They suggested deleting “transfer” in item 11, since it is not the health institutes’ responsibility to transfer patients except in an emergency. Additionally, the experts suggested reordering the ten dimensions in the original version, to make it more easily understood for completion by Chinese-speaking individuals. In the pre-testing, the content validity analysis showed a content validation index of 0.84, which is slightly higher than the recommended level of 0.80. Some of the 14 participants in the pre-testing replied that they had difficulty in imagining the specific scenario described in the question, so they suggested more detailed examples, such as a specific action or regulation. After discussions with the developer (P.V.) of the original RMIC-MT care providers version, the author (X.W.) kept all 48 items and made some detailed modifications (Table 1).

Table 1 Detailed modifications in the translation and adaptation process

Psychometric results

Among all 1784 CCPs in 79 CHSs of Nanshan district, 1610 (90.4%) replied to our invitation. Table 2 summarizes the characteristics of the participants.

Table 2 Characteristics of participants

Psychometric sensitivity

Distribution analysis of responses to each item showed that there was no item with a skewness value > 3 or kurtosis > 7, and there was no item with a floor or ceiling effect of > 75%. No item was deleted because of psychometric sensitivity.

Validity

Construct validity was assessed by EFA. The KMO value of 0.957 and significant Bartlett’s test met the requirements for factor analysis. In the EFA, six factors with 45 items were determined by eigenvalues (> 1), accounting for 58.60% of the total variance (Table 3).

Table 3 Eigenvalue and variance contribution rate of each factor

Hence, a six-factor solution was obtained. Factor 1 was named “Care integration” (14 items, 32.03% of variable), factor 2 “person- & community-centeredness” (9items, 10.21% of variable), factor 3 “Cultural competence” (8 items, 6.87% of variable), factor 4 “Professional integration” (5 items, 4.52% of variable), factor 5 “Organizational integration” (4 items, 2.85% of variable), factor 6 “Technical competence” (5 items, 2.11% of variable). Items 33,34, and 43 with poor factor loading (< 0.4) were removed from the questionnaires.

All subscales of the C-RMIC-MT provider version were positively and significantly correlated with each other, see Table 4. In addition, Pearson’s correlations between the C-RMIC-MT care provider version total score and two overall perceived coordination questions showed good construct validity. The correlation coefficient between total score and the question “How do you rate coordination among CCPs inside your CHS?” was 0.531(p < 0.01), while that between total score and the question “How do you rate coordination among CCPs outside your CHS?” was 0.554(p < 0.01). In addition, care providers who indicated a better care coordination ability were more satisfied about the adaptive reserve (r = 0.69, p < 0.01) .

Table 4 Correlation between subscale scores C-RMIC-MT provider version (n = 1610)

Internal consistency reliability

The internal consistency analysis showed that the six dimensions of the C-RMIC-MT provider version could be reliably measured. Correlation coefficients between/among all items in the scale were > 0.4. Items scale correlations showed that all individual items highly correlated with their respective subscale compared with the competing scales. The C-RMIC-MT provider version is also a reliable instrument (Cronbach’s alpha 0.940) of 45 items to measure integrated care.

Goodness to fit factor model

A structural equation model with maximum likelihood evaluated he proposal model fit with RMSEA 0.056, SRMR 0.061, CFI 0.857, TLI 0.848. The model passed the majority of goodness-to-fit test by confirmation factor analysis.

Discussion

Changing the behaviors of health professions is an essential precondition for achieving integrated care. Given this, it is essential for researchers and policy-makers to measure and evaluate the implementation of integrated care from the perspective of care providers. This study provides the first assessment of the validity and reliability of the C-RMIC-MT provider version. It was used to measure integrated primary care from the perspective of CCPs.

Statistical analysis indicated that the construct validity and internal consistency for C-RMIC-MT provider version (45 items, 6 subscale) were good. The proposed model also passed the majority of goodness-to-fit test. This suggests that the C-RMIC-MT provider version is a valuable psychometric tool for evaluating care integration perceived by care providers.

The C-RMIC-MT provider version yielded six dimensions reflecting parts of hypothesized dimensions of the RMIC (person- & community- centeredness, care integration, professional integration, organizational integration, technical competence, and cultural competence). The factor analysis leads us to conclude that care provider in CHSs of China don’t differentiate original clinical, organizational and system integration dimensions of integrated care. Care providers in previous study also found it difficult to differentiate between the organizational and system dimensions of RMIC [43]. The items hypothesized to belong to Person-centeredness dimension were absorbed by the community-centeredness dimension, which might reflect that care providers in CHSs of Chinese health systems don’t recognize it and differentiate between person-centeredness and community-centeredness.

The validation of RMIC-MT provider version in China primary care setting show different results comparing with its validation in 19 countries [25], which reflects cultural backgrounds. Most of the variance of C-RMIC-MT provider version was explained by care integration, while most of the variance in international validation was explained by cultural competence. It also highlights the importance of person and community-orientation (10.21% of variable) as a principle of integrated care, which was highlighted by the validation in 19 countries.

Strengths of the tool and limitations of the study

This study reports the first validation of the C-RMIC-MT care provider version. In summary, this instrument has three highlights. First, based on the Rainbow Model, it is not only easy to use but also focuses on comprehensive dimensions comparing with previous international integrated care instruments. A recent systematic review found that none of the instruments for measuring integrated care assessed system or normative integration [11]. The C-RMIC-MT care provider version filled this measurement gap. Second, it measures integrated care from the perspective of care providers, different from traditional patient’s perspective. Third, as the first Chinese instrument to measure integrated care, it shows good psychometrics properties in Chinese primary care settings. It may be useful as a tool to monitor the degree of integrated care in a region over time, providing evidence of strategy adjustment.

However, the study had some limitations. First, while the number of respondents met the requirement (over 10 times the item number) and the response rate was very high, but only stakeholders from the public CHSs in a single district were presented. As mentioned, high homogeneity of the sampled CHSs might influence the validity assessment. Future studies with diverse samples crossing regions would be needed to further test the psychometric properties for the Chinese primary care context. Second, this was a computer-based assessment. Evidence suggests that a model of administration (such as computer-based vs pencil version) has little impact on psychometric properties of measures [44, 45]. But authors do lose the chance to explore reasons for the answers of respondents when the data are collected by computers rather than collected by paper questionnaire face-to-face. Therefore, we conducted six interviews with CCPs in six CHSs after data collection by computer, including two GPs, two nurses, a public health physician, and an internist, to find the influencing factors of professionals’ experiences with integrated care and potential effective collaboration mechanisms. It is necessary to have more communication with CCPs in further psychometric assessments of the C-RMIC-MT care provider version.

Conclusions

Value-based integrated health systems with strong primary care are being built in the whole of China. Primary integrated care has the potential to achieve the triple aim, but implementation determines the final practical results. The Chinese RMIC-MT care provider version initially validated in this study exhibits good psychometric properties in Chinese primary context. Its application in China could contribute to the measurement of different dimensions of integrated primary care from the perspective of health professions, revealing problems during the implementation of related reforms, and then promoting the building of high-performing heath systems. Moreover, its validation and application in European countries may help in establishing international compassion for integrated primary care.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

WHO:

World Health Organization

RMIC:

Rainbow Model of Integrated Care

RMIC-MT:

Rainbow Model of Integrated Care-Measurement Tool

HEP:

Health Economics and Policy

EV:

English version

CV:

Chinese version

CCP:

Community care professional

CHS:

Community health stations

GP:

General practitioner

KMO:

Kaiser-Meyer-Olkin

EFA:

Exploratory factor analysis

SRMR:

Standardized root-mean-square residual

CFI:

Comparative fit index

TLI:

Tucker-Lewis index

References

  1. Global action plan for the prevention and control of noncommunicable diseases 2013–2020. Geneva: World Health Organization 2013. Available from: http://www.who.int/nmh/publications/ncd-action-plan/en/. Accessed 8 Oct 2019.

  2. Slama S, Kim HJ, Roglic G, et al. Care of non-communicable diseases in emergencies. Lancet. 2017;389(10066):326–30.

    Article  Google Scholar 

  3. WHO global strategy on people-centred and integrated health services. Interim report. Geneva: World Health Organization; 2016. Available from: http://apps.who.int/iris/bitstream/handle/10665/155002/WHO_HIS_SDS_2015.6_eng.pdf?sequence=1 [cited 2018 Aug 3].

  4. People-centred and integrated health services: an overview of the evidence: interim report. Geneva: World Health Organization 2015. Available from: http://apps.who.int/iris/bitstream/handle/10665/155004/WHO_HIS_SDS_2015.7_eng.pdf?sequence=1. [cited 2018 Aug 3].

  5. Ye C, Browne G, Grdisa VS, et al. Measuring the degree of integration for an integrated service network. Int J Integr Care. 2012;12(5):e137.

    Article  Google Scholar 

  6. European Observatory on Health Systems and Policies. Health systems and the financial crisis. London: Eurohealth; 2012.

    Google Scholar 

  7. Health systems and the financial crisis. Eurohealth, volume 18, no.1. London: London School of Economics and Political Science; 2012.

    Google Scholar 

  8. Busetto L, Luijkx K, Vrijhoef, et al. Development of the COMIC model for the comprehensive evaluation of integrated care interventions. Int J Care Coordination. 2016;19:47–58.

    Article  Google Scholar 

  9. Goodwin N. Understanding integrated care: A complex process, a fundamental principle. Int J Integr Care. 2013;13:e011.

  10. World Bank Group, World Health Organization, Ministry of Finance, National Health and Family Planning Commission and Ministry of Health Resources and Social Security. Deeping health reform in China-building high-quality and value-based service delivery. 2016. https://documents.worldbank.org/en/publication/documents-reports/documentdetail/800911469159433307/deepening-health-reform-in-china-building-highquality-and-value-based-service-delivery-policy-summary.

  11. Bautista MAC, Uurjono M, Lim YW, et al. Instruments measuring integrated care: a systematic review of measurement properties. Milbank Q. 2016;94(4):862–917.

    Article  Google Scholar 

  12. Suter E, Oelke ND, da Silva Lima MAD, et al. Indicators and measurement tools for health systems integration: A knowledge synthesis. Int J Integr Care. 2017;17(6):4 1–17.

    Article  Google Scholar 

  13. Lyngso AM, Godtfredsen NS, Host D, et al. Instruments to assess integrated care: a systematic review. Int J Integr Care. 2014;14:e027.

    Article  Google Scholar 

  14. Stephenson MD, Campbell JM, et al. Assessing healthcare professionals’ experiences of integrated care: do surveys tell the full story? Int J Evid-Based Hea. 2017;15:90–101.

    Article  Google Scholar 

  15. Mc Hugh S, O’Mullane M, Perry IJ, et al. Barriers to, and facilitators in, introducing integrated diabetes care in Ireland: a qualitative study of views in general practice. BMJ Open. 2013;3:e003217.

    Article  Google Scholar 

  16. Bruner P, Waite R, Davey MP. Providers? Perspectives on collaboration. Int J Integr Care. 2011;11:e123.

    Article  Google Scholar 

  17. Vanneste D, Vermeulen B, Declercq A. Healthcare professionals? Acceptance of BelRAI, a web-based system enabling person-centred recording and data sharing across care settings with interrail instruments: a UTAUT analysis. BMC Med Inform Decis Mak. 2013;13:129.

    Article  Google Scholar 

  18. Valentijn PP, Schepman SM, Opheij W, et al. Understanding integrated care: a comprehensive conceptual framework based on the integrative functions of primary care. Int J Integr Care. 2013;13:e010.

  19. Valentijn PP. Rainbow of chaos: A study into the theory and practice of integrated primary care. PhD Thesis, Tilburg University, The Netherlands, 2015. Accessed July 18, 2016. https://pure.uvt.nl/portal/files/9272073/Valentijn_Rainbow_16_12_2015.pdf.

  20. Bentham WD, Ratzliff A, Harrison D, et al. The experience of primary care providers with an integrated mental health care program in safety-net clinics. Fam Commun Health. 2015;38:158–68.

    Article  Google Scholar 

  21. Boesveld IC, Valentijn PP, Hitzert M, et al. An approach to measuring integrated care within a maternity care system: experiences from the maternity care network study and the Dutch birth Centre study. Int J Integr Care. 2017;17(2):6. https://doi.org/10.5334/ijic.2522.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Angus L, Valentijn PP. From micro to macro: assessing implementation of integrated care in Australia. Aust J Prim Health. 2018;24:59–65.

    Article  Google Scholar 

  23. Fares J, Chung KSK, Passey M, Longman J, Valentijn PP. Exploring the psychometric properties of the rainbow model of integrated care measurement tool for care providers in Australia. BMJ Open. 2018;9:e027920. https://doi.org/10.1136/bmjopen-2018-027920.

    Article  Google Scholar 

  24. Nurjono M, Valentijn PP, Mary Ann C, et al. A Prospective Validation Study of a Rainbow Model of Integrated Care Measurement Tool in Singapore. Int J Integr Care. 2016;16(1):1 1–6.

    Article  Google Scholar 

  25. Valentijn PP, Pereira F, Sterner CW, Vrijhoef HJM, Ruwaard D, Hegbrant J, et al. Validation of the rainbow model of integrated care measurement tools (RMIC-MTs) in renal care for patient and care providers. PLoS One. 2019;14(9):e0222593. https://doi.org/10.1371/journal.pone.0222593.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Valentijn PP. Rainbow Model of Integrated Care Measurement Tools (RMIC-MT's) for Patientand Care providers. Essenburgh Research & Consultancy 2019. Available from https://www.essenburgh.com/the-rainbow-model-measurements-tools-for-integrated-care. Accessed 15 Oct 2019.

  27. Helfrich CD, Li YF, Sharp ND, Sales AE. Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci. 2009;4:38. 5908-4-38. https://doi.org/10.1186/1748-5908-4-38.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Brislin RW. The wording and translation of research instruments. In: Lonner WL, Berry JW, editors. Field methods in cross-cultural research. Newbury Park, CA: Sage Publications; 1986. p. 137–64.

    Google Scholar 

  29. Streiner D, Norman G. Health measurement scales a practical guide to their development and use. 4th ed. Oxford: Oxford University Press; 2008.

    Book  Google Scholar 

  30. General Office of the State Council. Guiding of Promoting the Construction and Development of Medical Consortia. General Office of the State Council 2017, No.32 [cited 2018 May 1]. Available from: http://www.gov.cn/zhengce/content/2017-04/26/content_5189071.htm.

  31. China Webmaster. Introduction of so jump. https://top.chinaz.com/site_www.sojump.com.html. Accessed 15 Oct 2019.

  32. Kim H-Y. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dentistry Endodontics. 2013;38(1):52–4.

    Article  Google Scholar 

  33. Zwart DLM, Langelaan M, van de Vooren RC, et al. Patient safety culture measurement in general practice. Clinimetric properties of “SCOPE”. BMC Fam Pract. 2011;12:117.

    Article  Google Scholar 

  34. Cerny BA, Kaiser HF. A study of a measure of sampling adequacy for factor-analytic correlation matrices. Multivar Behav Res. 1977;12:43–7.

    Article  CAS  Google Scholar 

  35. Spangenberga L, Glaesmera H, Hallenslebena N, et al. Psychometric properties of the German version of the suicide cognitions scale in two clinical samples. Psychiatry Res. 2019;274:254–62.

    Article  Google Scholar 

  36. Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ Res Methods. 2000;83(1):107–40.

  37. Finch AP, Brazier JE, Mukuria C, Bjorner JB. An exploratory study on using principal-component analysis and confirmatory factor analysis to identify bolt-on dimensions: The EQ-5D Case Study. Value Health. 2017;20(10):1362–75.

  38. Vaingankar JA, Abdin E, Chong SA. Exploratory and confirmatory factor analyses of the Multidimensional Scale of Perceived Social Support in patients with schizophrenia. Compr Psychiatry. 2012;53(3):286–91.

  39. Cadeyrn JG, Brenda H. On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. Int J Nurs Stud. 2014;51(3):511–21.

  40. Clark RA, Pua Y-H, et al. Reliability and concurrent validity of the Microsoft Xbox one Kinect for assessment of standing balance and postural control. Gait Posture. 2015;42(2):210–3.

    Article  Google Scholar 

  41. Reuther S, Dichter MN, Bartholomeyczik S, et al. Construct validity and internal consistency of the neuropsychiatric inventory – nursing home (NPI-NH) in German nursing homes. Int Psychogeriatr. 2016;28(6):1017–27.

    Article  Google Scholar 

  42. Streiner DL, Norman GR, Cairney J. In: Oxford Unversity Press, editor. Health measurement scales: a practical guide to their development and use (fifth edition); 2015.

    Chapter  Google Scholar 

  43. Valentijn PP, Vrijhoef HJ, Ruwaard D, Boesveld I, Arends RY, Bruijnzeels MA. Towards an international taxonomy of integrated primary care: a Delphi consensus approach. BMC Fam Pract. 2015;16:64 015-0278-x.

    Article  Google Scholar 

  44. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11:322–33.

    Article  Google Scholar 

  45. Thammaiah S, Manchaiah V, Easwar V, et al. Translation and adaption of five English language self-report health measures to south Indian Kannada language. Audiol Res. 2016;6(1):22–7.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Health Commission of Nanshan District for coordination of field work.

Funding

This work was supported by the National Natural Science Foundation of China (grant number 71804202) and the National Social Science Fund of China (grant number 18BGL218). The funders had no role in preparation of the manuscript or decision to publish.

Author information

Authors and Affiliations

Authors

Contributions

XW and PV conceptualized this study. PZ and LC collected and analyzed the data. YH and XW wrote the first draft of the manuscript. YH, PV and XW critically commented the paper. The final version submitted for publication was read and approved by all authors.

Corresponding author

Correspondence to Xin Wang.

Ethics declarations

Ethics approval and consent to participate

The protocol for the research project has been approved by Ethics Committee of School of Public Health, SUN Yat-Sen University (ref 2019 No.073). All participants gave written informed consent before recruited into the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Zhu, P., Chen, L. et al. Validation of the care providers version of the Rainbow Model of Integrated Care-measurement tool in Chinese primary care systems. BMC Health Serv Res 20, 727 (2020). https://doi.org/10.1186/s12913-020-05562-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-020-05562-2

Keywords