Skip to main content
Download PDF

Psychometric evaluation of the 8-item Altarum Consumer Engagement (ACE) Measure™ in community-dwelling adults in Singapore

BMC Health Services Research volume 21, Article number: 1347 (2021) Cite this article

Abstract

Background

A valid and reliable measure is essential to assess patient engagement and its impact on health outcomes. This study aimed to examine the psychometric properties of the 8-item Altarum Consumer Engagement Measure™ (ACE Measure) among English-speaking community-dwelling adults in Singapore.

Methods

This cross-sectional study involved 400 randomly selected community-dwelling adults (mean age: 49.7 years, 50.0% were female, 72.3% were Chinese) who completed the English version of the 8-item ACE Measure independently. The item-level statistics were described. The internal consistency of the measure was measured by Cronbach alpha and item-rest correlations. Validity of the tool was assessed by 1) factorial validity using confirmatory factor analysis (CFA), 2) hypothesis-testing validity by correlating ACE subscales (Commitment and Navigation) with health-related outcomes, and 3) criterion validity against the Patient Activation Measure and Health Confidence Measure.

Results

There was no floor or ceiling effect for Commitment and Navigation subscales, and the Cronbach alpha for each subscale was 0.76 and 0.54, respectively. The two-factor structure was confirmed by CFA. In general, Commitment and Navigation subscales were positively correlated with frequency of activity participation (rho = 0.30 - 0.33) and EQ-5D visual analog scale (rho = 0.15 - 0.30). Individuals who perceived better health than peers had higher subscale scores (p < 0.01). Each subscale score had moderate and positive correlations with activation score (rho = 0.48 - 0.55) and health confidence score (rho = 0.35 - 0.47).

Conclusions

The two-subscale ACE Measure demonstrated good construct validity in English-speaking Singapore community-dwelling adults. Evidence in internal consistency was mixed, indicating further investigation.

Peer Review reports

Background

With the shift from a disease-centered to patient-centered healthcare model that focuses on disease prevention and sustained management of chronic conditions management [1, 2], healthcare systems and providers are increasingly recognizing the importance of engaging healthcare consumers in their treatment decision making process and empowering them to take on more active roles in their own health and healthcare management.

Used interchangeably, patient or health engagement is increasingly recognized as a critical component of patient-centered care. Yet it remains a poorly understood concept [3, 4] with inconsistent definitions and operationalization. Prior literature has suggested that patient engagement is a multidimensional concept – where individuals are not only required to have the knowledge, skills, ability, and willingness to manage their own health, but also to be actively involved in making competent and well-informed decisions together with their healthcare providers and commit to take actions for own health care [3, 5, 6]. A growing body of evidence demonstrated that individuals who are engaged in their health care tend to express a stronger motivation in self-management and are better able to make informed decisions about their care options, which contributes to improved adherence [7,8,9], better self-perceived health [10, 11] and health outcomes (i.e. fewer depressive symptom and better physical function) [8, 12, 13], and lower healthcare costs [14, 15].

Measuring engagement in health care using a valid and reliable measure is essential to evaluate its impact on health outcomes and healthcare utilization. A few instruments have been developed in recent years to measure different domains of patient engagement including patient activation [16], motivation [17], self-management [18,19,20], and shared decision making [21]. The 13-item Patient Activation Measure (PAM-13) is a powerful tool that is widely used to assess an individual’s self-perceived knowledge, skills and confidence to manage one’s own health and healthcare requirements [16, 22]. The Patient Health Engagement (PHE) scale, which was developed based on the PHE model, aims to grasp the complexity and dynamicity of the psychological experience of engagement [9]. However, there is still a lack of concrete instruments which encompass the full range of dimensions of patient engagement [9, 23]. Recently, Duke and colleagues developed the Altarum Consumer Engagement Measure™ (ACE Measure) [23] to address the multi-dimensionality of health engagement.

The ACE Measure was developed to assess the health engagement of individuals and population from multiple aspects of patient perceptions, activities of participation in health and healthcare, and the use of information to compare and choose providers or services [23]. It is not only meant for disease management, but also suitable for population health management. The original 21-item ACE Measure is a multi-dimensional health engagement measure which consists of four subscales: Commitment - confidence and ability to maintain a healthy lifestyle and manage one’s health, Ownership - perceived role in and responsibility for one’s health, Informed Choice - patterns of seeking and using information about health and healthcare, and Navigation - confidence and ability to ask about and participate in treatment decisions [23]. It was subsequently shortened to 12 items consisting of three subscales: Commitment (4 items), Informed Choice (4 items), and Navigation (4 items). The Ownership subscale was not included due to its similarity in outcome prediction with that of the Commitment subscale [24].

The 12-item ACE Measure has been validated among United States prediabetes patients [24] and has been used in many ways including population benchmarking, generating patient’s personalized engagement reports, triaging resources, determining course of treatment and health coaching since its development [25]. Each subscale of the 12-item ACE Measure can be used independently. In a recent study conducted by the United States Air Force, diabetes patients with high Commitment achieved better improvement in hemoglobin A1c over 8 months [26]. However, the ACE measure has never been validated among Singapore adult population. Hence, the aim of the study was to examine the psychometric properties of the ACE Measure among English-speaking community-dwelling adults in Singapore.

Methods

Study participants

Data of the study was collected in the second-year follow-up survey of the Population Health Index (PHI) study via surveyor-administered face-to-face interviews at participants’ homes or places preferred by participants. The PHI survey was a longitudinal health survey conducted among community-dwelling adult population (aged 21 years and above) living in the Central region of Singapore. The sampling methods and procedures were described elsewhere [27, 28]. A total of 1942 individuals participated in the survey and they were followed up at one-year and two-year after the first survey. During the second-year follow-up survey, the English-speaking participants who were capable to complete the questionnaire independently (n = 978) were also invited to complete the English version ACE Measure, the PAM-13, and the Health Confidence Measure. Individuals who had invalid responses to the PAM-13 (including those with less than 10 questions answered and those with “3 = agree” responses for all 13 items) were excluded from the analysis (n = 152). As suggested by Nicholas and colleagues [29], a sample size of 300 or more are necessary for a confirmatory factor analysis (CFA). Hence, we randomly selected 400 participants from the remaining 826 participants for the analysis of the study.

The study obtained ethics approval from Singapore’s National Healthcare Group Domain Specific Review Board (Reference Number: 2015/00269) and was conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained from each participant after being informed of the study purpose, procedures, and confidentiality of the data collected.

Measures

The 8-item Altarum consumer engagement measure™

The three subscales of the 12-item ACE Measure and their respective items were listed in Table 1. The Informed Choice subscale of the 12-item ACE Measure contains two items related to official ratings of doctors (“Q8. I compare doctors using official ratings about how well their patients are doing.” and “Q10. When choosing a new doctor, I look for official ratings based on patient health.”). Unlike the US healthcare system, there are no official ratings of healthcare providers in Singapore. As such, these two items are not relevant to Singapore’s context. Upon discussion with developers of the tool, these two items were excluded from the survey. The wording of the remaining 10 items was reviewed by two authors for understandability in Singapore. As both authors felt the wording was commonly used and easy to be understood by Singapore English-speaking population, all 10 items were included in the survey without any changes. The general feedbacks obtained via the trained surveyors from the first 30 participants indicated that the items were well understood by the study population.

Table 1 The three subscales of the 12-item ACE Measure and the 8-item ACE Measure included for validation
Full size table

With the removal of the two items, the Informed Choice subscale was left with two items that would cause identification problem in confirmatory factor analysis as the minimum number of items for a factor was not reached. After consulting the developer of the ACE Measure and considering that each subscale could be used independently, we only examined the psychometric properties of the two subscales - Commitment and Navigation of the 8-item ACE Measure in this study.

The 8 items of the ACE Measure used the same rating scale with 5 response options (0 = strongly disagree, 1 = disagree, 2 = neither agree nor disagree, 3 = agree, and 4 = strongly agree). Following the scoring procedure for the 21-item ACE Measure, the mean score of items in each subscale was multiplied by 6.25 to create a subscale score ranging from 0 to 25, with a higher score representing greater commitment or better navigation.

Health-related outcomes for hypothesis-testing validity

Frequency of activity participation and health-related quality of life (HRQoL) were the two health-related outcomes selected to examine the hypothesis-testing validity of the two subscales of the 8-item Measure. The Frequency domain of the Late Life Function and Disability [30] was used to measure frequency of activity participation and the EQ-5D-5L [31] (including EQ-5D Index and EQ-5D Visual Analog Scale (VAS)) was used to measure HRQoL. In addition, self-perceived health status was obtained by asking the question “In comparison with other people of the same age, how do you consider your health status?” with four response options (“not as good”, “does not know”, “as good”, and “better”).

Patient activation measure and health confidence measure for criterion validity

The 13-item Patient Activation Measure® (PAM-13) and Health Confidence Measure were chosen to examine the criterion validity of the two subscales of the 8-item ACE Measure. Developed by Hibbard and colleagues [16] using Rasch analyses, the PAM-13 is an interval-level, unidimensional and Guttman-like measure that contains 13 questions measuring self-assessed knowledge, skills and confidence for self-management of one’s health or chronic condition. PAM-13 is the most commonly used measure for patient’s activation in health and has been shown to be associated with a broad range of health-related outcomes [16, 32]. Each item of the PAM-13 has 5 response options (1 = strongly disagree, 2 = disagree, 3 = agree, and 4 = strongly agree, with additional “not applicable” option). To calculate the total PAM score, the PAM-13 scoring spreadsheet 2017 (the scoring algorithm obtained from the license provider Insignia Health) was used to calculate individual participant’s PAM-13 score ranging from 0 to 100, and to determine their activation levels. The PAM-13 scoring spreadsheet only calculated PAM score for participants with 10 or more questions answered, and participants with “3 = agree” responses for all 13 items were treated as outliers. Only participants with valid PAM-13 were sampled for this analysis.

Health Confidence Measure [20] is an effective proxy for engagement using a single question “How confident are you that you can control and manage most of your health problems?” The respondents were asked to rate their health confidence on a scale from 0 (not very confident) to 10 (very confident). A score of 7 or higher was categorized as high health confidence and a score of 6 or lower was categorized as low health confidence.

Data analysis

Item-level descriptive analysis and internal consistency

Descriptive analyses for each of the 8 items were conducted as initial exploration of the data. Mean, standard deviation (SD), median and score distribution were reported for individual items. Furthermore, floor or ceiling effects were considered present if > 15% of participants achieved the lowest score / floor effect (0/4 for individual items, 0/25 for subscales) or the highest score / ceiling effect (4/4 for individual items, 25/25 for subscales) [33].

The internal consistency of each subscale of the 8-item ACE Measure was assessed by calculating Cronbach’s alpha, item-rest correlations (the correlation of the item with the total score on the other items), and average inter-item correlations (a way of measuring whether different items that are meant to measure the same general construct give similar scores). A Cronbach’s alpha of 0.70 or higher was defined as the acceptable value [34]. The ideal range of average inter-item correlation was 0.15 to 0.50 [35].

Factorial validity

As the 8-item ACE Measure comprises two correlated subscales - Commitment and Navigation in the 12-item ACE Measure, to examine factorial validity of the construct of the 8-item ACE Measure, two-factor CFA with the respective 4 items in each of the original subscales was conducted. Each item could load onto one latent factor with the first item’s loading onto the latent factor fixed at 1.0. For the remaining factor loadings, residual variances were freely estimated. Standardized factor loading values were calculated and multiple goodness of fit statistics of the model were performed, including the ratio of the chi-square to degrees of freedom chi-square (χ2/df), the Comparative Fit Index (CFI), the Tucker-Lewis Fit Index (TLI), and the Root Mean Square Error of Approximation (RMSEA). The cutoff values proposed by Hu and Bentler: χ2/df ≤ 3, CFI ≥0.95, TLI ≥0.95, and RMSEA ≤0.06 [36] were used to evaluate the goodness of model fit.

Hypothesis-testing validity

To examine the hypothesis-testing validity of the 8-item ACE Measure, the variation in the two ACE Measure subscale scores across socio-demographic subgroups were examined by Mann Whitney U tests or Kruskal-Wallis tests (followed by Dunn’s tests [37]). The difference in ACE Measure subscale scores across self-perceived health status groups was evaluated using the Kruskal-Wallis test, followed by Dunn’s tests. Furthermore, the relationships between ACE Measure scores and frequency of activity participation as well as HRQOL (measured by EQ-5D index EQ-5D VAS scores) were assessed using Spearman’s rank-order correlations. We hypothesized that 1) those with higher education levels have higher Commitment and Navigation scores; 2) those who perceived better health status than peers have higher Commitment score than their counterparts; and 3) greater engagement, especially higher commitment is associated with more frequent activity participation and higher health-related quality of life (HRQOL).

Criterion validity

Criterion validity of the 8-item ACE Measure was carried out against the PAM-13 and the Health Confidence Measure by examining the relationships between the two ACE subscale scores and PAM-13 and health confidence scores using Spearman’s rank-order correlations. The two ACE subscale scores were also compared across participants with different PAM activation levels and between those with high and low health confidence. We hypothesized that the two subscale scores have positive, moderate correlation with PAM-13 and health confidence scores. We also hypothesized that those with higher PAM levels and those with higher health confidence should have higher ACE subscale scores over their counterparts.

All the analyses were conducted using Stata/SE 16 for Windows. The result was considered significant if a p value was < 0.05.

Results

The characteristics of the participants were described in Table 2. The average age of the 400 participants was 49.2 years (standard deviation: 15.0 years, range: 23-90 years). Half of the participants were women and 72.3% were Chinese. Majority (59.3%) of the participants had post-secondary or higher education.

Table 2 Participant characteristics and mean subscale scores of the 8-item ACE Measure for each subgroup (N = 400)
Full size table

Item-level statistics and internal consistency

Table 3 shows the descriptive statistics of individual items of the two subscales of the 8-item ACE Measure and Fig. 1 presents the distribution of the responses for each item. The means of the 4 items under Commitment and Navigation subscales were 3.0 and 2.8, respectively. Item ACE8_2 had the highest percentage of participants with response of “strongly agree” (22.5%) followed by item ACE8_8 (19.0%) and ACE8_1 (17.8%). However, the proportions of participants having the lowest or highest mean score of the 4 items under Commitment and Navigation subscales were all below 5.0%, indicating no floor or ceiling effect for each subscale.

Table 3 Item-level statistics, item-rest correlation, and average inter-item covariance for the two-subscale ACE Measure (N = 400)
Full size table
Fig. 1
figure 1

The distribution of responses by items of the two-subscale ACE Measure

Full size image

The internal consistency of the 8-item ACE Measure was acceptable for Commitment subscale with a Cronbach’s alpha of 0.76. However, it was not satisfactory for Navigation subscale with a Cronbach’s alpha of 0.54. The average inter-item correlation for Commitment subscale was 0.20, with individual inter-item correlations ranging from 0.18 to 0.23, falling within the ideal range of 0.15 – 0.50. All four items for Commitment subscale had item-rest correlations ranging from 0.50 – 0.61, exceeding the cutoff score for strong correlations (≥0.50). The average inter-item correlation for Navigation subscale was 0.15, with items ACE8_4 and ACE8_6 falling below 0.15 (Table 1).

Factorial validity

The CFA confirmed the two-factor latent construct of the 8-item ACE Measure. The standardized factor loading for each item onto the respective latent construct (Commitment and Navigation) was provided in Fig. 2. All four items under Commitment subscale had a factor loading higher than 0.50. Two items under Navigation subscale had relatively lower factor loadings (0.48 for item ACE8_2 and 0.30 for item ACE8_7). The Chi-square test result (χ2(17) =27.43, p = 0.05) and goodness of fit indices (CFI = 0.98, TLI = 0.97, RMSEA = 0.04) indicated a good fit between the two-factor model and the observed data.

Fig. 2
figure 2

The path diagram for the two-factor CFA model: standardized estimates

Full size image

Hypothesis-testing validity

As expected, there were no differences in the Commitment and Navigation scores between males and females or across ethnicity groups (Table 2). Dunn’s test showed that participants aged 60-74 years reported higher Commitment scores compared to those aged 21-39 years, and participants having higher education levels had higher Navigation scores.

Compared to participants who perceived health not as good as peers, those who perceived health as good as or better than peers had significantly higher Commitment scores (Table 2). Commitment score was positively associated with higher frequency of activity participation (rho = 0.30, p < 0.01) and higher EQ-5D VAS (rho = 0.30, p < 0.01) and Index scores (rho = 0.15, p < 0.01); Navigation score was associated with higher frequency of activity participation (rho = 0.33, p < 0.01) and higher EQ-5D VAS (rho = 0.15, p < 0.01) but not associated with higher EQ-5D Index scores (rho = 0.03, p = 0.570) (Table 4).

Table 4 Spearman correlation between two subscales of the 8-item ACE Measure and health-related outcomes (N = 400)
Full size table

Criterion validity

To examine criterion validity, the two subscales were evaluated in relation to PAM-13 score and health confidence score. The Spearman correlation results showed that Commitment and Navigation scores were moderately correlated with PAM-13 score (for Commitment: rho = 0.55, 95% CI = 0.48, 0.62; for Navigation: rho = 0.48, 95% CI = 0.40, 0.55) and health confidence score (for Commitment: rho = 0.47, 95% CI = 0.39, 0.54; for Navigation: rho = 0.35, 95% CI = 0.26, 0.43).

The comparison of the two subscale scores across PAM activation levels and between high and low health confidence groups (Table 5) showed that Commitment and Navigation scores increased with the increase of PAM and health confidence levels (all p < 0.001). Dunn’s test results showed that there was a significant difference in Commitment and Navigation scores between any two PAM levels among PAM level 2 to level 4. The insignificant score difference between level 1 and level 2 could probably be explained by the small number of participants with PAM level 1 (n = 8).

Table 5 The mean subscale scores of the 8-item ACE Measure by PAM activation and health confidence levels (N = 400)
Full size table

Discussion and conclusions

Discussion

The present study examined the psychometric properties of the 8-item ACE Measure among English-speaking community-dwelling adults in Singapore. The CFA results confirmed that the two-factor structure (Commitment and Navigation) fit the data well with some model modifications. The considerably high factor loadings demonstrated simple structure of each subscale and indicated good factorial validity – one of the four types of evidence for construct validity [38]. The analysis of floor and ceiling effects for individual items revealed that “strongly disagree” was very rarely selected (0 – 2.3%) and the percentages of “strongly agree” ranged from 8.5 to 22.5%, with half of the items presenting ceiling effects. However, there were no floor and ceiling effects for each subscale of the 8-item ACE Measure.

The internal consistency reliability results were good for Commitment. The item-rest correlations for Navigation subscale were lower than 0.50 for all four items and the reliability coefficient was lower than 0.70, suggesting the internal consistency reliability of the Navigation subscale is questionable. However, the unsatisfactory internal consistency of the Navigation subscale was also pronounced in the previous validation studies (23,24), indicating the issue might lie with the original items of Navigation. Items like “I feel comfortable talking to my doctor about my health.” and “Different doctors give different advice; it’s up to me to choose what’s right for me.” had relatively lower factor loadings. This could be due to patient’s comfort level in being able to talk to the doctor about health or a person’s attitude towards doctors’ advice may be influenced by many other factors (e.g. personality, trust or doctor-patient relationship, and cultural beliefs) [39] other than one’s confidence and ability to ask about and participate in treatment decisions. This indicates that the items under Navigation needs for further improvement by either revising the existing questions or including additional items.

Comparison of the Commitment and Navigation scores across individuals with different self-perceived health status showed that those with perceived health not as good as peers tended to have lower Commitment score than those who perceived their health as good as or better than peers, which is consistent with our hypothesis and the results of the original ACE validation study [23]. Furthermore, consistent with findings reported by studies using other patient engagement measures [11, 40, 41], those with poorer health-related outcomes, i.e. lower frequency of activity participation and poorer HRQoL demonstrated being less engaged. Although the study cannot infer causal relationships between engagement and health-related outcomes based on the nature of cross-sectional design, the associations serve as hypothesis-testing validity, and thus support the construct validity. Our hypothesis of a significant relationship between education levels and Navigation scores was supported by the data, which further supports the hypothesis-testing validity. Although those with higher education levels also had higher Commitment scores, the association was not significant.

The moderate associations between each subscale and the scores of the well-established PAM-13 and Health Confidence Measure, and the linear trends of the two subscale scores across PAM levels support the criterion validity.

It is worth mentioning that different from the ACE-12 Measure, the 8-item ACE Measure only have two subscales (Commitment and Navigation). In Singapore, primary healthcare system includes government polyclinics and private general medical practitioner (GP) clinics. There is no official primary care doctors’ information available online at national wide and residents seeking treatment at polyclinics are not able to select a particular doctor as the next available doctor will be randomly allocated to them. Although GP clinics offer higher selection feasibility, the higher cost of care and lack of comprehensive facilities (e.g. onsite laboratory and imaging services) result in most residents, especially chronic disease patients, seeking treatment mainly from government polyclinics [42, 43]. With these characteristics of primary healthcare system, residents in Singapore are less prone to choose doctors or look for information online when choosing a new doctor, especially older residents. Given that Informed Choice should be an essential component of patient engagement [23], it is necessary to develop items relevant to the Singapore context. Instead of capturing choosing a new doctor for care, capturing awareness of options to choose from several diagnostic tests or treatments, knowing the details, benefits, risks and expected outcome of each for informed choice might be more appropriate. Further research on selecting relevant items to measure this dimension would be useful in making the patient engagement measure more holistic.

With the current shift of health care from hospital to community in Singapore and given the relationship between patient engagement and improved health outcomes, it is important to better understand and measure patient engagement among our population. A validated measure will enable us to examine how a patient is ready to take care of their own health in the community and to evaluate the effectiveness of health coaching programs on patient activation or engagement. It will also allow for better intervention recommendation and health resources allocation.

Conclusions

The two-subscale ACE Measure demonstrated good factorial validity and criterion validity. It also had moderate correlation with frequency of activity participation in the study population. However, evidence in internal consistency was mixed with unsatisfactory Cronbach alpha for Navigation, indicating a need for improvement for Navigation subscale. Further research incorporating items reflecting informed choice which are relevant to the Singapore context is needed to make the measure more holistic.

Availability of data and materials

According to the Data Protection Act Commission Singapore-Advisory Guidelines for the Healthcare Sector, all the individual data collected for the Population Health Index study are protected under the Act. As such, the datasets analyzed during the current study are not publicly available. However, minimal dataset underlying the findings in the manuscript is available from the corresponding author on reasonable request.

Abbreviations

ACE Measure:

Altarum Consumer Engagement Measure™

CFA:

Confirmatory Factor Analysis

CFI:

Comparative Fit Index

CI:

Confidence interval

GP:

General medical practitioner

HRQoL:

Health-related quality of life

PAM:

Patient Activation Measure

PHE:

Patient Health Engagement

PHI:

Population Health Index

SD:

Standard deviation

TLI:

Tucker-Lewis Fit Index

RMSEA:

Root Mean Square Error of Approximation

VAS:

Visual Analog Scale

References

  1. M.A S, W. M. Expanding patient-centered care to empower patients and assist providers. AHRQ: Research in Action. 2002;(5). Available from: http://www.ahrq.gov/research/findings/factsheets/patient-centered/ria-issue5/index.html. [cited 14 Aug 2018]

  2. Santilli J, Vogenberg FR. Key strategic trends that impact healthcare decision-making and stakeholder roles in the new marketplace. Am Health Drug Benefits. 2015;8(1):15–20.

    PubMed  PubMed Central  Google Scholar 

  3. Higgins T, Larson E, Schnall R. Unraveling the meaning of patient engagement: a concept analysis. Patient Educ Couns. 2017;100(1):30–6.

    Article  Google Scholar 

  4. Barello S, Graffigna G, Vegni E. Patient engagement as an emerging challenge for healthcare services: Mapping the literature. Nurs Res Pract. 2012; Available from: https://www.hindawi.com/journals/nrp/2012/905934/ [cited 18 Jun 2018].

  5. Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: A state of the science review. J Med Internet Res. 2015;17(6) Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4526960/ [cited 14 Aug 2018].

  6. Athenahealth. 5 elements of a successful patient engagement strategies [Whitepaper][Internet]. athenahealth, Inc.; 2014 [cited 2021 Dec 10]. Available from: https://caas.athenahealth.com/sites/default/files/2019-03/Patient_Engagement_Whitepaper.pdf

    Google Scholar 

  7. Zimbudzi E, Lo C, Ranasinha S, Kerr PG, Polkinghorne KR, Teede H, et al. The association between patient activation and self-care practices: a cross-sectional study of an Australian population with comorbid diabetes and chronic kidney disease. Health Expect. 2017;20(6):1375–84.

    Article  Google Scholar 

  8. Parchman ML, Zeber JE, Palmer RF. Participatory decision making, patient activation, medication adherence, and intermediate clinical outcomes in type 2 diabetes: a STARNet study. Ann Fam Med. 2010;8(5):410–7.

    Article  Google Scholar 

  9. Graffigna G, Barello S, Bonanomi A, Lozza E. Measuring patient engagement: development and psychometric properties of the patient health engagement (PHE) scale. Front Psychol. 2015;6:274.

    Article  Google Scholar 

  10. Smith SG, Pandit A, Rush SR, Wolf MS, Simon CJ. The role of patient activation in preferences for shared decision making: results from a national survey of U.S. adults. J Health Commun. 2016;21(1):67–75.

    Article  Google Scholar 

  11. Gleason KT, Tanner EK, Boyd CM, Saczynski JS, Szanton SL. Factors associated with patient activation in an older adult population with functional difficulties. Patient Educ Couns. 2016;99(8):1421–6.

    Article  Google Scholar 

  12. Shortell SM, Poon BY, Ramsay PP, Rodriguez HP, Ivey SL, Huber T, et al. A multilevel analysis of patient engagement and patient-reported outcomes in primary care practices of accountable care organizations. J Gen Intern Med. 2017;32(6):640–7.

    Article  Google Scholar 

  13. Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff. 2013;32(2):207–14.

    Article  Google Scholar 

  14. Greene J, Hibbard JH, Sacks R, Overton V, Parrotta CD. When patient activation levels change, health outcomes and costs change, too. Health Aff. 2015;34(3):431–7.

    Article  Google Scholar 

  15. Ngorsuraches S, Rosa PD, Ge X, Djira G, Michael S, Wey H. Patient engagement as a predictor for health outcomes and costs in multiple chronic conditions. Value Health. 2018;21:S88–9.

    Article  Google Scholar 

  16. Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and testing of a short form of the patient activation measure. Health Serv Res. 2005;40(6 Pt 1):1918–30.

    Article  Google Scholar 

  17. Miedany YE, Gaafary MME, Youssef S, Palmer D. Development of the patient motivation questionnaire: conceptualizing and measuring motivation in patients with inflammatory arthritis. Ann Rheum Dis. 2016;75(Suppl 2):469.

    Google Scholar 

  18. Lenferink A, Paap M, Harvey P, Battersby M, Frith P, der Valk PV, et al. Validation of the 12-item partners in health scale to measure patient self-management behaviour and knowledge in Dutch patients with COPD. Eur Respir J. 2015;46(suppl 59):PA726.

    Google Scholar 

  19. Veldman K, Reijneveld SA, Lahr MMH, Uittenbroek RJ, Wynia K. The Partners in Health scale for older adults: design and examination of its psychometric properties in a Dutch population of older adults. Health Expect. 2017;20(4):601–7.

    Article  Google Scholar 

  20. Wasson J, Coleman EA. Health confidence: a simple, essential measure for patient engagement and better practice. FPM. 2014;21(5):8–12.

    Google Scholar 

  21. Simon D, Schorr G, Wirtz M, Vodermaier A, Caspari C, Neuner B, et al. Development and first validation of the shared decision-making questionnaire (SDM-Q). Patient Educ Couns. 2006;63(3):319–27.

    Article  CAS  Google Scholar 

  22. Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39(4 Pt 1):1005–26.

    Article  Google Scholar 

  23. Duke CC, Lynch WD, Smith B, Winstanley J. Validity of a new patient engagement measure: the Altarum consumer engagement (ACE) measure. Patient. 2015;8(6):559–68.

    Article  Google Scholar 

  24. Castellon-Lopez Y, Skrine Jeffers K, Duru OK, Moreno G, Moin T, Grotts J, et al. Psychometric properties of the Altarum consumer engagement (ACE) measure of activation in patients with prediabetes. J Gen Intern Med. 2020;35(11):3159–65.

    Article  Google Scholar 

  25. The Altarum Consumer Engagement (ACE) Measure. Altarum. 2020. Available from: https://altarum.org/solution/ace-patient-engagement-measure [cited 5 Feb 2021]

  26. Wardian J, Bersabe D, Duke C, Sauerwein TJ. Patient commitment and its relationship to A1C. Clin Diabetes. 2018;36(4):295–304.

    Article  Google Scholar 

  27. Yap CW, Ge L, Ong R, Li R, Heng BH. Development of a scalable and extendable multi-dimensional health index to measure the health of individuals. PLoS One. 2020;15(10):e0240302.

    Article  CAS  Google Scholar 

  28. Ge L, Yap CW, Ong R, Heng BH. Social isolation, loneliness and their relationships with depressive symptoms: a population-based study. PLoS One. 2017;12(8):e0182145.

    Article  Google Scholar 

  29. Myers N. Sample Size and Power Estimates for a Confirmatory Factor Analytic Model in Exercise and Sport: A Monte Carlo Approach. RQES. 2011;82(3) Available from: http://www.ingentaconnect.com/content/aahperd/rqes/2011/00000082/00000003/art00006 [cited 19 Sep 2019].

  30. Haley SM, Jette AM, Coster WJ, Kooyoomjian JT, Levenson S, Heeren T, et al. Late life function and disability instrument II. Development and evaluation of the function component. J Gerontol A Biol Sci Med Sci. 2002;57(4):M217–22.

    Article  Google Scholar 

  31. Cheung YB, Luo N, Ng R, Lee CF. Mapping the functional assessment of cancer therapy-breast (FACT-B) to the 5-level EuroQoL Group’s 5-dimension questionnaire (EQ-5D-5L) utility index in a multi-ethnic Asian population. Health Qual Life Outcomes. 2014;12:180.

    Article  Google Scholar 

  32. Magnezi R, Glasser S, Shalev H, Sheiber A, Reuveni H. Patient activation, depression and quality of life. Patient Educ Couns. 2014;94(3):432–7.

    Article  Google Scholar 

  33. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293–307.

    Article  CAS  Google Scholar 

  34. Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ. 1997;314(7080):572.

    Article  CAS  Google Scholar 

  35. Stephanie. Average inter-Item correlation: definition, example. StatisticsHowTo.com: Elementary Statistics for the rest of us! 2018. Available from: https://www.statisticshowto.com/average-inter-item-correlation/ [cited 11 Aug 2020]

    Google Scholar 

  36. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999;6(1):1–55.

    Article  Google Scholar 

  37. Dinno A. Nonparametric pairwise multiple comparisons in independent groups using Dunn’s test. Stata J. 2015;15(1):292–300.

    Article  Google Scholar 

  38. Bolarinwa. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Nigerian Postgrad Med J Nigeria. 2016;22(4):195–201.

    Article  Google Scholar 

  39. Vahdat S, Hamzehgardeshi L, Hessam S, Hamzehgardeshi Z. Patient involvement in health care decision making: A review. Iran Red Crescent Med J. 2014;16(1) Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3964421/ [cited 13 Aug 2018].

  40. Miedany YE, Gaafary ME, Youssef S, Aroussy NE. Meaningful patient engagement in inflammatory arthritis: development of the Patient Motivation Questionnaire. Clin Rheumatol [Internet]. 2017. Available from: https://doi.org/10.1007/s10067-017-3605-x. [cited 2021 Dec 10]

  41. Skolasky RL, Green AF, Scharfstein D, Boult C, Reider L, Wegener ST. Psychometric properties of the patient activation measure among multimorbid older adults. Health Serv Res. 2011;46(2):457–78.

    Article  Google Scholar 

  42. Epidemiology & Disease Control Division. National Health Surveillance Survey 2007. Singapore: Ministry of Health; 2009. Report no.: ISBN 978-981-08-2420-4. Available from: https://www.moh.gov.sg/content/dam/moh_web/Publications/Reports/2009/nhss2007.pdf [cited 26 Aug 2018]

    Google Scholar 

  43. Chow WL, Wang VW, Low YS, Tse DWL, Lim JFY. Factors that influence the choice of seeking treatment at polyclinics. Singap Med J. 2012;53(2):109–15.

    CAS  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study did not receive any funding from any external body.

Author information

Authors and Affiliations

  1. Health Services and Outcomes Research, National Healthcare Group, 3 Fusionopolis Link #03-08, Nexus@one-north, Singapore, 138543, Singapore

    Lixia Ge, Chun Wei Yap, Palvinder Kaur, Reuben Ong & Bee Hoon Heng

Authors
  1. Lixia Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chun Wei Yap
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Palvinder Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Reuben Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bee Hoon Heng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LG, PK, and BHH contributed to the conceptualization of the study and study design. LG, RO and CWY performed the project administration and data curation. LG conducted data analysis and wrote up and revised the manuscript. All authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Lixia Ge.

Ethics declarations

Ethics approval and consent to participate

Ethics approval for this study was obtained from the ethical committee of the National Healthcare Group Domain Specific Review Board, Singapore (Reference Number: 2015/00269). The study and was conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained from all participants of the study after being informed about the study purpose, procedure and confidentiality of the data collected.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests. The authors confirm that all authors have approved the manuscript for submission and the content of the manuscript has not been published or submitted for publication elsewhere.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ge, L., Yap, C.W., Kaur, P. et al. Psychometric evaluation of the 8-item Altarum Consumer Engagement (ACE) Measure™ in community-dwelling adults in Singapore. BMC Health Serv Res 21, 1347 (2021). https://doi.org/10.1186/s12913-021-07369-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-07369-1

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us