- Research article
- Open access
- Published:
Validation of the Partners at Care Transitions Measure (PACT-M): assessing the quality and safety of care transitions for older people in the UK
BMC Health Services Research volume 20, Article number: 608 (2020)
Abstract
Background
The Partners at Care Transitions Measure (PACT-M) is a patient-reported questionnaire for evaluation of the quality and safety of care transitions from hospital to home, as experienced by older adults. PACT-M has two components; PACT-M 1 to capture the immediate post discharge period and PACT-M 2 to assess the experience of managing care at home. In this study, we aim to examine the psychometric properties, factor structure, validity and reliability of the PACT-M.
Methods
We administered the PACT-M over the phone and by mail, within one week post discharge with 138 participants and one month after discharge with 110 participants. We performed principal components analysis and factors were assessed for internal consistency, reliability and construct validity.
Results
Reliability was assessed by calculating Cronbach’s alpha for the 9-item PACT-M 1 and 8-item PACT-M 2 and exploratory factor analysis was performed to evaluate dimensionality of the scales. Principal components analysis was chosen using pair-wise deletion. Both PACT-M 1 and PACT-M 2 showed high internal consistency and good internal reliability values and conveyed unidimensional scale characteristics with high reliability scores; above 0.8.
Conclusions
The PACT-M has shown evidence to suggest that it is a reliable measure to capture patients’ perception of the quality of discharge arrangements and also on patients’ ability to manage their care at home one month post discharge. PACT-M 1 is a marker of patient experience of transition and PACT-M 2 of coping at home.
Background
Care transition is a sequence of activities before and after discharge, designed to ensure coordination between providers and continuity of care between services, as patients move across health care services [1]. Most transitions require considerable and accurate cooperation between various health services providers and clinical teams; the hospital, the primary care team, social services, home care clinicians, community health professionals, the patient and their relatives or carers [2]. For older, more vulnerable adults with multiple chronic conditions the transition of care from hospital to home is a complicated process often associated with inadequate communication between providers, lack of patient involvement, delays in service provision and limited follow-up [3]. Risk factors such as; patients’ individual needs, level of frailty, support network, accessing health services and social barriers [3,4,5] contribute to the decrease of overall health [1, 6].
Findings from multiple studies indicate that older adults experience more frequent interactions with health care professionals including emergency department visits and hospital admissions that increase the risk of safety incidents occurring during the transfer across settings and between providers [1, 3, 7]. Older adults transferred from hospital to home with more complex health needs are in greater risk to experience safety incidents such as unplanned readmissions within the first thirty days post discharge when compared to the general population [8,9,10]. Safety incidents and adverse events are often related to; insufficient clinical information, medication errors and poor communication between the hospital and primary and community care services. Adverse events during transitions have been observed to occur to one in five patients [11,12,13] and one in six patients over 75 years old are readmitted to hospital within the first month post discharge [14].
Improving the quality of transition relies on developing reliable measurement tools that are scientifically rigorous, evidence based, enable monitoring of safety markers, require easily collected data using available resources, help health services providers to see safety through the eyes of patients and outcomes can be used as a common language of service quality across the organisation [15,16,17]. For instance, the Medical Home Care Coordination Survey (MHCCS) [18] has been developed to assess key components of care continuity (e.g. relationships with providers in hospital, communication of information to patients, post hospital care etc.) from patient’s perspective and has been used as a quality indicator of discharge care [19]. A systematic review by Doyle et al. [20] showed that patient experience self-report measures are a reliable marker of the quality of health services and have been positively associated with patient safety and clinical efficiency for a wide array of health problems and clinical settings, including discharge care. Patient experience surveys have been shown to objectively evaluate clinical outcomes such as continuity of care, compliance with clinical practice guidelines, coordination between professionals and adverse events [20]. Patient reports of the quality of transition obviously reflect their own experience but could be a reasonable reflection of the quality and wider effectiveness of the transition process.
Patient experience is important in understanding the quality of discharge care as it reveals strengths and weaknesses in the safety of services. Patients and caregivers may not know who is coordinating their care needs [21, 22], which health service to contact for advice [23], often struggle to manage their condition and symptoms after discharge [24] and feel frustrated by not receiving clear medication directions [23]. Nevertheless, qualitative accounts of patients followed throughout care transitions showed that high level of comprehension of the information around their health at discharge does not ensure ongoing capability to cope with health and manage symptoms at home weeks after discharge [25].
The Care Transition Measure (CTM) [26] is the most common tool to evaluate transition experience from hospital to home. Although the CTM is a validated instrument designed to assess various components of transitional quality (for example, information transfer, patient empowerment, patient and caregiver preparation, etc.), it only focuses on the immediate post discharge period and does not include safety related components. Other tools measuring quality of care transitions using patients’ experiences have covered either a single area of transition, such as discharge, readmission or referral, or related concepts like care co-ordination, home care, continuity of care and patient satisfaction [18, 22, 26,27,28,29,30,31,32,33,34,35,36,37]. These measures have been designed to address transition challenges of a target population and healthcare system different to the UK (for example, US citizens) and have not included the concept of patient safety across the continuum of transitional care; from the immediate to the longer-term post discharge period.
The PACT-M [38] (Fig. 1) is the only measure developed within the UK health system, that assesses patient experience, quality and safety of care transition. The PACT-M addresses both the immediate post-discharge period and the longer-term experience of managing health and care at home. The development of the PACT-M was an iterative process which has been described elsewhere [38]. The main development stages involved: constructing a theoretical framework, identifying key areas of transitions, designing an item pool to address these areas, evaluating the items through Delphi groups and finally, pilot testing of the measure [38].
Aims of the present study
-
1.
To perform a preliminary assessment of the transition experience for older patients as they move from hospital to home.
-
2.
To explore the factor structure of the PACT-M 1 and PACT-M 2.
-
3.
To assess the scale’s psychometric properties, particularly inter-item correlations and establish the validity and reliability of the PACT-M. Specifically, the present study seeks to to test the internal reliability of the measure using Cronbach’s alpha criterion.
Methods
Study participants
Participants were recruited whilst they were in hospital, over a period of six months. We based the study in a large teaching NHS Foundation Trust in a sample of wards. Our sampling plan included identifying a series of wards caring for older people. These were; cardiothoracic, vascular, surgical, women centre, complex medical care, acute, trauma, stroke, infectious diseases and tropical medicine wards. Eligible patients were identified by senior nurses and were approached for inclusion in the study if the initial interview did not interfere with routine care. Then, the researcher in the field approached potential participants to discuss the study. Inclusion criteria were: English speaking population aged 65 years or older, due to be discharged home, able to provide informed consent, having spent at least one overnight stay in hospital.
This study received ethical approval from the NHS London - Chelsea Research Ethics Committee (ref: 18/LO/0568).
Participants were given the information sheet and consent form and once they agreed to participate, the study researcher (EO) collected demographic information about their age, gender, ethnicity, estimated discharge date and who they live with (Table 1). The researcher informed participants that they would be contacted at two time points post-discharge and offered the option to be contacted by phone or by post. Only participants that were followed-up at time 1 were also followed-up at time 2 (Fig. 2).
The measure
The PACT-M includes a survey designed to address eight components about the patient’s experience of transition of care from hospital to home (see Fig. 1 for the full list of components) [38]. In addition, PACT-M involves a list of seven questions in order to capture potential problems with care and adverse events that are commonly experienced by recently discharged patients [38] (see Supplementary file 1, Table 1 and Supplementary file 2, Table 2).
Measure administration
The majority of participants were contacted by phone. The researchers (EO, BP) performed short phone calls during which, participants were asked to complete the PACT-M. Participants were requested to rate the survey items on an agreement scale (1 to 5 where 1 is strongly disagree and 5 is strongly agree) with the option to choose that the item is ‘not applicable’ and then, they were asked to respond with yes or no answers about potential problems or adverse events with managing their health and care at home. Researchers attempted to call participants up to 3 times on 3 different days and if participants did not answer their phone, they were withdrawn from the study. Five participants, those with more complex health issues or with hearing difficulties indicated they wished to be contacted by post and the questionnaire was sent to the preferred address with a pre-paid, return envelope.
The role of the interviewer is crucial in ensuring low rates of missing values from non-responses; interviewers who are well trained and regularly supervised report low non-responses [39] when administering surveys. For the present study, both interviewers (EO, BP) were trained before administering PACT-M, had regular communication and received supervision from the principal investigator (CV), to discuss any issues or concerns. This enabled the interviews to run smoothly. Once participants had agreed to take part, they almost invariably were willing to complete the interview.
Statistical analyses
Analyses were conducted using IBM SPSS Statistics 25 Software Package [40]. Descriptive statistics and Pearson’s correlations were calculated for all items. We benchmarked correlations against similar work [41] and we defined a strong correlation as 0.70, moderate to substantial as 0.30–0.70 and weak as 0.30 or below.
There were very few (under 0.01%) missing values at time 1 and no missing values at time 2, as the surveys were completed by the project researchers while on the phone with participants or after receiving participants’ answers by mail.
For the purposes of this analysis, we followed the sample size strategy outlined Devellis’ work [42] which states that there is one rule of thumb for calculating acceptable sample size for factor analysis; a ratio of at least 10:1 between the number of participants and the number of items in a survey, with an ideal ratio being 15:1 or 20:1 [41, 43,44,45,46]. In our study, the minimum sample size for factor analysis was satisfied, with a ratio of 14 participants per item for PACT-M 1 and 13 participants per item for PACT-M 2.
Reliability was assessed by calculating Cronbach’s alpha for PACT-M 1 and PACT-M 2. Exploratory factor analysis was conducted to evaluate dimensionality of the scales.
Principal components analysis was chosen, to examine the factor structure of the measure on correlation matrices, using pair-wise deletion. Principal components analysis was chosen due to the exploratory nature of the PACT-M. Kaiser’s criterion was used for choosing the number of factors to retain, as it has been shown to be relatively accurate for samples sizes above 100. Subscales resulting from this analysis were assessed for reliability using Cronbach’s alpha.
Results
Participant characteristics
In total, 70% of people approached agreed to take part in the study, of whom, 138 completed PACT-M 1 (response rate 75%) and 110 responded to PACT-M 2 (response rate 60%).
The mean age of the sample (n = 138) was 79 years (range: 65–95). The majority of participants were male (59%), of Caucasian, British origin (98%) and living with their spouse or partner (69%). Most participants (73%) were recruited from the cardiothoracic/vascular and surgical emergency wards and almost half of participants had been hospitalised for 5 days or less at the point of consent. Further demographic characteristics are presented in Table 1.
The majority of participants (80%) were contacted within the first week after discharge. Participants that reported feeling unwell, requested to be contacted at a later point or participants that have been readmitted to the hospital but were keen to take part in the study were contacted later than one week from their first discharge. Ninety percent of respondents were contacted within 10 days after their return home, and one participant was contacted 48 days post discharge.
Patients’ experiences on the preparation for transition PACT-M 1
Table 2 depicts the 9 items of the PACT-M 1 frequencies. The percentage of participants that used the response options ‘strongly disagree’ varied from 0.7 to 13% per item and varied from 20 to 48% per item for the response option ‘strongly agree’. The percentage of participants that used the response options ‘not applicable’ varied from 0 to 33% per item; items 2 and 5, referring to patients being aware of further help and support when at home (such as; home care or community care arrangements, help with cooking, showering, grocery shopping etc.) received the highest ‘not applicable’ scores (23 and 33% respectively). There were almost no missing values (0.01%).
Item mean scores ranged between 3.59 and 4.37 with a mean of 4.1 and mean SD = 0.9. Seven out of nine items were answered by 90% or more participants (i.e. all participants gave a response other than not applicable). The frequency of responses showed the vast majority of participants (90%) felt prepared to be at home. Nine out of ten patients felt they could ask staff questions about their care at home and felt confident they understood how to manage their medications. More than 80% of participants felt ‘they knew what to do if their health deteriorated’, and agreed they could discuss their worries with hospital staff, and any concerns they might have, had been addressed before leaving the hospital. However, 20% of participants did not feel that staff helped them to prepare for non-medical needs at home (e.g. cooking, grocery shopping, showering) and more than one in ten participants did not know about home/community care arrangements made to support them at home and did not know how to get help from out-of-hospital services (e.g. GP, community care) if they needed to.
Patients’ experiences with managing their health and care at home: PACT-M 2
Table 3 shows the 8 PACT-M 2 item frequencies. The percentage of participants that used the response option ‘strongly agree’ ranged from 35 to 55% and varied from 0 to 4% per item for the response option ‘strongly disagree’. The percentage of participants that used the response option ‘not applicable’ varied from 1 to 8% per item. There were no missing values at time 2.
Mean scores ranged between 4 and 4.37 with a mean of 4.2 and mean SD = 0.9. Over 90% of participants felt confident managing their care at home and knew who to contact and what to do in an emergency. However, one in ten patients felt they could not manage their overall health and care safely at home, and did not feel they knew how to manage their medication. Similarly, one in ten patients reported they did not feel they could talk to health care professionals or a family member about their worries and did not feel supported by their community health care services to manage both medical and non-medical needs (e.g. cooking, cleaning, buying food, dressing).
Problems with healthcare and adverse events
We sought to capture potential problems with care and adverse events by asking participants if they have experienced a list of health related problems. At all follow up points, we asked participants to respond whether they have had any health problem, since their latest discharge from hospital (Table 4).
We asked participants to report any problems they had since their latest admission (rather than since we last spoke to them) to ensure that we will capture the experiences of people that have been readmitted to the hospital in an unplanned manner. The most common problems within the first week post discharge were; emergency appointments with local or hospital services, problems with managing medication and infections. Within the first month post discharge, people most often experienced problems with medication management, having difficulties arranging appointments with health care professionals and visiting health services in an unplanned manner.
Problems with healthcare and adverse events at Time 1: within one week post discharge
One in four participants reported having visited the emergency services or their local surgery/community service in an unplanned manner within the first week after they were discharged. One in six participants reported having issues managing their medication and having had an infection since their discharge. Fewer people (11%) had difficulties accessing their local health professionals and even less patients reported having experienced a fall (8%), issues with pressure ulcers (4%) or having difficulty accessing essential healthcare supplies (1%).
Half of participants reported suffering at least one transition related safety event within seven days after their discharge from hospital. One in four patients experienced between 2 and 3 health issues and 2% experienced four health problems.
Problems with healthcare and adverse events at Time 2: within one week postt discharge
One in three patients reported having visited the emergency services or their local surgery/community service in an unplanned manner one month after their discharge. One in five reported having acquired an infection and having difficulty arranging appointments with their local healthcare practitioners. One in ten participants experienced medication management problems (including accessing and taking medication) and suffered from sores that would not heal (for example, pressure ulcers). Fewer people experienced health problems such as having had a fall (8%) and having difficulty accessing essential healthcare supplies (3%).
Sixty-percent of this study’s participants reported having experienced at least one transition related safety event with their care after one month of their discharge from hospital to their home. 25% patients experienced between two and three health related issues and 2% experienced 4 or 5 health problems.
Respondents reported having experienced more problems with their care at one month post discharge (compared to one week post discharge) as five out of seven health related problems increased between 30 and 200% at Time 2. The problem that increased the most at time 2 was getting essential healthcare supplies. Issues with falls remained the same across both time points.
Exploratory scale structure
Given the low amount of ‘not applicable’ responses (between 0 and 33% for time 1 and 1 to 8% for time 2), for all the exploratory and confirmatory factor analyses we performed, we treated ‘not applicable’ responses as missing values so that the ‘not applicable’ answers would not be included in the analysis.
PACT-M 1: preparation for transition
We conducted an exploratory factor analysis using the principal components analysis extraction method on the 9 PACT-M 1 items to determine the factor structure. The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis; KMO = 0.8, above the commonly recommended value of .6, and Bartlett’s test of sphericity was significant (x2 (138) = 240 p < .001). KMO values for individual items were above the acceptable limit of 0.5 [47, 48]. Finally, the communalities were between 0.2 and 0.6 (Table 5), further confirming that each item shared some common variance with other items. These overall indicators showed that correlations between items were sufficiently large for factor analysis [42, 47, 48].
Using Kaiser’s criterion, the analysis revealed two factors with eigenvalues greater than 1 (4.1, 1.1), with the first factor explaining 45% of the variance and the second a further 13%. The scree plot showed inflexions suggesting one factor structure. The component matrix showed strong loadings of all items on the first factor, with values higher than 0.6 (above the recommended value of 0.3) (Table 5). Only one item (item 2) loaded strongly on the second factor (0.7) and showed good loading value on the first factor (0.5). These results implied a unidimensional factor model of the PACT-M 1.
A number of criteria are used to determine a single dimension of the scale; (i) ratio of first to second eigenvalues was equal or larger than 3, (ii) the variance explained by the first factor to be greater than 20%, (iii) eigenvalues accounting for 10% of variance and greater than 1 and (iv) examining the inflection point on the scree Test [49, 50]. Given the first to second eigenvalue ratio (=3), the high variance explained by the first factor (eigen value> 1), the scree plot presentation and the strong factor loadings of the majority of items on the first factor, we sought to explore the unidimensionality of the PACT-M 1 and conducted a confirmatory factor analysis retaining a single factor assumption.
To explore the internal reliability of the single factor post confirmatory factor analysis, Cronbach alpha was calculated for the PACT-M 1 (where ≥0.70 was considered ‘good’) and average inter-item correlations were explored. Reliability analysis showed an alpha value of 0.84, conveying high internal consistency which can further support the single factor solution (Table 5). PACT-M 1 demonstrated ‘good’ internal reliability with average inter-item correlations ranging between the optimal recommendations; 0.2 to 0.6 with the exception of Item 2 showing lower values than 0.2 when correlated with items 3 and 4 (see Supplementary file 3, Table 3).
PACT-M 2: managing health and care at home
Principal component analysis was conducted on the 8 PACT-M 2 items. The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis, KMO = 0.9 and Bartlett’s test of sphericity was significant (x2 (110) =509.7, p < .001). All KMO values for individual items were > .80 and the communalities were above 0.6 for 7 out of 8 items (see Table 6), indicating high common variance between items. These overall indicators showed that correlations between items were sufficiently large for factor analysis.
Using Kaiser’s criterion, the analysis revealed one factor with eigenvalue greater than 1 (5.1), explaining 64% of the variance. The scree plot clearly showed a single factor structure with strong factor loadings; all values higher than .6 (Table 6). The internal consistency of the one factor solution was excellent (Cronbach’s alpha = 0.92).
Reliability analysis showed an alpha value of 0.92, conveying high internal consistency which can further support the single factor solution. PACT-M 2 demonstrated excellent internal reliability with inter-item correlations ranging between the optimal recommendations; 0.4 to 0.8 (see Supplementary file 4, Table 4).
Discussion
This study explored the reliability and validity of the PACT-M, a tool measuring older patients’ experience during two time points; (i) patients’ experience of preparation for discharge using PACT-M 1 within a week of discharge and (ii) coping at home using PACT-M 2 at one month post discharge. Analysis of the factor structure revealed that both PACT-M 1 and PACT-M 2 showed unidimensional scale characteristics with good levels of internal consistency and reliability. As far as we are aware, the PACT-M is the first to evaluate care transition quality and safety relevant to UK population and can be a suitable marker of the quality of care transitions measuring both pre and post hospital experience.
Although respondents reported feeling prepared for their upcoming discharge for 6 out of 9 discharge arrangements, as many as 20% did not have key information to help continuity of care and half of participants experienced at least one transition related problem with their care within the first week after discharge. Some of participants (12–20%) were not aware of what (if any) arrangements have been made to support them at home, including clinical (e.g. linking patients to community care) or non-clinical needs (e.g. dressing, showering grocery shopping). These findings are consistent with a previous study which sought to measure continuity of care for older patients one month post discharge; the study found that one in five patients they surveyed, felt they had not been provided with information on their diagnosis and treatment requirements, did not have a follow-up plan and felt they have not been involved in discharge preparations [18].
Participants reported feeling confident about managing their medication just before leaving the hospital as well as one month after discharge. However, 1 in 7 experienced problems managing their medication after they returned home both within one week and within a month after discharge. One explanation about this discrepancy might be the complexity of the item phrasing; ‘managing’ medications is an inclusive statement that could involve several aspects including; which medications, when and how to take them and accessing prescriptions. Patients might feel knowledgeable about how to take their medication just before leaving the hospital but management might become more complicated when being at home, as they need to understand, take and access their medication on their own.
The discrepancy between understanding management of medication at hospital but experiencing medication related problems when being at home, might reflect that older patients may feel gratitude to the systems of care and be less likely to raise concerns [51]. Rydeman and Törnquist [52] reported that older people who felt unprepared to manage their care at home were more likely to accept their discharge when they perceived hospital staff as stressed or overburdened.
Participants reported having an overall good and safe transition process one month after discharge. However, our data showed that 60% of surveyed patients experienced at least one problem with their care or adverse event one moth post discharge. Our results also showed that patients are more likely to be readmitted or visit their local health services in an unplanned manner within their first month after discharge rather than within the first week post discharge.
Validation of the PACT-M
In line with Cook’s recommendations for evaluating a clinical instrument’s validity, we discuss the PACT-M validation process in the context of a broad, established argument that assess validity as reflecting “the degree to which a score can be interpreted as representing the intended underlying construct” [53, 54]. This integrated concept of ‘construct validity’ integrates information from five key areas: content, response process, internal structure, relations to other variables and consequences [54]. The first three areas of validity are addressed in present and previous studies, and the last two will be explored more fully in future work.
Content
The content criterion refers to the relationship between the tool’s content and the underlying construct it intends to measure. We used an evidence-based instrument development process to define the theoretical construct that would underline the PACT-M. We achieved that by reviewing existing measures of care transitions and interventions designed to improve transitions for older adults. We then reviewed and incorporated learnings from qualitative studies that explored patients’ and clinical professional’s views and experiences in order to identify key areas of care transition [38]. Onwards, generated an item pool after incorporating feedback from a Patient and Public Involvement (PPI) panel, followed by refining items using a modified Delphi process, to examine content validity of the draft measure [38]. The full PACT-M development process has been described elsewhere [38]. This integrated approach enabled us to identify the full spectrum of transition components, relevant to older patients.
The majority of the PACT-M responses were received over telephone administration. Once contact had been made, participants were almost invariably willing to complete the short interviews, suggesting that both the format and actual questions are understandable and appropriately framed. Telephone administered surveys that include easily understood items, contain middle options on the response scales, are pilot tested with a small sample of the target population and make the necessary adjustments to the wording, can ensure minimal non-responses [39, 55]. Having established these steps during the PACT-M development process [38] we believe that the content of the PACT-M is relevant to the target population and includes items that are easy and quick to respond to.
Response process
This process explores and establishes the researchers’ or observers’ level of understanding of the instrument’s theoretical construct. We designed a training manual for the PACT-M administration which spanned from screening patients and recruitment activities, to the PACT-M administration and following up of participants at all time points, with specific instructions on managing attrition and withdrawal incidences. The training instructions were discussed with all researchers and clinicians involved in recruitment activities. Onwards, we organised consultation sessions with 5 members of our PPI panel in order to explore and ratify the researchers’ understanding of the PACT-M overall format, individual items and scoring system. We then administered the measure in person and by telephone. We also collected feedback on the questionnaire administration from the researcher involved in the pilot testing (please see measure development paper [38] for more details on pilot testing) and discussed personal reflections of the measure administration in roundtable discussions with the project team [38].
This integrated process ensured a more robust measure administration technique that improved overall follow up rates during formal testing of the PACT-M; attrition at time 1 during the pilot stage was 46%, dropped to 25% at time 1 and was further reduced to 15% at time 2 during formal testing [38].
Internal structure
Evidence on the internal structure of the PACT-M derived from internal consistency analysis, reliability statistics, and factor structure. Reliability and validity of the scales was considered satisfactory; reliability analysis of PACT-M 1 and 2 showed alpha values greater than 0.8 conveying high internal consistency. Both PACT-M versions demonstrated ‘good’ internal reliability with average inter-item correlations meeting the optimal recommendations. This means that the PACT-M produced valid and consistent results across all items. The factor analysis revealed a single factor solution with good factor loadings for all items.
The PACT-M has shown evidence to suggest that it is a reliable measure to capture patients’ perception of the quality of discharge arrangements that can reflect on patients’ ability to manage their care at home further down their care journey. Research findings indicate that patients have a deep understanding of the strengths and deficiencies of health services they are provided with and they are able to identify errors that professionals may fail to notice [56], report safety incidents that may have been overlooked [11] and identify factors related to adverse events [57,58,59]. Lower item scores in either component of PACT-M may help clinicians identify areas that could be improved.
Relationship to other measures
In order to assess the relationship between other measures, PACT-M scores need to be correlated with scores from other similar instruments assessing the same underlying construct. We do not yet have evidence of comparing the PACT-M variables to other scoring systems but that would be an area for future research. This should be done with caution as most measures assessing discharge care only address one area of the transition process at one time point [1, 18, 33] whereas the PACT-M evaluates patient experience across a broader transition time span. Boge et al. [30], recently developed a measure to assess quality of discharge process in older patients called the Discharge Care Patient Experience Survey (DICARES). Validity testing showed a three factor structure distinguishing between items referring to discharge and post discharge care which is in line with our two-time point approach of measure administration. However, in this study, all surveys were completed while patients were in hospital, which suggests a limited insight into the full range of care transition.
Consequences
The validity criterion of ‘consequences’ refers to the outcomes of the tool implementation; for example, whether the instrument has contributed to changing health services delivery or improved quality of patient care. We have provided evidence that the PACT-M can be used as a metric for discharge and continuity of care quality and can be used to identify areas for improvement of services targeted to care transitions. The present research is one of five work streams that collectively aim to gain a more insightful understanding of care transitions from patients’ [25] and clinical professionals’ [60] point of view. We will evaluate the PACT-M implementation as part of the intervention evaluation process that will be conducted by our fellow researchers of this programme of work, through a randomised controlled trial, which will provide evidence of the role and impact of PACT-M in the wider context of healthcare improvement.
Strengths and limitations
The main strength of the study is the development of a valid and reliable tool that can be used to improve safety at care transitions. The PACT-M showed it can be a reliable indicator of quality and safety of care at two time points; at preparation for discharge and coping at home. An additional benefit of the PACT-M is that it is quick and not burdensome for patients to complete. Furthermore, the PACT-M tool is the only available measure that collects patient feedback on the experience of care transition at the immediate and longer post discharge period. This study has some limitations. Study participants were English speaking with cognitive capacity to consent, mostly of Caucasian ethical background. Non-English speaking older people unable to compete the PACT-M were not included in the study. Patients who were unable to speak English, unable to read and write or unable to complete the questionnaire were excluded. Thus, the findings may not be applicable to the potentially more vulnerable populations who may have difficulties communicating. Future studies should investigate ways to use the PACT-M in trials including a more diverse population, especially older more vulnerable people who live alone.
Conclusion
The results of the study suggest that the PACT-M shows acceptable validity and reliability. Further research is needed to assess the test-retest reliability criterion, as well as the PACT-M’s usability by a patient’s proxy, for example, a caregiver. Future research should explore the PACT-M validity by comparing the PACT-M with other similar scoring systems. Health services providers aiming to evaluate and improve transition quality may use the PACT-M to identify areas for improvement; by exploring the number of patients that disagree with the statements included in the PACT-Mat the point of discharge and at the immediate and longer term post-discharge period.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Abbreviations
- PACT-M:
-
Partners at Care Transitions Measure
- NHS:
-
National Health Service
- GP:
-
General Practitioner
- CTM:
-
Care Transitions Measure
- PPI:
-
Patient and Public Involvement
References
Coleman EA, Boult C. Improving the Quality of Transitional Care for Persons with Complex Care Needs. J Am Geriatr Soc [Internet]. 2003 1 [cited 2019 Mar 19];51(4):556–7. Available from: http://doi.wiley.com/10.1046/j.1532-5415.2003.51186.x.
Buurman BM, Hoogerduijn JG, de Haan RJ, Abu-Hanna A, Lagaay AM, Verhaar HJ, et al. Geriatric Conditions in Acutely Hospitalized Older Patients: Prevalence and One-Year Survival and Functional Decline. Lama JR, editor. PLoS One [Internet]. 2011 14 [cited 2019 Mar 19];6(11):e26951. Available from: https://dx.plos.org/10.1371/journal.pone.0026951.
Naylor MD. Advancing high value transitional care: the central role of nursing and its leadership. Nurs Adm Q [Internet]. 2012;36(2):115–26 Available from: https://journals.lww.com/naqjournal/Fulltext/2012/04000/Advancing_High_Value_Transitional_Care__The.5.aspx.
Zurlo A, Zuliani G. Management of care transition and hospital discharge. Aging Clin Exp Res [Internet]. 2018 8 [cited 2019 Mar 19];30(3):263–70. Available from: http://link.springer.com/10.1007/s40520-017-0885-6.
McMullan M, Jones R, Lea S. Patient safety: numerical skills and drug calculation abilities of nursing students and Registered Nurses. J Adv Nurs [Internet]. 2010 1 [cited 2019 Mar 19];66(4):891–9. Available from: http://doi.wiley.com/10.1111/j.1365-2648.2010.05258.x.
Naylor MD, Feldman PH, Keating S, Koren MJ, Kurtzman ET, Maccoy MC, et al. Translating research into practice: transitional care for older adults. J Eval Clin Pract [Internet]. 2009;15(6):1164–70. Available from: https://doi.org/10.1111/j.1365-2753.2009.01308.x.
Anderson G. Chronic care [Internet]. 2010. Available from: https://www.rwjf.org/en/library/research/2010/01/chronic-care.html.
Casalini F, Salvetti S, Memmini S, Lucaccini E, Massimetti G, Lopalco PL, et al. Unplanned readmissions within 30 days after discharge: improving quality through easy prediction. Int J Qual Heal Care [Internet]. 2017 1 [cited 2019 Aug 7];29(2):256–61. Available from: https://academic.oup.com/intqhc/article-lookup/doi/10.1093/intqhc/mzx011.
Readmission rates: what can we learn from the Netherlands? | The Nuffield Trust [Internet]. [cited 2019 Aug 7]. Available from: https://www.nuffieldtrust.org.uk/news-item/readmission-rates-what-can-we-learn-from-the-netherlands.
Jencks SF, Williams M V, Coleman EA. Rehospitalizations among Patients in the Medicare Fee-for-Service Program. N Engl J Med [Internet]. 2009 2 [cited 2019 Aug 7];360(14):1418–28. Available from: http://www.nejm.org/doi/abs/10.1056/NEJMsa0803563.
Vincent C. Patient safety [Internet]. Wiley-Blackwell; 2010 [cited 2019 Apr 16]. 416 p. Available from: https://www.wiley.com/en-gb/Patient+Safety%2C+2nd+Edition-p-9781405192217.
Friedman B, Jiang HJ, Elixhauser A. Costly hospital readmissions and complex chronic illness. Inq J Heal Care Organ Provision, Financ. 2008;45(4):408–21.
Neale G, Woloshynowych M, Vincent C. Exploring the Causes of Adverse Events in NHS Hospital Practice. J R Soc Med [Internet]. 2001 23 [cited 2019 Aug 7];94(7):322–30. Available from: http://journals.sagepub.com/doi/10.1177/014107680109400702.
England Safe, compassionate care for frail older people using an integrated care pathway [Internet]. [cited 2019 Aug 7]. Available from: https://www.england.nhs.uk/wp-content/uploads/2014/02/safe-comp-care.pdf.
Vincent C, Amalberti R. Safer healthcare : strategies for the real world 157 p.
Vincent C, Burnett S, Carthey J, Riddell Bamber J. The measurement and monitoring of safety Drawing together academic evidence and practical experience to produce a framework for safety measurement and monitoring Spotlight THE MEASUREMENT AND MONITORING OF SAFETY [Internet]. 2013 [cited 2019 Jan 16]. Available from: https://www.health.org.uk/sites/default/files/TheMeasurementAndMonitoringOfSafety_fullversion.pdf.
Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB, Thompson DA, et al. Creating High Reliability in Health Care Organizations. Health Serv Res [Internet]. 2006 1 [cited 2019 Aug 7];41(4p2):1599–617. Available from: http://doi.wiley.com/10.1111/j.1475-6773.2006.00567.x.
Hadjistavropoulos H, Biem H, … DS-… J for Q in, 2008 undefined. Patient perceptions of hospital discharge: reliability and validity of a Patient Continuity of Care Questionnaire. academic.oup.com [Internet]. [cited 2019 Mar 19]; Available from: https://academic.oup.com/intqhc/article-abstract/20/5/314/1795431.
Jones JM, Ferguson S, Edwards E, Walton T, McCurdy N, Howell D. Experiences of care delivery: endometrial cancer survivors at end of treatment. Gynecol Oncol [Internet]. 2012;124(3):458–64 Available from: http://www.sciencedirect.com/science/article/pii/S009082581100881X.
Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open [Internet]. 2013 [cited 2019 Aug 7];3(1):e001570. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23293244.
Howard-Anderson J, Busuttil A, Lonowski S, Vangala S, Afsar-manesh N. From discharge to readmission: Understanding the process from the patient perspective. J Hosp Med [Internet]. 2016 1 [cited 2019 Mar 19];11(6):407–12. Available from: http://www.journalofhospitalmedicine.com/jhospmed/article/127428/patients-perspective-readmissions.
Vom Eigen KA, Walker JD, Edgman-Levitan S, Cleary PD, Delbanco TL. Carepartner Experiences with Hospital Care [Internet]. Vol. 37, Care. 1999 [cited 2019 Mar 19]. Available from: https://www.jstor.org/stable/pdf/3767205.pdf?refreqid=excelsior%3Ac3e831a867c769c76d6f8fd59bbf7061.
Greysen SR, Harrison JD, Kripalani S, Vasilevskis E, Robinson E, Metlay J, et al. Understanding patient-centred readmission factors: a multi-site, mixed-methods study. BMJ Qual Saf [Internet]. 2017 [cited 2019 mar 19];26(1):33–41. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26769841.
Waring J, Marshall F, Bishop S. Understanding the occupational and organizational boundaries to safe hospital discharge. J Health Serv Res Policy [Internet]. 2015 2 [cited 2019 Mar 19];20(1_suppl):35–44. Available from: http://journals.sagepub.com/doi/10.1177/1355819614552512.
Hardicre N, Birks Y, Murray J, Sheard L, open LH-B, 2017 undefined. Partners at Care Transitions (PACT)—exploring older peoples’ experiences of transitioning from hospital to home in the UK: protocol for an observation and. bmjopen.bmj.com [Internet]. [cited 2019 Mar 19]; Available from: https://bmjopen.bmj.com/content/7/11/e018054?int_source=trendmd&int_medium=trendmd&int_campaign=trendmd.
Coleman E, Mahoney E, care CP-M, 2005 undefined. Assessing the quality of preparation for posthospital care from the patient’s perspective: the care transitions measure. JSTOR [Internet]. [cited 2019 Mar 19]; Available from: https://www.jstor.org/stable/3768223.
Parry C, Mahoney E, Chalmers S, care EC-M, 2008 undefined. Assessing the quality of transitional care: further applications of the care transitions measure. journals.lww.com [Internet]. [cited 2019 Mar 19]; Available from: https://journals.lww.com/lww-medicalcare/Abstract/2008/03000/Assessing_the_Quality_of_Transitional_Care_.12.aspx.
Teale E, ageing JY-A and, 2015 undefined. A Patient Reported Experience Measure (PREM) for use by older people in community services. academic.oup.com [Internet]. [cited 2019 Mar 19]; Available from: https://academic.oup.com/ageing/article-abstract/44/4/667/66221.
Baker R, Preston C, … FC-BQ&, 1999 undefined. Measuring patients’ attitudes to care across the primary/secondary interface: the development of the patient career diary. qualitysafety.bmj.com [Internet]. [cited 2019 Mar 19]; Available from: https://qualitysafety.bmj.com/content/8/3/154.short.
Boge RM, Haugen AS, Nilsen RM, Harthug S. Elderly patients’ (≥65 years) experiences associated with discharge; Development, validity and reliability of the Discharge Care Experiences Survey. Sullivan P, editor. PLoS One [Internet]. 2018 7 [cited 2019 May 29];13(11):e0206904. Available from: http://dx.plos.org/10.1371/journal.pone.0206904.
Burke RE, Guo R, Prochazka A V, Misky GJ. Identifying keys to success in reducing readmissions using the ideal transitions in care framework. BMC Health Serv Res [Internet]. 2014 23 [cited 2019 Mar 19];14(1):423. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-423.
Leppin A, Gionfriddo M, … MK-J internal, 2014 undefined. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. jamanetwork.com [Internet]. [cited 2019 Mar 19]; Available from: https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/1868538.
Zlateva I, Anderson D, Coman E, Khatri K, Tian T, Fifield J. Development and validation of the Medical Home Care Coordination Survey for assessing care coordination in the primary care setting from the patient and provider perspectives. BMC Health Serv Res [Internet]. 2015 7 [cited 2019 Mar 19];15(1):226. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-015-0893-1.
Uittenbroek RJ, Reijneveld SA, Stewart RE, Spoorenberg SLW, Kremer HPH, Wynia K. Development and psychometric evaluation of a measure to evaluate the quality of integrated care: the Patient Assessment of Integrated Elderly Care. Heal Expect [Internet]. 2016 Aug [cited 2019 Mar 19];19(4):962–72. Available from: http://doi.wiley.com/10.1111/hex.12391.
McLeod J, Stolee P, Walker J, Heckman G. Measuring Care Transition Quality for Older Patients with Musculoskeletal Disorders. Musculoskeletal Care [Internet]. 2014 Mar [cited 2019 Mar 19];12(1):13–21. Available from: http://doi.wiley.com/10.1002/msc.1043.
Jenkinson C, Coulter A, … SB-QSH, 2002 undefined. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. qualitysafety.bmj.com [Internet]. [cited 2019 Mar 19]; Available from: https://qualitysafety.bmj.com/content/11/4/335?utm_source=TrendMD&utm_medium=cpc&utm_campaign=BMJ_Qual_Saf_TrendMD-0.
Grimmer K, Health JM-IJ for Q in, 2001 undefined. The development, validity and application of a new instrument to assess the quality of discharge planning activities from the community perspective. academic.oup.com [Internet]. [cited 2019 Mar 19]; Available from: https://academic.oup.com/intqhc/article-abstract/13/2/109/1815707.
Oikonomou E, Chatburn E, Higham H, Murray J, Lawton R, Vincent C. Developing a measure to assess the quality of care transitions for older people. BMC Health Serv Res [Internet]. 2019 19 [cited 2019 Jul 23];19(1):505. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-019-4306-8.
de Leeuw ED. Reducing missing data in surveys: an overview of methods. Qual quant [Internet]. 2001;35(2):147–60. Available from: https://doi.org/10.1023/A:1010395805406.
IBM Corp. Released 2017. IBM SPSS statistics for windows, version 25.0. Armonk, NY: IBM Corp.
Morgado FFR, Meireles JFF, Neves CM, Amaral ACS, Ferreira MEC. Scale development: ten main limitations and recommendations to improve future research practices. [cited 2019 Apr 15]; Available from: http://www.scielo.br/pdf/prc/v30/1678-7153-prc-s41155-016-0057-1.pdf.
DeVellis R. Scale development: Theory and applications [Internet]. 2016 [cited 2019 Mar 19]. Available from: https://books.google.co.uk/books?hl=en&lr=&id=231ZDwAAQBAJ&oi=fnd&pg=PT15&dq=Scale+development:+Theory+and+applications&ots=hE2TwwI6Ty&sig=kfuitgG2J_dP0i1IIUYmUdwxmlU.
Tsang S, Royse CF, Terkawi AS. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth [Internet]. 2017 e. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28616007.
Prinsen CAC, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” – a practical guideline. Trials [Internet]. 2016 13 [cited 2019 Apr 15];17(1):449. Available from: http://trialsjournal.biomedcentral.com/articles/10.1186/s13063-016-1555-2.
Belle G Van. Statistical Rules of Thumb. John Wiley Sons [Internet]. 2011; Available from: http://vanbelle.org/presentations/CHSTalkNoPictures.pdf.
Briggs SR, Cheek JM. The role of factor analysis in the development and evaluation of personality scales. J Pers [Internet]. 1986 1 [cited 2019 Apr 15];54(1):106–48. Available from: http://doi.wiley.com/10.1111/j.1467-6494.1986.tb00391.x.
Osborne JW, Costello AB, Kellow JT. Best Practices in Exploratory Factor Analysis. In: Best Practices in Quantitative Methods [Internet]. 2455 Teller Road, Thousand Oaks California 91320 United States of America : SAGE Publications, Inc.; [cited 2019 Apr 15]. p. 86–99. Available from: http://methods.sagepub.com/book/best-practices-in-quantitative-methods/d8.xml.
Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, et al. Psychometric Evaluation and Calibration of Health-Related Quality of Life Item Banks. Med Care [Internet]. 2007 [cited 2019 Apr 15];45(Suppl 1):S22–31. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17443115.
Slocum-Gori SL, Zumbo BD. Assessing the Unidimensionality of Psychological Scales: Using Multiple Criteria from Factor Analysis. Soc Indic Res [Internet]. 2011 Jul 13 [cited 2019 Apr 15];102(3):443–61. Available from: http://link.springer.com/10.1007/s11205-010-9682-8.
Schjoedt L, Craig JB. Development and validation of a unidimensional domain-specific entrepreneurial self-efficacy scale. Int J Entrep Behav Res [Internet]. 2017 9 [cited 2019 Apr 15];23(1):98–113. Available from: http://www.emeraldinsight.com/doi/10.1108/IJEBR-11-2015-0251.
Hvalvik S, Dale B. The Transition from Hospital to Home: Older People’s Experiences. Open J Nurs [Internet]. 2015 [cited 2019 16];05(07):622–31. Available from: http://www.scirp.org/journal/doi.aspx?DOI=10.4236/ojn.2015.57066.
Rydeman I, Törnkvist L. Getting prepared for life at home in the discharge process - from the perspective of the older persons and their relatives. Int J Older People Nurs [Internet]. 2010 1 [cited 2019 Apr 16];5(4):254–64. Available from: http://doi.wiley.com/10.1111/j.1748-3743.2009.00190.x.
Cook DA, Beckman TJ. Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application. Am J Med [Internet]. 2006 Feb 1 [cited 2019 Jul 16];119(2):166.e7–166.e16. Available from: https://www.sciencedirect.com/science/article/pii/S0002934305010375.
Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Heal Sci Educ [Internet]. 2014 2 [cited 2019 Jul 16];19(2):233–50. Available from: http://link.springer.com/10.1007/s10459-013-9458-4.
Vallance JK, Eurich DT, Gardiner PA, Taylor LM, Stevens G, Johnson ST. Utility of telephone survey methods in population-based health studies of older adults: an example from the Alberta Older Adult Health Behavior (ALERT) study. BMC Public Health [Internet]. 2014;14:486. Available from: https://pubmed.ncbi.nlm.nih.gov/24884997.
Lawton R, Armitage G. The role of the patient in clinical safety [Internet]. [cited 2019 Apr 16]. Available from: https://www.health.org.uk/sites/default/files/TheRoleOfThePatientInClinicalSafety.pdf.
Hernan AL, Giles SJ, Fuller J, Johnson JK, Walker C, Dunbar JA. Patient and carer identified factors which contribute to safety incidents in primary care: a qualitative study. BMJ Qual Saf [Internet]. 2015 [cited 2019 Apr 16];24(9):583–93. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25972223.
Lang S, Garrido MV, Heintze C. Patients’ views of adverse events in primary and ambulatory care: a systematic review to assess methods and the content of what patients consider to be adverse events. BMC Fam Pract [Internet]. 2016 27 [cited 2019 Apr 16];17(1):6. Available from: http://www.biomedcentral.com/1471-2296/17/6.
Giles SJ, Lawton RJ, Din I, McEachan RRC. Developing a patient measure of safety (PMOS). BMJ Qual Saf [Internet]. 2013 [cited 2019 Apr 16];22(7):554–62. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23447658.
Baxter R, O’Hara J, Murray J, Sheard L, open AC-B, 2018 undefined. Partners at Care Transitions: exploring healthcare professionals’ perspectives of excellence at care transitions for older people. bmjopen.bmj.com [Internet]. [cited 2019 Mar 19]; Available from: https://bmjopen.bmj.com/content/8/9/e022468.abstract.
Acknowledgements
We would like to thank Dr. Alex Bottle who provided valuable feedback for this paper’s statistical analysis and Dr. Gurpreet Reen whose conversation supported the statistical development. We would also like to thank Ann Murphy for her invaluable contribution to the recruitment activities.
Funding
This report is independent research funded by the National Institute for Health Research (National Institute for Health Research Programme Grants for Applied Health Research, Partners at Care Transitions (PACT): Improving patient experience and safety at transitions in care, RP-PG-1214-20017. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
Role of funding body: Our representative from the funding body, the NIHR, attended steering groups occasionally but the funding body has no other direct involvement in any aspect of the study design; collection, analysis and interpretation of the data; in the writing of the report or in the decision to submit the paper for publication.
Disclaimer: The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.
Charles Vincent is supported by the Health Foundation.
Author information
Authors and Affiliations
Contributions
CV and RL conceived the study and CV oversaw the scientific direction. EO recruited, followed up participants, conducted the analysis and drafted the paper. BP contributed in following up recruited participants. CV, BP, JM, RL and HH revised the paper. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
This study has received ethical approval from the NHS Research Ethics Committee: London - Chelsea Research Ethics Committee [ref: 18/LO/0568] and is currently recruiting participants. The aim of the study is to develop and validate a measure to assess quality and safety of transitions.
We obtained written informed consent from all study participants.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Additional file 1.
Table 1. PACT-M 1 items.
Additional file 2.
Table 2. PACT-M 2 items.
Additional file 3.
Table 3. Inter-item correlations PACT-M 1, n = 138.
Additional file 4.
Table 4. Inter-item correlations PACT-M 2, n = 110.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Oikonomou, E., Page, B., Lawton, R. et al. Validation of the Partners at Care Transitions Measure (PACT-M): assessing the quality and safety of care transitions for older people in the UK. BMC Health Serv Res 20, 608 (2020). https://doi.org/10.1186/s12913-020-05369-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12913-020-05369-1