Skip to main content

Usability and perceived usefulness of patient-centered medication reconciliation using a personalized health record: a multicenter cross-sectional study

Abstract

Background

Adoption of a personal health record (PHR) depends on its usability and perceived usefulness. Therefore, we aimed to assess the usability and perceived usefulness of an online PHR used for medication reconciliation and to assess the association between patient-, clinical-, hospital-, and ICT-related factors and the usability and perceived usefulness at both the in- and outpatient clinics.

Methods

A multicenter cross-sectional study was conducted with patients with either an outpatient visit (rheumatology ward) or planned admission in the hospital (cardiology, neurology, internal medicine or pulmonary wards). All patients received an invitation to update their medication list in the PHR 2 weeks prior to their appointment. One month after the hospital visit, PHR-users were asked to rate usability (using the System Usability Scale (SUS)) and perceived usefulness on a 5-point Likert scale. The usability and perceived usefulness were classified according to the adjective rating scale of Bangor et al. The usability was furthermore dichotomized in the categories: low (SUS between 0 and 51) and good (SUS 51–100) usability. Associations between patient-, clinical-, hospital-, and ICT-related factors and the usability and perceived usefulness were analysed.

Results

255 of the 743 invited PHR-users completed the questionnaire. 78% inpatients and 83% outpatients indicated that usability of the PHR was good. There were no significant association between patient-, clinical-, hospital-, and ICT-related factors and the usability of the PHR. The majority of the patients (57% inpatients and 67% outpatients) classified perceived usefulness of the PHR as good, excellent, or best imaginable. Outpatients who also used the PHR for other drug related purposes reported a higher perceived usefulness (adjusted odds ratio 20.0; 95% confidence interval 2.36–170). Besides that, there was no significant association between patient-, clinical-, hospital-, and ICT-related factors and the perceived usefulness of the PHR.

Conclusions

The majority of the patients indicated that the PHR for medication reconciliation was useful and easy to use, but there is still room for improvement. To improve the intervention, further research should explore patients’ barriers and facilitators of using a PHR for medication reconciliation.

Peer Review reports

Background

For safe and effective clinical care it is essential to know patients actual medication use and to record this in the patient file [1]. It is proven that an accurate medication list can reduce medication errors, especially in high risk patients [2,3,4,5]. However, an accurate list of a patients current medication is hardly available, as up to 100% of the in- and outpatients have at least one medication discrepancy, defined as an unexplained difference between the patient’s drug list and the medication a patient is actually taking [6,7,8,9,10,11].

The gold standard for reducing medication discrepancies is medication reconciliation [12]. The Institute of Healthcare Improvement defines medication reconciliation as “the process of identifying the most accurate list of a patient’s current medicines including the name, dosage, frequency and route — and comparing them to the current list in use, recognizing and documenting any medication discrepancies, finally resulting in a complete list of medications” [13]. A recent study indicated that an online personal health record (PHR), a secure online website that gives patients access to personal health information, is comparable with medication reconciliation for the identification of clinically relevant medication discrepancies in patients’ drug lists [14]. Furthermore, a PHR has the ability to increase patient empowerment which is defined as “the involvement in their own care by individuals, with the goal to make competent, well-informed decisions about their health and health care and take action to support those decisions” [15].

According to the technology acceptance model, acceptance and usage of a PHR depends on the usability and patients’ perceived usefulness of the PHR [16]. Usability of a PHR for the identification of medication discrepancies has previously been investigated with a validated questionnaire (System Usability Scale (SUS) or Post-Study System Usability Questionnaire) in three studies [17,18,19]. In these studies an acceptable usability of the PHR was observed. Besides that, Marien et al. observed a high perceived usefulness (agreement range: 80–100%) of the PHR [18]. However, in all three studies a limited number of patients derived from a heterogeneous population was included (N < 30) and these studies were performed in a single centre [17,18,19]. As recent studies showed that the capability and motivation of patients using a PHR is influenced by personal factors such as age, educational level, ethnicity and socioeconomic status [15, 20], the usability and perceived usefulness of a PHR used for medication reconciliation may also be associated with patient related factors. Up till now, only one study examined the association between patient-, clinical-, hospital-, and ICT-related factors and the usability and perceived usefulness of such a PHR. This study observed a positive association between the usability and the number of logins to the PHR, but only a limited number of variables was included [18]. Knowledge of these factors may contribute to a more targeted approach for increasing patients’ usage of a PHR. Therefore, the aim of this study was to assess the usability and perceived usefulness of an online PHR used for medication reconciliation and to assess the association between patient-, clinical-, hospital-, and ICT-related factors and the usability and perceived usefulness at both the in- and outpatient clinics.

Methods

Setting

A multicenter cross-sectional study was conducted at an outpatient (department of rheumatology of the Sint Maartenskliniek, Nijmegen, the Netherlands) and inpatient clinic (departments of cardiology, neurology, internal medicine and pulmonary of the Amphia Hospital, Breda, the Netherlands). Other than using the same PHR, the in- and outpatient clinics have no relationship to each other.

Patients scheduled for a hospital admission in the period of May to October 2019 or an outpatient rheumatologic visit in the period of October and November 2018 were eligible for this study. Two weeks prior to the planned visit all patients received an invitation to verify their medication list in the online PHR and to adjust their medication list if necessary. About 1 month after the hospital visit, all patients who completed the medication check using the PHR at the Amphia Hospital and 300 random selected PHR-users (of the 2425 PHR-users) of the Sint Maartenskliniek received an invitation by email to participate in this study. The questionnaire was sent using Castor EDC, a cloud-based Electronic Data Capture platform [21]. Two weeks after the first invitation a reminder was sent to complete the questionnaire.

Ethical considerations

The methods were carried out in accordance with relevant guidelines and regulations like the Declaration of Helsinki and the European General Data Protection Regulation. The study performed at the Amphia Hospital (N2019–0212) was approved by the Medical Ethics Committee of Utrecht, the Netherlands, and the study performed at the Sint-Maartenskliniek (2018–4873) was approved by the Commission on Research Involving Human Subjects of Arnhem-Nijmegen, the Netherlands. All patients provided written informed consent prior to starting the questionnaire.

Participants

Eligible patients were included when they were 18 years or older, able to read the Dutch language, had an email address recorded in the electronical heath record and gave informed consent. Patients were excluded if they did not complete the questionnaire in Castor EDC.

Personal health record

In this study a PHR (Zorgdoc®, Eindhoven, the Netherlands) designed for patients to update their own medication list was evaluated. The PHR system could be accessed with two interfaces: a website for patients and one for healthcare professionals. Both components contain a patient’s medication file; one owned by the patients (Additional file 1) and one by the healthcare professional (Additional file 2). Both components are synchronized, giving the users (patients and professionals) access to the information that has been captured in either file.

The patient’s medication files in the PHR are composed of patient’s input and drug information derived from the Nationwide Medication Record System, a digital nationwide network which exchanges medication dispensing data form all pharmacies in the Netherlands [22]. Patients received an automated invitation to update their medication file approximately 2 weeks prior to their visit. During the verification process, patients were asked to verify the shown medication information derived from the Nationwide Medication Record System. When the patient had finished the verification process, a healthcare professional validated the entered drug information and the drug list was updated in the electronic health record file.

Outcome measures

The primary outcome of this study was the number of patients that indicated a good usability of the PHR at both the in- and outpatient setting. The secondary outcomes were the score of the perceived usefulness and the association between usability and perceived usefulness and patient-, clinical-, hospital-, and ICT-related factors.

Measuring usability

Based on the technology acceptance model, the usability and perceived usefulness were examined. Usability of the PHR was measured with the validated SUS translated to Dutch [23, 24]. The SUS questionnaire consists of ten statements (shown in Fig. 1) rated on a 5-point Likert scale. SUS-scores range from 0 (negative) to 100 (positive) and were calculated according to Davis’s guidelines [16]. To calculate the SUS, first the score contributions from each item were summed. Each item’s score contribution ranged from 0 to 4. For the positively worded items of the questionnaire, the score contribution was the scale position minus one point. For the negatively worded items of the questionnaire, the contribution was five points minus the scale position. The sum of all these scores was then multiplied by 2.5 to get the overall SUS-score [16].

The usability was categorized into seven categories using an adjective rating scale as described by Bangor et al.:

  • worst imaginable: SUS-score 0–12.5;

  • awful: SUS-score 12.6–20.3;

  • poor: SUS-score 20.4–35.7;

  • ok: SUS-score 35.8–50.9;

  • good: SUS-score 51.0–71.4;

  • excellent: SUS-score 71.5–85.5; and

  • best imaginable: SUS-score 85.6–100 [25].

As we expected no linear relation between the SUS-score and the patient-, clinical-, hospital-, and ICT-related factors, we dichotomized these seven categories in the categories: low (SUS between 0 and 51) and good (SUS 51–100) usability [26].

Measuring perceived usefulness

Perceived usefulness was assessed with the perceived usefulness questionnaire by Davis et al. as basis and adjusted to fit the purpose of our PHR [16]. The created questionnaire consisted of two parts rated on a 5-point Likert scale. The first part contained seven questions (shown in Fig. 2) focusing on the perceived usefulness of the PHR itself and part two consisted of three questions which explored the perceived usefulness of the PHR compared to the gold standard, medication reconciliation performed by a healthcare professional. In our research, medication reconciliation was performed by a pharmacy technician who compiled the best possible medication history according to the standard operating procedure according to the World Health Organization [12].

Like the SUS, a score of the perceived usefulness was calculated by summing the score contributions (scale position minus one point) of the first seven questions of the perceived usefulness questionnaire. Based on the method described by ‘measuring usability’, the sum of the seven scores was multiplied by 3.57 (to reach a maximum score of 100) to get the overall perceived usefulness score. As there is no validated categorization of the perceived usefulness, the adjective rating scale of Bangor et al. was applied [25].

Data collection

The data were collected in Castor EDC. Based on literature [15, 18, 27], patient-, clinical-, hospital-, and ICT-related factors were collected (Table 1). Over-the-counter medication was defined as drugs that were sold to patients without a doctor’s prescription [28]. Known comorbidities were extracted from the EHR which were classified according to the International Classification of Diseases-10 [29]. All diagnoses in the patient’s medical record with the status ‘current’ were taken into account. To make sure that the list was complete and correct, the information was checked and supplemented with comorbidities based on drug information of the best possible medication history.

Table 1 Collected patient-, clinical-, hospital-, and ICT-related factors. During the study, the following patient-, clinical-, hospital-, and ICT-related factors were collected from the patient, electronical health record or personal health record

Statistical analysis

First, descriptive analyses were performed to describe usability and perceived usefulness of the PHR at both the in- and outpatient setting. Descriptive statistics were provided using mean (± standard deviation [SD]) or median (interquartile range [IQR]) values depending on the (non-)parametric distribution of measured variables.

In order to examine the association of the patient-, clinical-, hospital-, and ICT-related factors and the usability (low usability versus good usability) and perceived usefulness (first quartile (Q1) versus fourth quartile (Q4) of the perceived usefulness score), the factors were first entered into a univariate logistic regression model after which the significant risk factors (P < .1) were selected and incorporated into the full model logistic regression analyses. An independent-samples nonparametric test was used to compare patient characteristics between the in- and outpatient setting. To determine the reliability of our results, an independent-sample nonparametric test was used to compare the age, number of drugs and number of logins to the PHR between the participating and non-participating patients. Furthermore, a Chi-square test was used to compare the gender and the admitted medical department between the participating and non-participating patients. Results were considered statistically significant at P < .05. Data were analysed using IBM SPSS Statistics software version 25.

Results

About half of the patient (43% inpatients and 46% outpatients) accepted the invitation to perform medication reconciliation by using a PHR (Additional file 3). All patients who started using the PHR also finished the medication reconciliation process. Of the PHR-users, 743 patients were invited for the questionnaire of which 255 (34%) patients completed the questionnaire. The survey response rate was slightly higher at the inpatient setting (40%) compared to the outpatient setting (26%). Responders had a median age of 65 and 59 (for inpatients and outpatients, respectively), covered all educational levels, were fairly experienced with digital devices, used a median number of 7 and 5 drugs (for inpatients and outpatients respectively), were very knowledgeable regarding their medications and had many changes to their drugs in the last 12 months (Table 2). Non-participating outpatients were significantly younger (median age 48; IQR: 36–60, P < .001) and used less drugs (median 2; IQR: 1–6, P < .001) than participating outpatients. Furthermore, non-participating inpatients were more often a female (43% versus 31%, P = .02) compared to the participating inpatients.

Table 2 Characteristics of patients with an outpatient visit (at the rheumatology ward) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards). More detailed information about the (collection of the) characteristics are described in Table 1

Usability

The majority of the patients (78% inpatients and 83% outpatients) indicated that the usability of the PHR for medication reconciliation was good or even better (Table 3).

Table 3 Usability of a personal health record classified according to the adjective rating scale of the System Usability Scale. The data were collected for patients with an outpatient visit (at the rheumatology ward) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards)

The mean SUS-score of all in- and outpatients was 63 (SD: 14) and 64 (SD: 14) respectively. In- and outpatients most often (strongly) agreed with the item ‘I did not need the support of a technical person to be able to use the PHR’ (72 and 78% respectively) followed by ‘I did not need to learn a lot of things before I could get going with this system’ (61 and 68% respectively). Patients most often (strongly) disagreed with the item ‘most patients would learn to use this system very quickly’ (19 and 17% respectively) followed by ‘I think that I would like to use this system frequently’ (18 and 15% respectively) (Fig. 1). Patients noted in their remarks that the PHR was especially difficult to use for patients with low literacy and the elderly.

Fig. 1
figure 1

Scores of the items of the usability questionnaire of the personal health record. The data were collected for patients with an outpatient visit (at the rheumatology ward, N = 78) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards, N = 177)

Association between patient-, clinical-, hospital-, and ICT-related factors and the usability of a personal health record used for medication reconciliation

Inpatient clinic

In the univariate analyse, four factors were significantly (P < .05) associated with a good usability. These concerned experience with digital devices, patients’ knowledge about the indication(s) of their drug(s), number of different prescribers and the device used to log in to the PHR (Table 4). In the adjusted analyses, there was no significant association between usability and patient-, clinical-, hospital-, and ICT-related factors (Table 4).

Table 4 Patient-, clinical-, hospital-, and ICT-related factors associated with a good usability of a PHR. The data were collected for patients with an outpatient visit (at the rheumatology ward, N = 78) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards, N = 177)

Outpatient clinic

In the univariate analyse, experience with digital devices and hours a week of private internet use were significantly (P < .05) associated with a good usability (Table 4). In the adjusted analyses, there was no significant association between usability and patient-, clinical-, hospital-, and ICT-related factors (Table 4).

Perceived usefulness

The mean score of the perceived usefulness of all in- and outpatients was 53 (SD: 18) and 58 (SD: 18) respectively. The majority of the patients (57% inpatients and 67% outpatients) indicated that the perceived usefulness of the PHR for medication reconciliation was good or even better (Table 5). Most in- and outpatients (76 and 78%, respectively) agreed that the PHR yielded at least one benefit (out of seven) with regard to their visit to the physician (Fig. 2). In- and outpatients most often (strongly) agreed with the item ‘necessary for the visit with the specialist’ (54 and 63%, respectively) followed by ‘increased preparation for the visit’ (49 and 58%, respectively) and ‘gives more control over medication data’ (50 and 56%, respectively).

Table 5 Perceived usefulness of a personal health record classified according to the adjective rating scale of the System Usability Scale. The data were collected for patients with an outpatient visit (at the rheumatology ward) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards)
Fig. 2
figure 2

Scores of the items of the perceived usefulness questionnaire of the personal health record. The data were collected for patients with an outpatient visit (at the rheumatology ward, N = 78) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards, N = 177)

Association between patient-, clinical-, hospital-, and ICT-related factors and the perceived usefulness of a personal health record used for medication reconciliation

Inpatient clinic

In the univariate analyse, the number of outpatient visits to the specialist in the last 12 months and the number of hospital admissions at the admitted department in the last 12 months were significantly (P < .05) associated with perceived usefulness (SUS ≤ 39.3 (Q1) versus SUS ≥ 67.8 (Q4)). In the adjusted analyses, there was no significant association between perceived usefulness and patient-, clinical-, hospital-, and ICT-related factors (Table 6).

Table 6 Patient-, clinical-, hospital-, and ICT-related factors associated with the perceived usefulness of a personal health record. The data were collected for patients with an outpatient visit (at the rheumatology ward, N = 78) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards, N = 177)

Outpatient clinic

When outpatients with a perceived usefulness score of ≤ 50.0 (Q1) were compared with patients with a score of ≥ 67.8 (Q4), we observed that outpatients who also used the PHR for other drug related purposes, reported more often a higher perceived usefulness (adjusted odds ratio 20.0; 95% confidence interval 2.36–170) (Table 6).

Perceived usefulness of the personal health record compared to the gold standard, medication reconciliation by a healthcare professional

When the PHR was compared to the gold standard, medication reconciliation performed by a healthcare professional, 48% of the inpatients and 47% of the outpatients preferred the PHR above medication reconciliation. They (strongly) agreed that the PHR was faster (47% inpatients and 45% outpatients) and easier (49% inpatients and 46% outpatients) to use compared to medication reconciliation performed by a healthcare professional (Fig. 3).

Fig. 3
figure 3

Utility of an online personal health record (PHR) compared to the gold standard. The data were collected for patients with an outpatient visit (at the rheumatology ward, N = 78) or a planned admission in the hospital (at the cardiology, neurology, internal medicine or pulmonary wards, N = 177). The gold standard was medication reconciliation (MR) performed by a healthcare professional

Discussion

In this study, we examined usability and perceived usefulness of an online PHR used for medication reconciliation. Our results show that the usability and perceived usefulness of the PHR was good. There were no patient-, clinical-, hospital-, and ICT-related factors significantly associated with usability.

This study is the first multicenter study assessing usability of a PHR used for detecting medication discrepancies. In two other monocenter studies performed in European hospitals the usability of a PHR for the identification of medication discrepancies has also been studied with the validated SUS [17, 18]. Like in our research, the mean SUS-score (of 68) observed by Webering Buning et al. was classified to the adjective rating ‘good’. Meanwhile, the median SUS-score (of 75) observed by Marien et al. was classified to the adjective rating ‘excellent’ [17, 18]. Compared to our research, Marien et al. used another PHR, included patients who used at least five drugs and the majority of the patients (63%) had a high education level (university of applied sciences or university). Our more diverse study sample may have resulted in a higher external validity and potentially resulted in the inclusion of patients who were less able to use a PHR causing a lower overall SUS-score.

In our research, usability and perceived usefulness were both classified according to the adjective rating scale of Bangor et al. [25]. We observed that most patients classified the usability and perceived usefulness as good. If we look closer to the classification categories, we realized that a score of at least 51 out of 100 points was categorized as good. However, it should be questioned if this actually reflects a good usability and perceived usefulness as just half of the points were scored. As we observed that the majority (83%) of the patients classified the perceived usefulness as poor, ok or good, we assume that perceived usefulness of PHRs used for medication reconciliation should be classified as fair to good instead of good. In our population, only two inpatients and none of the outpatients classified the usability of the PHR as ‘worst imaginable’ or ‘awful’. This skewed distribution of scores towards good usability is potentially related with the fact that patients become more experienced in using digital devices [30, 31]. Besides that, the PHR used in our research is especially designed for patients, and the number of data entries by the patient is limited, increasing the usability of the system.

According to the technology acceptance model, patients’ perceived usefulness and perceived ease of use are the most important factors for adoption of technology devices [16]. If we focus on the items of the perceived usefulness questionnaire which were most relevant for medication reconciliation (including necessary for the visit, increased preparation for the visit and increased time to discuss during the visit), our agreement range (defined as the minimum and maximum proportion of patients who (strongly) agreed with an item of the questionnaire) was higher (39–63%) compared to the agreement range (17–63%) when all items of the perceived usefulness questionnaire were taken into account. This suggested that for medication reconciliation purposes the perceived usefulness of the PHR was acceptable.

In the study of Marien et al. the perceived usefulness of a PHR was higher compared to our results (agreement range of 80–100% versus 17–63%). In the study of Marien, prior to the use of the PHR, patients were educated how to use the PHR and also encouraged to use the system, which was not done in our study [18]. Information provision and support by a healthcare professional are related with a higher intention to use a PHR [32, 33]. So, in order to increase the ease of use and the perceived usefulness by patients, it is recommended that hospitals inform patients about how to use a PHR and about the usefulness of using a PHR.

The difference in perceived usefulness between Marien et al. and our study may also be related with the type of questions: Marien et al. primarily focused on time and communication benefits of the created medication list and we focused on the perceived usefulness in relation to patient’s drug use and the benefits of the current drug list during the hospital visit. Besides that, PHR users who had a negative opinion about the PHR and/or had a suggestion to improve the PHR were potentially more motivated to participate in this study which may have caused inclusion bias. Furthermore, Marien et al. included patients with pulmonary transplants who have potentially a higher perceived usefulness of their (live saving) drugs compared to our study sample [34].

The strength of our study is that we compared usability and perceived usefulness between patient-, clinical-, hospital-, and ICT-related factors at both the in- and outpatient setting. This resulted in a better knowledge about which patients have difficulties in using the PHR and which patients have negative thoughts about the usefulness of the PHR. Until now, only one study investigated the association between patient characteristics and usability. Marien et al. observed an association between the number of logins to the PHR (> 4 times during the study) and usability [18]. As our patient sample had less logins to the PHR, we did not observe this effect. Besides that, we observed that only 9% of the outpatients had a follow-up appointment. Because of this, it is unknown how repeated use influences the usability and perceived usefulness of the PHR. Meanwhile, in the univariate analyse of the in- and outpatient setting, we observed that experience with digital devices was significantly associated with a good usability. Irizarry et al. also concluded that computer skills are an important factor that contribute to successful patient engagement via a PHR [15]. To increase the acceptance and use of the PHR in practice, hospitals should offer education about how to use an online PHR for patients with less experience with digital devices.

We observed a high number of drug changes in the last 12 months in our population, which argues for the necessity of a continuous use of PHRs by patients. However, the number of patients that used the PHR more than once in the last 12 months was small, although the majority of the patients indicated that PHRs are easy to use and useful. In other words, the full benefits of a PHR are not reached and therefore patients should be stimulated to continuously update their drug list in the PHR. This can for example be reached by educating patients about usability and perceived usefulness of PHRs and/or by reminding patients to update their drug list in the PHR. As electronic reminders are positively associated with patient adherence to medication [35], we assume that patients will become more empowered when they are (continuously) reminded to update their own drug list, especially when the number of drug changes is high. Therefore, we recommend that future studies should focus on usability of the combination of electronic reminders with PHRs.

Unlike the usability, there is no standardized method to score perceived usefulness. Because of this, we scored the perceived usefulness in a similar way as the SUS. As there was also no standardized dichotomization for analysing the association between perceived usefulness and patient-, clinical-, hospital-, and ICT-related factors, we compared the first and fourth quartile of the perceived usefulness score. To confirm the robustness of our results, we also performed additional analyses. These analyses did not show any consistent associations between the variables and the perceived usefulness. Because of this, we assume that there is no significant association between the perceived usefulness and patient-, clinical-, hospital-, and ICT-related factors at both the in- and outpatient setting.

Finally, we observed that around half of the patients (strongly) agreed that the PHR was faster and easier to use compared to medication reconciliation performed by a healthcare professional. Therefore, further research should examine the time-advantages and the accuracy benefits of using PHRs for medication reconciliation in usual care.

Our study had several limitations. Firstly, the Dutch version of the SUS questionnaire used in our study has not been validated. Nevertheless, the Dutch version has been widely used and have similar internal reliability to the original English version [18, 24, 36,37,38,39,40].

Secondly, there is a chance of non-response bias. More than two-third of the patients refused to participate in this study which may have affected usability and perceived usefulness of the PHR.

Thirdly, there is a chance of recall bias, because the time between using the PHR and performing the questionnaire was about a month. As Boyd et al. demonstrated that there is no significant change of the user’s recollection of usability of digital product over a period of 6 months [41], we assume that the risk of recall bias is minimal.

Fourthly, there may have been errors in documenting the types of comorbidities based on prescribed medications. However, this should not have led to major errors in the count of comorbidities, which was the covariate used in analyses.

Fifthly, the results of our study with regard to the inpatients may not be generalizable to other countries where the numbers of planned hospitalizations for medication conditions are small.

Sixthly, the questionnaire was only sent to patients who completed the medication check in the PHR. Patients who did or could not use the PHR or patients who quitted halfway through the process due to difficulties were not participating which potentially resulted in an overestimated usability and perceived usefulness of the PHR. As the acceptance rate of the PHR is low (about 44%) [14], the adoption of the PHR should be improved by increasing the information provision (about the usefulness of the PHR and how to use it) to patients and involve patients by improving the design of the PHR. Further research should focus on the barriers and facilitators of a PHR among PHR users and non-users. Knowledge of these barriers and facilitators may contribute to a better targeted approach to improve the adoption of the PHR. Improvement of the PHR may have the potential to positively affect the quality of medication discrepancies detection with the PHR, thereby increasing the success of the PHR.

Conclusions

In conclusion, the majority of the patients indicated that the PHR used for medication reconciliation was useful and easy to use, but there is still room for improvement to increase the adoption of PHRs for medication reconciliation. To further increase acceptance and use of a PHR, further research should explore the barriers and facilitators of the population who rated the usability and perceived usefulness low.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to privacy restrictions as the databases contain information that could compromise the privacy of research participants. However, the datasets are available from the corresponding author on reasonable request.

Abbreviations

IQR:

Interquartile range

PHR:

Personal health record

SD:

Standard deviation

SUS:

System Usability Scale

Q1:

First quartile

Q4:

Fourth quartile

References

  1. The Joint Commission. National patient safety goals effective; 2017. p. 1–17.

    Google Scholar 

  2. Tam VC, Knowles SR, Cornish PL, Fine N, Marchesano R, Etchells EE. Frequency, type and clinical importance of medication history errors at admission to hospital: a systematic review. CMAJ. 2005;173(5):510–5. https://doi.org/10.1503/cmaj.045311.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18(8):646–51. https://doi.org/10.1046/j.1525-1497.2003.20722.x.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Division of Pharmacoepidemiology & Pharmacotherapy UI for PS. Hospital admissions related to medication (HARM): een prospectief, multicenter onderzoek naar geneesmiddel gerelateerde ziekenhuisopnames; 2006. p. 1–80.

  5. Alqenae FA, Steinke D, Keers RN. Prevalence and nature of medication errors and medication-related harm following discharge from hospital to community settings: a systematic review. Drug Saf. 2020;43(6):517–37. https://doi.org/10.1007/s40264-020-00918-3.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Nachar C, Lamy O, Sadeghipour F, Garnier A, Voirol P. Medication reconciliation in a Swiss hospital: methods, benefits and pitfalls. Eur J Hosp Pharm. 2019;26(3):129–34 https://www.ncbi.nlm.nih.gov/pubmed/31428319.

    Article  Google Scholar 

  7. Giannini O, Rizza N, Pironi M, et al. Prevalence, clinical relevance and predictive factors of medication discrepancies revealed by medication reconciliation at hospital admission: Prospective study in a Swiss internal medicine ward. BMJ Open. 2019;9:1–8. https://doi.org/10.1136/bmjopen-2018-026259.

    Article  Google Scholar 

  8. Bonaudo M, Martorana M, Dimonte V, et al. Medication discrepancies across multiple care transitions: A retrospective longitudinal cohort study in Italy. PLoS One. 2018;13(1):1–13. https://doi.org/10.1371/journal.pone.0191028.

    CAS  Article  Google Scholar 

  9. Stewart AL, Lynch KJ. Identifying discrepancies in electronic medical records through pharmacist medication reconciliation. J Am Pharm Assoc. 2012;52:59–66. https://doi.org/10.1331/JAPhA.2012.10123.

    Article  Google Scholar 

  10. van der Gaag S, Janssen MJA, Wessemius H, Siegert CEH, Karapinar-Çarkit F. An evaluation of medication reconciliation at an outpatient Internal Medicines clinic. Eur J Intern Med. 2017;44:e32–4. https://doi.org/10.1016/j.ejim.2017.07.015.

    Article  PubMed  Google Scholar 

  11. Milone AS, Philbrick AM, Harris IM, Fallert CJ. Medication reconciliation by clinical pharmacists in an outpatient family medicine clinic. J Am Pharm Assoc. 2014;54(2):181–7. https://doi.org/10.1331/JAPhA.2014.12230.

    Article  Google Scholar 

  12. World Health Organization. The high 5s project: standard operating protocol: assuring medication accuracy at transitions in care: medication reconciliation; 2014.

    Google Scholar 

  13. Institute for Healthcare Improvement. How to guide: prevent adverse drug events by implementing medication reconciliation. http://app.ihi.org/LMS/Content/2cf9e482-3e91-4218-afe3-22f77b5025bc/Upload/HowtoGuidePreventADEs.pdf. Accessed 25 Feb 2019.

  14. van der Nat DJ, Taks M, Huiskes VJB, van den Bemt BJF, van Onzenoort HAW. A comparison between medication reconciliation by a pharmacy technician and the use of an online personal health record by patients for identifying medication discrepancies in patients’ drug lists prior to elective admissions. Int J Med Inform. 2021;147:104370. https://doi.org/10.1016/j.ijmedinf.2020.104370.

    Article  PubMed  Google Scholar 

  15. Irizarry T, Dabbs AD, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res. 2015;17(6):148.

  16. Davis FD. A technology acceptance model for empirically testing new end-user information systems; 1985.

    Google Scholar 

  17. Werumeus Buning A, Klopotowska JE, Duyvendak M, Engelen LJLPG, Arts J. Patient empowerment through provision of a mobile application for medication reconciliation: a proof of concept study. BMJ Innov. 2016;0:1–6. https://doi.org/10.1136/bmjinnov-2015-000110.

    Article  Google Scholar 

  18. Marien S, Legrand D, Ospina G, et al. A web application to involve patients in the medication reconciliation process: a user-centered usability and usefulness study. J Am Med Inform Assoc. 2018;25(11):1488–500. https://doi.org/10.1093/jamia/ocy107.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Russ AL, Jahn MA, Patel H, et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users’ detection of medication discrepancies. J Biomed Inform. 2018;82:178–86. https://doi.org/10.1016/j.jbi.2018.05.002.

    Article  PubMed  Google Scholar 

  20. Abd-alrazaq AA, Bewick BM, Farragher T, et al. Factors that affect the use of electronic personal health records among patients: A systematic review. Int J Med Inform. 2019;126:164–75. https://doi.org/10.1016/j.ijmedinf.2019.03.014.

    Article  PubMed  Google Scholar 

  21. Castor. Castor. https://www.castoredc.com/. Accessed 10 April 10 2019.

  22. Vereniging van Zorgaanbieders voor Zorgcommunicatie. Over het LSP. https://www.vzvz.nl/over-het-lsp. Accessed 20 May 2019.

  23. Brooke J. SUS-A quick and dirty usability scale; 1986. p. 1–7.

    Google Scholar 

  24. Jansen-Kosterink S, Huis in’t Veld M. Differences in Use of a Exercise-Based Tele-Rehabilitation Service Delivered as Substitute of or Supplement to Conventional Care. In: Proceedings of the Sixth International Symposium on e-Health Services and Technologies and the Third International Conference on Green IT Solutions; 2012. https://www.scitepress.org/Papers/2012/44739/pdf/index.html. Accessed 19 Jan 2021.

  25. Bangor A, Kortum P, Miller J. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J Usability Stud. 2009;4(3):114–23.

    Google Scholar 

  26. Oh SM, Kim JY, Han S, et al. Development and usability of a virtual reality-based filler injection training system. Aesthet Plast Surg. 2020;44(5):1833–42. https://doi.org/10.1007/s00266-020-01872-2.

    Article  Google Scholar 

  27. Georgsson M, Staggers N. Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. J Am Med Inform Assoc. 2016;23(1):5–11. https://doi.org/10.1093/jamia/ocv099.

    Article  PubMed  Google Scholar 

  28. Food and Drug Administration. OTC Drug. https://www.accessdata.fda.gov/scripts/cder/training/OTC/topic2/topic2/da_01_02_0080.htm. Accessed 3 Jul 2020.

  29. WHO Collaborating Centre for the Family of International Classifications (FIC). ICD-10: International Classification of Diseases and Related Health Problems. https://www.whofic.nl/familie-van-internationale-classificaties/referentie-classificaties/icd-10. Accessed 22 Dec 2020.

  30. Eurostat. Individuals - frequency of computer use. http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=isoc_ci_cfp_fu&lang=en Accessed 30 April 2021.

  31. Eurostat. Individuals - frequency of internet use. https://ec.europa.eu/eurostat/databrowser/view/isoc_ci_ifp_fu/default/table?lang=en Accessed 30 April 2021.

  32. Agarwal R, Anderson C, Zarate J, Ward C. If we offer it, will they accept? Factors affecting patient use intentions of personal health records and secure messaging. J Med Internet Res. 2013;15(2):e43. https://doi.org/10.2196/jmir.2243.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Zhao JY, Song B, Anand E, et al. Barriers, facilitators, and solutions to optimal patient portal and personal health record use: a systematic review of the literature. AMIA Annu Symp Proc. 2017;2018(16):1913–22.

    Google Scholar 

  34. Drangsholt SH, Cappelen UW, von der Lippe N, Høieggen A, Os I, Brekke FB. Beliefs about medicines in dialysis patients and after renal transplantation. Hemodial Int. 2019;23(1):117–25. https://doi.org/10.1111/hdi.12717.

    Article  PubMed  Google Scholar 

  35. Tao D, Xie L, Wang T, et al. A meta-analysis of the use of electronic reminders for patient adherence to medication in chronic disease care. J Telemed Telecare. 2015;21(1):3–13. https://doi.org/10.1177/1357633X14541041.

    Article  PubMed  Google Scholar 

  36. Brooke J. SUS: a retrospective. J Usability Stud. 2013;8(2):29–40.

    Google Scholar 

  37. Zijp TR, Touw DJ, van Boven JFM. User acceptability and technical robustness evaluation of a novel smart pill bottle prototype designed to support medication adherence. Patient Prefer Adherence. 2020;14:625–34. https://doi.org/10.2147/PPA.S240443.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Jonker LT, Plas M, de Bock GH, Buskens E, van Leeuwen BL, Lahr MMH. Remote home monitoring of older surgical cancer patients: perspective on study implementation and feasibility. Ann Surg Oncol. 2021;28(1):67–78. https://doi.org/10.1245/s10434-020-08705-1.

    Article  PubMed  Google Scholar 

  39. van Dijsseldonk RB, van Nes IJW, Geurts ACH, Keijsers NLW. Exoskeleton home and community use in people with complete spinal cord injury. Sci Rep. 2020;10(1). https://doi.org/10.1038/s41598-020-72397-6.

  40. Mol M, Van Schaik A, Dozeman E, et al. Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis. BMC Psychiatry. 2020;20(1). https://doi.org/10.1186/s12888-020-02627-.

  41. Boyd K, Bond R, Bournemouth AV, et al. How users judge a user experience at different time points: is there evidence for memory decay, recall bias or temporal bias? Interacing Comput. 2019;31(2):221–30. https://doi.org/10.1093/iwc/iwz019.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

BP performed the data collection at the Sint-Maartenskliniek and DN performed the data collection at the Amphia Hospital. DN analysed and interpreted the data, and was a major contributor in writing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hein A. W. van Onzenoort.

Ethics declarations

Ethics approval and consent to participate

The methods were carried out in accordance with relevant guidelines and regulations like the Declaration of Helsinki and the European General Data Protection Regulation. The study performed at the Amphia Hospital (N2019–0212) was approved by the Medical Ethics Committee of Utrecht, the Netherlands, and the study performed at the Sint-Maartenskliniek (2018–4873) was approved by the Commission on Research Involving Human Subjects of Arnhem-Nijmegen, the Netherlands. All patients provided written informed consent prior to starting the questionnaire.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

van der Nat, D.J., Huiskes, V.J.B., Taks, M. et al. Usability and perceived usefulness of patient-centered medication reconciliation using a personalized health record: a multicenter cross-sectional study. BMC Health Serv Res 22, 776 (2022). https://doi.org/10.1186/s12913-022-07967-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-07967-7

Keywords

  • Medication reconciliation
  • Perceived usefulness
  • Personal health record
  • Technology acceptance model
  • Usability