Skip to main content

Implementing recommendations for inpatient healthcare provider encouragement of cardiac rehabilitation participation: development and evaluation of an online course



A policy statement recommending that healthcare providers (HCPs) encourage cardiac patients to enroll in cardiac rehabilitation (CR) was recently endorsed by 23 medical societies. This study describes the development and evaluation of a guideline implementation tool.


A stepwise multiple-method study was conducted. Inpatient cardiac HCPs were recruited between September 2018–May 2019 from two academic hospitals in Toronto, Canada. First, HCPs were observed during discharge discussions with patients to determine needs. Results informed selection and development of the tool by the multidisciplinary planning committee, namely an online course. It was pilot-tested with target users through a think-aloud protocol with subsequent semi-structured interviews, until saturation was achieved. Results informed refinement before launching the course. Finally, to evaluate impact, HCPs were surveyed to test whether knowledge, attitudes, self-efficacy and practice changed from before watching the course, through to post-course and 1 month later.


Seven nurses (71.4% female) were observed. Five (62.5%) initiated dialogue about CR, which lasted on average 12 s. Patients asked questions, which HCPs could not answer. The planning committee decided to develop an online course to reach inpatient cardiac HCPs, to educate them on how to encourage patients to participate in CR at the bedside. The course was pilot-tested with 5 HCPs (60.0% nurse-practitioners). Revisions included providing evidence of CR benefits and clarification regarding pre-CR stress test screening. HCPs did not remember the key points to convey, so a downloadable handout was embedded for the point-of-care. The course was launched, with the surveys. Twenty-four HCPs (83.3% nurses) completed the pre-course survey, 21 (87.5%) post, and 9 (37.5%) 1 month later. CR knowledge increased from pre (mean = 2.71 ± 0.95/5) to post-course (mean = 4.10 ± 0.62; p ≤ .001), as did self-efficacy in answering patient CR questions (mean = 2.29 ± 0.95/5 pre and 3.67 ± 0.58 post; p ≤ 0.001). CR attitudes were significantly more positive post-course (mean = 4.13 ± 0.95/5 pre and 4.62 ± 0.59 post; p ≤ 0.05). With regard to practice, 8 (33.3%) HCPs reported providing patients CR handouts pre-course at least sometimes or more, and 6 (66.7%) 1 month later.


Preliminary results support broader dissemination, and hence a genericized version has been created ( Continuing education credits have been secured.

Peer Review reports


Cardiovascular diseases (CVDs) are among the leading burdens of disease and disability worldwide [1, 2]. In 2015, there were 422.7 million CVD cases globally [3], and these patients are at high risk of recurrent cardiac events and death [2]. Thus, secondary prevention is needed [4].

Cardiac Rehabilitation (CR) is a proven, cost-effective, outpatient model of care comprised of structured exercise training, patient education and counselling, as well as risk factor management [5, 6]. The benefits of CR include 20% reductions in morbidity and CV mortality [7]. Despite the benefits, CR utilization is low [8,9,10,11,12]. One of the main reasons is lack of referral and encouragement by healthcare providers (HCPs) [13].

The recent update of the Cochrane Collaboration review on interventions to promote CR utilization undertaken by our group [14] identified effective strategies. Findings included that CR enrolment is significantly greater when an intervention is delivered by a healthcare provider (HCP), face-to-face. This could be undertaken most feasibly and affordably through communication at the bedside with CV inpatients prior to hospital discharge. A position statement to forward recommendations based on the findings was subsequently developed, and endorsed by 23 medical societies [15, 16].

However, the development of a guideline or position statement is insufficient to change clinical practice and hence achieve greater patient utilization; therefore implementation tools are needed [17]. Indeed, a 2016 Cochrane review showed that implementation tools developed and disseminated with guidelines positively influence clinician behavior and patient outcomes [18]. Accordingly, implementation tools are recommended in standards for guideline development [19,20,21].

We undertook an environmental scan and consulted experts globally through the International Council of Cardiovascular Prevention and Rehabilitation (ICCPR), and could not identify tools to support implementation of the recommendations (what is available is shown here: Thus, a needs assessment was undertaken to discern the best type of tool(s) to promote patient-provider bedside discussions regarding CR; results were used to develop and then test a guideline implementation tool. The objectives of this paper were to describe the needs assessment, implementation tool development process, and evaluation of its’ efficacy, with regard to learner knowledge, attitudes, self-efficacy, and practice. This tool could improve awareness and discussion about, as well as utilization of, CR, leading to greater secondary prevention of CVD.


Team composition and stakeholder engagement

The team / planning committee was comprised of the co-chairs of the policy statement on interventions to increase CR utilization (CSAP, SLG) [15, 16], as well as the methodologist with expertise in guideline implementation (AG). Clinicians who would be implementing the recommendations also served (NS, physician; CSAP, physiotherapist; AL, nurse-practitioner). A final team member served who has expertise in the various evaluative methods being applied (FA). We also solicited input from patient partners regarding whether the points to be conveyed at the bedside resonated with their information needs and preferences, was comprehensive, to ascertain if there were any omissions, and to ensure patient-centeredness. All 23 position statement-endorsing associations were informed about the plan to develop guideline implementation tool(s), with a request for eventual input and dissemination facilitation.

Design and procedure

This was a multiple-method study, using a step-wise approach. Guideline implementation tool development process best practices were followed [22, 23], to consider needs for and barriers to implementation, determine the type of tool(s), develop it/them, evaluate and disseminate. The process is summarized in Fig. 1.

Fig. 1
figure 1

Process for development, evaluation and dissemination of the guideline implementation tool for promoting patient utilization of cardiac rehabilitation*. *steps based on cite [22,23,24,25,26]. §outcomes selected based on cite [27]. (Kirkpatrick’s levels for training evaluation) [27]

HCPs treating inpatients indicated for CR were recruited between September 2018 and May 2019 from two hospitals of an Academic Health Sciences Centre in Toronto, Canada (University Health Network; UHN), for the needs assessment (September–November 2018; structured observation), pilot test (November 2018–January 2019; interviews) and then evaluation (January–March 2019; prospective design). There are different CR referral processes on the various cardiac units at the hospitals; for some it was an electronic systematic referral, on another referral is included on the paper-based discharge order set, and on others there is no systematic process in place and hence physician referral is ad-hoc. Note that CR services are covered by government healthcare sources in Ontario.

Needs assessment

First, literature regarding patient-provider discussions about CR was reviewed, including barriers [28, 29]. Experts on patient-provider discussions regarding CR were consulted.

Second, cardiac HCPs were observed (structured) [30] during inpatient-provider discussions regarding discharge, to learn what information was being conveyed regarding CR (and not), and what questions inpatients often have about CR, to ensure providers have the answers in the future (Additional file 1: Appendix 1). To decrease the risk of reactivity, HCPs were informed we were interested in patient-provider communication regarding discharge instructions (CR was not mentioned until after the observation). The observer stood against a wall at some distance from the patient and HCP with a clipboard, and did not speak during the interaction.

After the observation, the observer debriefed with the HCP to get further detail regarding what information they felt they were lacking with regard to CR, and how they can be supported to discuss CR with patients. All observations and a discussion summary were recorded in writing immediately.

Implementation tool type and development

A review of guideline implementation tool types [31, 32] was considered by the team. Results of the literature review and structured observation were discussed with the team, and expert opinion was also considered to decide on the type of tool(s). Development ensued in accordance with best practices [22]. Input from patient partners was sought, and incorporated.

Implementation tool pilot test: think aloud protocol and semi-structured interview

Once developed and hosted on UHN’s eLearning centre, inpatient cardiac HCPs were recruited to view the online course (including the pre and post-course survey), in accordance with Level 1 of Kirkpatrick’s model (reaction) [27]. It was pilot-tested with the intended audience using a think aloud protocol (TAP) with subsequent semi-structured interviews (i.e., retrospective questioning for triangulation), until saturation was achieved. This was undertaken in person at UHN. The purpose was to determine whether the drafted online course was applicable to target HCPs / realistic, met their information needs, was an acceptable length, to get input on graphics / visuals, ways to promote implementation of the ideas at the bedside, and how it could be revised to better meet their needs. Results informed refinement before launching the course.

The instructions for the TAP are shown in Additional file 1: Appendix 2. The encounters were audio-recorded, with permission. They were transcribed verbatim, except to preserve anonymity. The senior investigator (SLG) attended the first few pilot tests for training purposes, and to finalize the drafted TAP protocol and semi-structured interview guide.

Implementation tool finalization and soft launch

Results of the pilot-test were used to finalize the course. It was launched for all users at UHN.

Formative evaluation: survey

HCPs were surveyed to test whether knowledge, attitudes, confidence/self-efficacy and practice / behaviour (e.g., if HCPs provided materials like pamphlet or handouts to patients about CR to take home) changed following completion of the course. These outcomes were chosen based on Kirkpatrick’s model of training evaluation (level 2, learning) [27]. The questionnaire was administered online using Google forms: (1) before viewing the course; (2) immediately after viewing the course; and (3) 1 month later, via email. HCPs were emailed on several occasions with reminders to complete the 1-month post-course survey if they had not done so, to optimize response rate.

Participants: recruitment and sample size

For each element of the project, participants consisted of acute cardiac care providers (e.g., nurses / nurse-practitioners, physicians, physiotherapists) on wards treating patients indicated for CR at UHN (e.g., short stay unit for percutaneous coronary intervention, cardiovascular surgery unit, general cardiology ward). There were no exclusion criteria.

Structured observation

To recruit for this initial needs assesment, all HCPs in the cardiology program were contacted through email by the clinical director, with a request to be observed during patient interactions regarding discharge (CR was not mentioned). Unit nurse managers also identified some staff to approach. Attempt was made to observe HCPs on several cardiac wards. The plan was to observe interactions until no novel observations were made.

Observations with patients who were eligible for CR (see indications and exclusions in policy statement) [15, 16] and who were soon to be discharged were undertaken. On the day of observation, HCPs approached patients in their circle of care without the observer present, to ask for their voluntary consent that an observer be present during the discharge discussion. Patients were informed that the observer was recording information about the HCP provision of discharge information, and only any questions or issues the patient raised would be notated (the rest of the observation pertained to the HCPs), and that their identity would remain anonymous. Willing patients provided verbal informed consent.

Think aloud protocol and semi-structured interview

After the tool was developed, eligible HCPs were contacted through email by the senior investigator (SLG), with a request to preview the drafted online course and provide input. Recruitment was targeted to solicit feedback from several relevant disciplines, with emails sent to physicians, nurses and physiotherapists. Sample size was determined by saturation; they continued until no novel input was received.


After the course revision and launch, eligible HCPs were contacted through email with a request to complete the online course, with the surveys. The emails were sent by the clinical director, and the senior investigator (SLG) later followed-up. The new course was also advertised in the monthly cardiology and cardiovascular surgery email blast. We also attended team meetings on the cardiac wards to promote the course. The clinical director offered a pizza lunch to the cardiac ward with the highest completion rate.


Needs assessment: structured observation

The observer used a checklist (Additional file 1: Appendix 1) to record observations and short descriptions of the interactions. The checklist was developed by CSP and SLG, and pilot-tested in 2 interactions. Some revisions were made. The senior author observed the first few observations and subsequent debriefings, to provide feedback for training purposes. The senior author independently completed the observation checklist, and discrepancies were discussed with CSP. This was repeated until no further discrepancies arose following an observation.

Pilot-test: think aloud protocol and semi-structured interview

The TAP and interview guide are shown in Additional file 1: Appendix 2. The TAP was performed using best practices [33]. The semi-structured interview guide was developed by CSP and SLG, and pilot-tested as outlined above.

Formative tool evaluation: survey

The surveys administered at each point are shown in Additional file 1: Appendix 3. They consisted of multiple choice and true-false questions, as well as items with a 5-point Likert type scale for responding. They assessed the basic characteristics of the HCP (e.g., profession), as well as their CR knowledge (e.g., how familiar HCPs were with what is offered and delivered to patients in CR) and attitudes, self-efficacy in discussing CR with inpatients(e.g., how confident HCPs were addressing barriers patients raised regarding CR attendance), as well as their practices (e.g., giving CR program pamphlets to patients; Table 2).



SPSS version 24.0 was used for quantitative analysis. Elements of the observation coded as present or absent were described using descriptive statistics. Analysis of the qualitative data involved bringing order and structure to the information recorded to inform development of the online course [34].

Think-aloud and interviews

The transcripts of the TAP and subsequent questioning [35] were segmented into sensible chunks or communication units, which were coded, all by the first author [36, 37]. The coding of the TAP speech focused on thoughts reflecting ways in which the course could be improved (researcher inference; literal as much as possible), and of the interviews focused on validating interpretation of the think-aloud utterances, as well as extracting additional suggestions relating to how the course could be improved [38]. Initial coding by CSP was reviewed and discussed with the senior author, who was there for the initial interviews and reviewed the transcripts. Final coding / thematic content analysis was discussed between researchers to determine the course of action for revising the online course.


SPSS version 24.0 was used for analysis. All surveys were included. Descriptive statistics were used to describe the sample, as well as survey responses. Pre- and post-course survey responses were compared using paired t-tests or chi-square analyses as applicable (repeated measures analysis of variance was planned, but the sample size for the survey 1-month post-course was insufficient).


Needs assessment: structured field observation

Seven HCPs (all nurses) were observed (8 interactions); 5 (71.4%) were female. In most interactions, HCPs were rushing, to complete their “tasks”. Family or informal caregivers were present for 5 (62.5%) interactions. Five (71.4%) HCPs knew whether their patient had been referred or was going to be referred.

Five (62.5%) HCPs initiated a dialogue about CR with a patient; however, the dialogue lasted an average of 12 s and lacked detailed information. No patients raised CR. In all interactions, CR was raised after the discharge instructions, at the end of the interaction.

In 1 (20.0%) of these 5 interactions, HCPs explained what CR is (e.g., consists of education and physical exercise), and none of these interactions was the information conveyed all accurate. In no interactions did HCPs explain why the patient was being referred, in 1 (20.0%) interaction the HCP mentioned some of the benefits of CR (i.e., “faster recovery”, “get back on their feet”). In 2 (40.0%) interactions, the HCPs provided strong and explicit positive endorsement of CR, which the observer rated as a mean of 4.5 / 5 (Additional file 1: Appendix 1, item 8). In 1 (20.0%) interactions the HCP explained next steps to enroll (see below).

In only 1 (20.0%) interaction where CR was raised did the HCP invite questions about CR; and in 1 (20.0%) patients raised questions. Some patients asked about when the program would start and whether family members could attend, yet most HCPs did not know the answers. In 2 (40.0%) interactions barriers were raised, and 2 HCPs discussed ways to overcome them (e.g., HCP explained patient would be directed to CR program closest to home); in most cases, barriers were not sufficiently addressed. Overall, there was 2-way discussion about CR in 1 of the 8 encounters, and patients were provided a means to find out more information in 5, however this consisted of a brochure included among other brochures provided to patients at discharge, and the HCPs did not refer patients to it specifically.

Most commonly the HCPs gave the following 2 points when discussing CR: “The cardiac rehab program will call you in 2 weeks, and you will be referred to the program closest to home.” One HCP stated that the patient did not have to attend CR if they did not want to.

In an observation where CR was not initially raised, a senior nurse was training a new hire on how to go through the discharge summary with patients; the nurse lacked information about who should refer the patient to CR and stated: “If a patient asks about CR, just tell them that the family doctor will decide if they need to go, and they will be referred [to a site] close to home.” During the observation the nurse trainee did not mention CR to the patient, and after debriefing with the researcher, the trainee felt compelled to go back and explain about CR to the patient, who asked many questions and seemed interested and likely to attend. In the other observation where it was not raised, the patient had an interpreter because he could not understand the English language; during the observation the HCP failed to mention CR to the patient. When asked the reason during debriefing, the HCP responded: “I just forgot to mention CR.”

During the debriefing after the observations, most staff seemed aware of the importance of discussing CR participation with their patients. Overall, the observations revealed that HCPs are insufficiently discussing CR with their patients, wanted to know about who was eligible, and what were valid reasons patients should not go as well as what was not.

Implementation tool development

Based on the results from the structured observation, for policy statement recommendation implementation support, the team elected to develop training material for HCPs [31]. It was decided to develop an online course given how busy inpatient HCPs are, and that they complete online courses annually as a requirement for continuing professional education. The course was sponsored by the hospital’s CR program, and built by an eLearning and instructional design specialist from UHN in alignment with their best practices (Fig. 1).

The training course was designed to inform inpatient cardiac care providers about: (1) what is cardiac rehab (and provide a corresponding patient handout); (2) the benefits of participation; (3) the importance of, and how to provide a positive endorsement regarding participation to patients; and (4) the importance of letting patients ask questions and discuss any barriers they may have. Input was also gathered from patient partners and other stakeholders (e.g., policy statement-endorsing societies) [15, 16] on the main points to convey. With the patient partners, we considered how to convey risk associated with non-participation when stating CR benefits, and also considered evidence on how best to do this to encourage patient enrolment (e.g., gain frame – 25% less likely to die if go to CR) [39].

Pilot test: think Aloud Protocol & Semi-structured Interviews

The TAP and interviews were conducted with female HCPs (2 nurse managers, 2 cardiology fellows [MD], and 1 nurse-practitioner), and averaged 22 min. Data collected from the TAP and interviews suggest that HCPs were satisfied with the content and length of the course. Themes are shown in Table 1 with examples and corresponding revisions made to the online course.

Table 1 Selected coding from think-aloud protocol and subsequent interviews, with corresponding changes made to course

Revisions included providing evidence (i.e., forest plot and citation) [7] on the benefits of CR, as well as clarifying that pre-CR stress tests are performed under physician supervision and only after patient evaluation for readiness/safety. Additionally, HCPs did not remember the key points to convey to patients, so we developed and embedded a “key points” handout learners can print to use at the point-of-care (Fig. 2).

Fig. 2
figure 2

Point-of-care tool: Key points for patient-provider discussion. CR = cardiac rehabilitation

Formative tool evaluation: knowledge, attitudes, self-efficacy, practice

Twenty-four HCPs (20 registered nurses [83.3%], 1 nurse-practitioner [4.2%], 1 physiotherapy assistant [4.2%], and 2 other HCPs [8.3%]; 23 female [95.8%]; mean age = 36.4 ± 11.6 years) viewed the online course and completed the surveys (retention shown in Table 2).

Table 2 Survey responses by assessment point

When asked pre-course whether their patients were generally referred to CR, 7 (29.2%) HCPs reported that patients are referred most of the time, 14 (58.3%) reported sometimes, and 3 (12.5%) indicated they are not referred. When asked whether they discuss CR with patients, 9 (37.5%) HCPs reported most of the time, 7 (29.2%) sometimes, and 8 (33.3%) never.

Survey responses are displayed in Table 2 by assessment point. As shown, viewing the online course resulted in significant increases in knowledge of what CR entails, having sufficient information to comprehensively discuss CR with patients, self-efficacy in addressing patient questions about CR and barriers, and attitudes toward discussing CR with patients. In terms of knowledge regarding types of patients that are eligible, pre-course HCPs were accurate for a mean of 3.33 ± 0.87 of the 5 patient profiles, post-course HCPs were accurate for a mean of 4.33 ± 0.86 of the 5 profiles (paired t = 3.90 p = .001), and 1 month later for 3.78 ± 0.66. Differences in practice could not be tested. Overall, for all items that could be tested, significant improvements were observed following viewing the course.


Guidelines and Position Statements can play an important role in health policy formation and health care delivery [40]. However, the development of guidelines with recommendations is insufficient to change practice; the recommendations must be implemented. A multitude of determinants influence if recommendations are implemented, at the guideline, clinician, patient, organization and healthcare system levels [23]. To our knowledge, there are no other implementation tools that are evidence-based which address how to increase CR utilization.

After an extensive literature review and needs assessment, a novel guideline implementation tool was developed to promote patient-provider bedside discussions regarding CR. The online course was pilot-tested with acute cardiac care providers. Subsequent evaluation revealed that viewing the course resulted increased CR knowledge, self-efficacy regarding discussing CR with patients, and more positive CR attitudes among HCPs. A point-of-care tool was also developed to support HCPs in having a fulsome discussion with patients at the bedside.

With these positive results, we went on to the implementation phase of the process (Fig. 1). The online course has been genericized for a broader audience of inpatient cardiac care providers globally. This involved primarily removing institution-specific referral and CR program information. The course is available here: (there are also French and Portuguese translations available; Chinese and Spanish translations are currently underway). We have applied for and secured continuing education credits for course completion (

Thus, we are now seeking to inform our target audience of the availability of the online course, to promote wide learning. We are submitting the policy statement to guideline clearinghouses (e.g.,;;, and including this as an implementation tool. We have asked the 23 position statement-endorsing societies and 39 ICCPR-member societies to disseminate the course to their members; they are doing this via email, websites and social media.

Directions for future research

As per the final step of the process in Fig. 1, the genericized course require evaluation in a broader, larger sample. The evaluation should include investigation of change in practice (i.e., occurrence of discussions; i.e., Level 3 of Kirkpatrick’s model- behavior) [27], the quality of CR discussions (e.g., structured observation pre and post-course viewing), and impact on CR utilization (i.e., Level 4 of Kirkpatrick’s model - results) [27]. Many CR associations have utilization quality indicators which could be used to quantify impact [41]. Impact on HCP practices and quality of CR discussions over the longer-term post-course also should be assessed; there was insufficient data in the current study to even determine effect 1-month post-course, but what data are available suggest there is some decay over time without reinforcement.

As a requirement for being an accredited learning activity for continuing education credits, the genericized course does have a pre and post-course knowledge survey as well as evaluation / feedback on the course, which will be collated in future. We are also capturing country of origin of learners, and monitoring usage / uptake.

Other important avenues for future research include investigating inpatient CR information needs and preferences, such that a more standardized discussion could be specified for HCPs. This should then be evaluated, in terms of acceptability by patients, satisfaction, and ultimate CR utilization. The point-of-care tool could be revised based on patient input, and with evidence of impact on CR use. There truly is little evidence or guidance regarding the content of CR discussions, and based on our observations it seems some of the discussions that do occur may dissuade patients from attending or reduce their likelihood of enrolling. A question prompt tool for patients may also be helpful [42].

In the United States, the Agency for Healthcare Research and Quality (AHRQ) recently awarded a contract for implementation of systematic referral with a “liaison” discussion at the bedside in 100 hospitals ( Participating hospitals will be supported in a learning community, and will work on an aspect of implementation each month over the course of a year. This reinforcement may ensure sustained implementation (versus the decay we seem to have observed by 1-month post-course viewing). The online course may be useful in educating HCPs regarding bedside CR discussions. To be successful, it is helpful that the project ensures the referral itself, but also patients should be consulted about their needs and preferences for CR information at the bedside (see above), and inpatient units need to collaborate closely with the CR programs to which they refer to ensure they can accept additional patients (or increase their capacity if not). While single-faceted and passive approaches can be effective [31], a systems approach, with tools for all aspects of the process (toolkit), is recommended to achieve CR enrolment targets [15, 16].

Caution is warranted in interpreting these results. The study was single centre, and therefore it must be tested whether findings generalize, particularly to non-academic centers. At least the course was piloted on various types of cardiac wards. Second, unfortunately only nurses were willing to be observed for the needs assessment, but the team / planning committee did represent the types of HCPs targeted for the course. Third, during the structured observation, some HCPs might have altered their behavior due to an awareness that they were observed; however, they were not informed which aspects of their patient interaction was being evaluated. Given the low quality and quantity of the CR-specific content observed, this is likely not a significant concern. Finally, the sample size was small for the survey, and retention low.


The online course developed is the first available to our knowledge to educate HCPs regarding communication at the bedside to encourage patient utilization of CR, as per policy statement recommendations. The results of the evaluation suggest that HCPs who completed the online course had increased CR knowledge, self-efficacy and more positive attitudes. These preliminary results suggest broader dissemination and evaluation is warranted. It is hoped this tool can support inpatient cardiac care units to achieve 70% CR enrolment of their patients, so the high burden of CVD can be ameliorated.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.



Agency for Healthcare Research and Quality


Cardiac rehabilitation


Cardiovascular diseases


Healthcare provider


Healthcare providers


International Council of Cardiovascular Prevention and Rehabilitation


Think aloud protocol


University Health Network


  1. World Health Organization. Cardiovascular Diseases (CVDs) [Internet]. 2017. Available from: [cited 2019 11 May].

  2. Kolansky DM. Acute coronary syndromes: morbidity, mortality, and pharmacoeconomic burden. Am J Manag Care. 2009;15(2 Suppl):S36–41.

    PubMed  Google Scholar 

  3. Roth GA, Johnson C, Abajobir A, Abd-Allah F, Abera SF, Abyu G, et al. Global, regional, and National Burden of cardiovascular diseases for 10 causes, 1990 to 2015. J Am Coll Cardiol. 2017;70(1):1–25.

    Article  Google Scholar 

  4. Tarride J-E, Lim M, DesMeules M, Luo W, Burke N, O’Reilly D, et al. A review of the cost of cardiovascular disease. Can J Cardiol. 2009;25(6):e195–202.

    Article  Google Scholar 

  5. Shields GE, Wells A, Doherty P, Heagerty A, Buck D, Davies LM. Cost-effectiveness of cardiac rehabilitation: A systematic review. Heart. 2018;104(17):1403–10.

    Article  Google Scholar 

  6. Grace SL, Turk-Adawi KI, Contractor A, Atrey A, Campbell NRC, Derman W, et al. Cardiac rehabilitation delivery model for low-resource settings: an International Council of Cardiovascular Prevention and Rehabilitation consensus statement. Prog Cardiovasc Dis. 2016;59(3):1–20.

    Article  Google Scholar 

  7. Anderson L, Oldridge N, Thompson DR, Zwisler A-D, Rees K, Martin N, et al. Exercise-based cardiac rehabilitation for coronary heart disease: Cochrane systematic review and meta-analysis. J Am Coll Cardiol. 2016;67(1):1–12.

    Article  Google Scholar 

  8. Brady S, Purdham D, Oh P, Grace S. Clinical and sociodemographic correlates of referral for cardiac rehabilitation following cardiac revascularization in Ontario. Hear Lung. 2013;42(5):320–5.

    Article  Google Scholar 

  9. Harrison RW, Simon D, Miller AL, De Lemos JA, Peterson ED, Wang TY. Association of hospital myocardial infarction volume with adherence to American College of Cardiology/American Heart Association performance measures: insights from the National Cardiovascular Data Registry. Am Heart J. 2016;178:95–101.

    Article  Google Scholar 

  10. Beatty AL, Truong M, Schopfer DW, Shen H, Bachmann JM, Whooley MA. Geographic variation in cardiac rehabilitation participation in medicare and veterans affairs populations: an opportunity for improvement? Circulation. 2018;137:1899–908.

    Article  Google Scholar 

  11. Kotseva K, Wood D, De Bacquer D, De Backer G, Rydén L, Jennings C, et al. EUROASPIRE IV: A European Society of Cardiology survey on the lifestyle, risk factor and therapeutic management of coronary patients from 24 European countries. Eur J Prev Cardiol. 2016;23(6):636–48.

    Article  Google Scholar 

  12. Ilarraza-Lomelí H, García-Saldivia M, Rojano-Castillo J, Justiniano S, Cerón N, Aranda-Ayala ZL, et al. National registry of cardiac rehabilitation programs in Mexico II (Renaprec II). Arch Cardiol Mex. 2017;87(4):270–7.

    PubMed  Google Scholar 

  13. Clark AM, King-shier KM, Spaling MA, Duncan AS, Stone JA, Susan B, et al. Factors influencing participation in cardiac rehabilitation programmes after referral and initial. Clin Rehabil. 2013;27(10):948–59.

    Article  Google Scholar 

  14. Santiago de Araújo Pio C, Chaves GS, Davies P, Taylor RS, Grace SL. Interventions to promote patient utilisation of cardiac rehabilitation. Cochrane Database Syst Rev. 2019;2:CD007131.

    PubMed  Google Scholar 

  15. Santiago de Araujo Pio C, Beckie T, Sarrafzadegan N, Babu A, Baidya S, Buckley J, et al. Promoting patient utilization of outpatient cardiac rehabilitation: A joint international council and Canadian Association of Cardiovascular Prevention and Rehabilitation position statement. Int J Cardiol. 2019;11(6):209–28.

    Google Scholar 

  16. Santiago De Araújo Pio C, Beckie TM, Varnfield M, Sarrafzadegan N, Babu AS, Baidya S, et al. Promoting patient utilization of outpatient cardiac rehabilitation:A joint international council and Canadian Association of Cardiovascular Prevention and Rehabilitation Position Statement. J Cardiopulm Rehabil Prev. 2020;40(2):79–86.

    Article  Google Scholar 

  17. Tabak RG, Khoong EC, Chambers D, Brownson RC. Models in dissemination and implementation research: useful tools in public health services and systems research. Front Public Heal Serv Syst Res. 2013;2(1):8.

    Google Scholar 

  18. Flodgren G, Hall AM, Goulding L, Eccles MP, Grimshaw JM, Leng GC, et al. Tools developed and disseminated by guideline producers to promote the uptake of their guidelines. Cochrane Database Syst Rev. 2016;8:CD010669.

    Google Scholar 

  19. Brouwers MC, Browman G, Cluzeau F, Davis D, Feder G, Graham I, et al. AGREE II: advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. 2010;182:E839–42.

    Article  Google Scholar 

  20. Institute of Medicine. Standards for Developing Trustworthy Clinical Practice Guidelines. Washington, D.C.; 2011.

  21. Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Can Med Assoc J. 2014 Feb 18;186(3):E123–42.

    Article  Google Scholar 

  22. Gagliardi AR, Brouwers MC, Bhattacharyya OK. The development of guideline implementation tools: a qualitative study. Can Med Assoc J Open. 2015;3(1):E127–33.

    Google Scholar 

  23. Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Barriers and strategies in guideline implementation—A scoping review. Healthcare. 2016;4(3):36.

    Article  Google Scholar 

  24. Grol R. Beliefs and evidence in changing clinical practice. BMJ. 1997;315:418–21.

    Article  CAS  Google Scholar 

  25. Zwerver F, Schellart AJ, Anema JR, Rammeloo KC, van der Beek AJ. Intervention mapping for the development of a strategy to implement the insurance medicine guidelines for depression. BMC Public Health. 2011;11(1):9.

    Article  Google Scholar 

  26. Bartolomew LK, Parcel GS, Kok G, Gottlieb N. Designing theory and evidence-based health promotion programs. In: intervention mapping. Columbus, OH: Mountain View; 2001.

  27. Kirkpatrick D. Great ideas revisited: revisiting Kirkpatrick’s four-level model. Train Dev. 1996;50(1):54–9.

    Google Scholar 

  28. Mitoff PR, Wesolowski M, Abramson BL, Grace SL. Patient-provider communication regarding referral to cardiac rehabilitation. Rehabil Nurs. 2005;30(4):140–6.

    Article  Google Scholar 

  29. Pourhabib S, Kentner AC, Grace SL. The impact of patient-healthcare provider discussions on enrollment in cardiovascular rehabilitation. J Rehabil Med. 2014;46(9):924–31.

    Article  Google Scholar 

  30. Mays N, Pope C. Qualitative research: observational methods in health care settings. BMJ. 1995;311(6998):182–4.

    Article  CAS  Google Scholar 

  31. Liang L, Abi Safi J, Gagliardi AR. Number and type of guideline implementation tools varies by guideline, clinical condition, country of origin, and type of developer organization: content analysis of guidelines. Implement Sci. 2017;12(1):136.

    Article  Google Scholar 

  32. Gagliardi AR, Brouwers MC, Bhattacharyya OK. A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools. Implement Sci. 2014;9(1):98.

    Article  Google Scholar 

  33. Eccles DW, Arsal G. The think aloud method: what is it and how do I use it? Qual Res Sport Exerc Heal. 2017;9(4):514–31.

    Article  Google Scholar 

  34. Wilson TD, Streatfield DR. Structured observation in the investigation of information needs. Soc Sci Inf Stud. 1981;1(3):173–84.

    Google Scholar 

  35. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.

    Article  Google Scholar 

  36. Fonteyn ME, Kuipers B, Grobe SJ. A description of think aloud method and protocol analysis. Qual Health Res. 1993;3(4):430–41.

    Article  Google Scholar 

  37. Hutchison P, Monaghan I, Morgan R. A multidimensional analysis method for think-aloud protocol data: Physics Education Research Conference Proceedings. American Association of Physics Teachers; 2015. p. 147–50.

  38. Charters E. The use of think-aloud methods in qualitative research an introduction to think-aloud methods. Brock Educ J. 2003;12(2):1–15.

    Article  Google Scholar 

  39. Naik G, Ahmed H, Edwards AG. Communicating risk to patients and the public. Br J Gen Pract. 2012;62(597):213–6.

    Article  Google Scholar 

  40. Turner T, Misso M, Harris C, Green S. Development of evidence-based clinical practice guidelines (CPGs): comparing approaches. Implement Sci. 2008;3(1):45.

    Article  Google Scholar 

  41. Moghei M, Oh P, Chessex C, Grace SL. Cardiac rehabilitation quality improvement: A narrative review. J Cardiopulm Rehabil Prev. 2019 Jan;39(4):226–34.

    Article  Google Scholar 

  42. Sansoni JE, Grootemaat P, Duncan C. Question prompt lists in health consultations: A review. Patient Educ Couns. 2015;98(12):1454–64.

    Article  Google Scholar 

Download references


The authors acknowledge Marcella Calouro, whose prior work informed the basis of the project, and Teresa Pollex who supported the launch of the eLearning course. We are grateful to our patient partners John Sawdon and Paul Moffat for providing valuable input on the course content. We also acknowledge Dr. Susan Marzolini for providing input into the early vision of the work, and Anda Lukic for serving on the planning committee. Finally, we acknowledge Dr. Gabriela Chaves who assisted with the quantitative survey data.


The authors declare that there was no funding to support this work.

Author information

Authors and Affiliations



CSAP was responsible for conceiving the project, collecting and analyzing data, as well as drafting the online course and the manuscript. AG gave substantive input regarding the design / methods of the evaluation, and in terms of the type of guideline implementation tool that was chosen. She also critically revised the manuscript for important intellectual content. NS gave substantive input regarding the development of the online course and its’ content, serving on the development committee. He also critically revised the manuscript for important intellectual content. FA gave substantive input regarding the design / methods of the evaluation. She also critically revised the manuscript for important intellectual content. She served on the dissertation committee of the first author. SLG was responsible for developing the project in conjunction with the first author and overseeing it, providing input into/ finalizing the course content and evaluation methods and tools, facilitating HCP recruitment and data collection, analysis and interpretation of data, and drafting the manuscript with the first author. All authors gave final approval of the manuscript.

Corresponding author

Correspondence to Sherry L. Grace.

Ethics declarations

Ethics approval and consent to participate

On the basis of the A pRoject Ethics Community Consensus Initiative (ARECCI; tool, an ethics waiver was granted by the University Health Network research ethics board, and then York University’s Office of Research Ethics, as this was a quality improvement initiative. Procedures outlined to minimize and mitigate ethical risks (e.g., privacy / confidentiality; data storage /security; informed consent) were strictly followed. All HCPs voluntarily agreed to participate and verbal consent was obtained. Written consent was not required by the ethics board considering also participants were providers and that the project was minimal risk. All patients observed also voluntarily agreed to participation, after the purpose of the observation was explained to them (except we focused on the full discharge discussion, rather than CR specifically), and they remained anonymous.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests related to this study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: Appendix 1.

Coding Guide for Structured Observation of Patient-Provider Interaction. Appendix 2. Think Aloud Protocol and Semi-Structured Interview Guide for Tool / Course Pilot Test. Appendix 3. Online Course Surveys.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santiago de Araújo Pio, C., Gagliardi, A., Suskin, N. et al. Implementing recommendations for inpatient healthcare provider encouragement of cardiac rehabilitation participation: development and evaluation of an online course. BMC Health Serv Res 20, 768 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: