- Study protocol
- Open Access
- Open Peer Review
Quasi-experimental trial of diabetes Self-Management Automated and Real-Time Telephonic Support (SMARTSteps) in a Medicaid managed care plan: study protocol
BMC Health Services Research volume 12, Article number: 22 (2012)
Health information technology can enhance self-management and quality of life for patients with chronic disease and overcome healthcare barriers for patients with limited English proficiency. After a randomized controlled trial of a multilingual automated telephone self-management support program (ATSM) improved patient-centered dimensions of diabetes care in safety net clinics, we collaborated with a nonprofit Medicaid managed care plan to translate research into practice, offering ATSM as a covered benefit and augmenting ATSM to promote medication activation. This paper describes the protocol of the Self-Management Automated and Real-Time Telephonic Support Project (SMARTSteps).
This controlled quasi-experimental trial used a wait-list variant of a stepped wedge design to enroll 362 adult health plan members with diabetes who speak English, Cantonese, or Spanish and receive care at 4 publicly-funded clinics. Through language-stratified randomization, participants were assigned to four intervention statuses: SMARTSteps-ONLY, SMARTSteps-PLUS, or wait-list for either intervention. In addition to usual primary care, intervention participants received 27 weekly calls in their preferred language with rotating queries and response-triggered education about self-care, medication adherence, safety concerns, psychological issues, and preventive services. Health coaches from the health plan called patients with out-of-range responses for collaborative goal setting and action planning. SMARTSteps-PLUS also included health coach calls to promote medication activation, adherence and intensification, if triggered by ATSM-reported non-adherence, refill non-adherence from pharmacy claims, or suboptimal cardiometabolic indicators. Wait-list patients crossed-over to SMARTSteps-ONLY or -PLUS at 6 months. For participants who agreed to structured telephone interviews at baseline and 6 months (n = 252), primary outcomes will be changes in quality of life and functional status with secondary outcomes of 6-month changes in self-management behaviors/efficacy and patient-centered processes of care. We will also evaluate 6-month changes in cardiometabolic (HbA1c, blood pressure, and LDL) and utilization indicators for all participants.
Outcomes will provide evidence regarding real-world implementation of ATSM within a Medicaid managed care plan serving safety net settings. The evaluation trial will provide insight into translating and scaling up health information technology interventions for linguistically and culturally diverse vulnerable populations with chronic disease.
As health systems and public health agencies grapple with the rising burden of diabetes, they face significant challenges with the persistent and widening disparities in the prevalence, quality of care, and clinical outcomes across such factors as race/ethnicity, language, health literacy, and socioeconomic status [1–3]. Patient-centered, culturally concordant care is a cornerstone for reducing disparities, but vulnerable populations with limited health literacy and limited English proficiency face barriers to access and communication that place them at risk for suboptimal treatment management and poor health outcomes [4–10, 10–13].
Self-management support programs are a critical component of chronic disease care delivery that can improve outcomes by providing individualized assessment, collaborative goal-setting, skills enhancement, follow-up, and access to resources and continuity of care [14–16]. Despite the proven benefits of self-management support and evidence that vulnerable populations desire these interventions , the translational gap between research and practice may be particularly wide in safety net settings, which disproportionately care for these patients [18, 19]. Providing self-management support is resource-intensive, requiring re-training of staff and organizational change, investments in information technology, and tailoring programs to engage and serve diverse populations [20–22]. Thus, traditional self-management support approaches often do not reach significant and growing segments of the chronic disease population, such as the uninsured or publicly-insured or those with communication barriers [1, 6, 17, 23–25]. Given the documented challenges with translating research into real-world practice, practical implementation and evaluation research is needed in settings where vulnerable populations receive care [4, 26–31].
Patient-facing health information technology (HIT) holds promise to increase access to self-management support and enhance health outcomes for vulnerable populations in safety net settings [32–34]. Automated telephone self-management support (ATSM) employs phone technology to provide surveillance and education and to prioritize further care management efforts for those most in need . ATSM can be used to promote collaborative goal-setting through behavioral action plans, by which patients set and achieve short-term goals to improve their self-management . ATSM can also provide individualized assessment, skills enhancement, live follow-up and support from health educators or coaches, access to community resources, and continuity of clinical care [32, 33]. The Institute of Medicine has highlighted ATSM as an exemplary strategy in its National Action Plan for Health Literacy .
The Improving Diabetes Efforts Across Language and Literacy (IDEALL) study - a randomized controlled trial of ATSM among English-, Spanish-, and Chinese-speaking patients with poorly controlled diabetes - demonstrated high engagement with ATSM, particularly among participants with limited health literacy and limited English proficiency . Compared with patients receiving usual care or group medical visits, patients exposed to ATSM had greater improvements in their experiences of chronic illness care, self-management behavior, fewer bed days per month, less interference in their daily activities, with a cost utility for functional outcomes comparable to other diabetes prevention and treatment interventions [33, 37]. We hypothesized that ATSM could be augmented by harnessing pharmacy claims and clinical registry data, allowing health coaches to identify patient non-adherence and missed opportunities for medication intensification and conduct patient-centered counseling outside of the time-constrained office visit [38, 39]. Given these potential benefits, the San Francisco Health Plan (SFHP) - a non-profit Medicaid managed care plan serving a diverse, low-income population - approached our research team at the University of California, San Francisco (UCSF) Center for Vulnerable Populations to evaluate the implementation of ATSM as a covered health benefit for a subset of its members with diabetes and add an augmented intervention harnessing medication claims to promote medication adherence and intensification.
This paper presents a description of the study design and recruitment results for the Self-Management Automated and Real-Time Telephonic Support Study (SMARTSteps/Pasos Positivos/ 明智進步計劃), a controlled quasi-experimental evaluation study to improve diabetes quality of life, self-management, and clinical outcomes with two variants of language-concordant ATSM intervention implemented by a Medicaid managed care plan serving a low-income, ethnically diverse urban population. We hypothesized that intervention participants would demonstrate greater improvements in diabetes self-management, patient-centered outcomes, and cardiometabolic outcomes compared to wait-list controls and that improvements would be greatest for patients exposed to ATSM with an enhanced medication activation communication strategy triggered by pharmacy claims and self-reported medication non-adherence.
Among ethnically and linguistically diverse persons with diabetes receiving care in a primary care safety net system:
Primary Aim: To investigate differences in 6-month changes in patient-centered outcomes including quality of life and functional status (SF-12 and number of days spent in bed due to illness), comparing patients exposed to ATSM with wait-list controls and comparing patients exposed to ATSM (SMARTSteps-ONLY) with ATSM augmented by medication adherence and intensification (SMARTSteps-PLUS).
Secondary Aims: To investigate differences in 6-month changes in diabetes self-management behaviors and self-efficacy, patient-centered processes of care, cardiometabolic outcomes (HbA1c, blood pressure, and LDL), receipt of recommended health services, and healthcare utilization, comparing patients exposed to ATSM with wait-list controls and comparing patients exposed to ATSM (SMARTSteps-ONLY) with ATSM augmented by medication adherence and intensification (SMARTSteps-PLUS).
The study was a controlled quasi-experimental evaluation trial with a wait-list variant of a stepped wedge design (see Figure 1), which has been described previously in the implementation science literature [40–42]. This design is characterized by staggered introduction of an intervention over time, cross-over of individual patients from control to intervention arm at designated times, and multiple data collection points. A stepped wedge design allows for practical allocation of intervention resources over time, with introduction of the intervention in stages, such that all members receive the intervention eventually, while still retaining elements of a controlled trial . Compared with a randomized clinical trial, or a simple before and after design, the stepped wedge design was more acceptable to SFHP, which felt it would be unethical to deny some members ATSM as a covered benefit while offering it to others, given prior evidence of ATSM's positive impact on health outcomes. This design also allowed SFHP to scale up recruitment and implementation in a controlled fashion, with less intensive staffing than required if offering the intervention to all eligible members at project initiation (as in a cluster randomized design). Finally, conducting four recruitment waves over 24 months allowed for collection of intervention and control data across multiple waves, allowing analysis to take into account possible variations over time .
The study sample was drawn from SFHP members who received primary care at one of four publicly-funded clinics in the Community Health Network of San Francisco (CHNSF). SFHP is a non-profit Medicaid managed care plan created in 1994 to provide high quality medical care to the largest possible number of low-income San Francisco residents. In addition to managing Medicaid benefits, SFHP administers "Healthy Workers," an insurance program for people working as in-home support service providers for elderly or disabled people in San Francisco. At the time SFHP contacted the UCSF investigative team in 2008, it managed the care of approximately 50,000 enrollees and served an ethnically diverse population comprised of 48% Asian-Americans, 26% Latinos, 12% African-Americans; 60% of their membership were non-English speaking.
CHNSF is the integrated healthcare delivery system of the San Francisco Department of Public Health and part of the UCSF San Francisco Bay Area Collaborative Research Network, one of the longest standing U.S. practice-based research networks (http://familymedicine.medschool.ucsf.edu/research/research_programs/crn/crn.aspx). CHNSF is comprised of 12 community health centers and hospital-based clinics centers affiliated with the county's acute care facility, San Francisco General Hospital. CHNSF maintains a single electronic medical information system with unique patient identifiers across the clinical sites and a clinical registry that integrates patient data such as diagnoses, vital signs, and laboratory data.
Eligible participants were characterized by the following: SFHP membership; ≥1 primary care clinic visit in the preceding 24 months at one of our designated clinics; age 18 or above; a diagnosis of diabetes (type 1 or 2); English-, Cantonese-, or Spanish-speaking; access to a touch-tone phone; and plans to remain in the region during the evaluation period (12 months). Patients who were pregnant or unable to provide verbal consent were excluded. The diagnosis of diabetes was assessed in two ways. First, we searched for SFHP members in the CHNSF electronic diabetes registry. Second, we searched for members with SFHP claims data consistent with diabetes (ICD-9 codes for utilization and pharmacy claims for oral or injectable glycemic medications or glucose self-monitoring supplies). If members were identified through SFHP claims data but were not found in the registry, we validated the diabetes diagnosis by reviewing electronic health records for fasting glucose ≥ 126 mg/dl and/or clinician-documented diagnosis of diabetes based on HbA1c ≥ 7% . During the course of the evaluation trial, the American Diabetes Association revised diagnostic criteria to include HbA1c ≥6.5% , but we did not modify our eligibility criteria to avoid introducing selection bias and because evidence suggests a variable delay in the adoption of guideline criteria by practicing providers [45, 46].
Sample recruitment procedures
SFHP mailed to potentially eligible members' homes and clinicians' offices SMARTSteps post cards describing this new benefit in English, Spanish, and Cantonese and offering an SFHP number to call for information. SFHP enrollment workers fluent in these three languages also actively recruited potentially eligible health plan members through scripted telephone calls. The enrollment workers confirmed eligibility for SMARTSteps by phone and offered $25 gift cards as incentives for participation. All eligible members who reported interest in SMARTSteps were then randomized by SFHP to one of four intervention statuses (described below) regardless of interest in the UCSF evaluation.
SFHP workers also assessed participant willingness to complete evaluation interviews administered by UCSF staff. Those members willing to participate in the interviews were then telephoned by UCSF research assistants to obtain verbal consent for participation in the evaluation interviews and were offered $50 gift cards as incentives for each interview completed (up to $150).
The Committee on Human Research at the University of California, San Francisco (H9894-32044-01) approved collection of interview data by UCSF from consenting participants after obtaining verbal consent. Because all recruitment and implementation was conducted by SFHP by phone, it was felt that requiring a separate visit to obtain written consent at UCSF or the primary care clinic sites reduce participation, leading to selection bias, and disrupt clinic workflow. Because the interviews offered minimal risk, the Committee approved verbal telephone consent for these interviews. In addition, ethics approval included waiver of informed consent to abstract and analyze CHNSF clinical data from the CHNSF electronic health record, CHNSF diabetes registry, SFHP claims data, and SFHP health coaching data for all SFHP members eligible for SMARTSteps.
All eligible SFHP members interested in the SMARTSteps program were assigned by SFHP through language-stratified randomization to one of four intervention statuses: SMARTSteps-ONLY, SMARTSteps-PLUS, wait-list for SMARTSteps-ONLY, or wait-list for SMARTSteps-PLUS.
Those randomized to the wait-list continued to receive usual care through their clinics, as well as all existing SFHP benefits (reminders and incentives for receipt of recommended health services, including laboratory testing, eye and foot examination, and influenza vaccination) [43, 47]. At the end of the 6-month wait-list period, each participant "crossed-over" to begin SMARTSteps-ONLY or SMARTSteps-PLUS, depending on initial randomization arm.
Participants randomized to SMARTSteps-ONLY received the ATSM intervention within 2 weeks. Developed with extensive input from patients to be sensitive to literacy, language, and culture in the target populations , this ATSM system provided 27 weeks of 8-12 minute weekly calls in English, Cantonese, or Spanish. Patients specified the weekday and time convenient for their schedules or called toll-free into the system if they missed their scheduled call. The content consisted of rotating sets of queries about self-care (such as diet, exercise, and medication adherence), psychosocial issues (such as depressive symptoms), and access to preventive services (such as eye care). Patients responded via touch-tone commands, and based on their answers, patients heard automated health education messages in the form of narratives.
Patients answering "out-of-range" on an item or selecting a "call back" option received a telephone call within 3 days from a language-concordant SFHP health coach. This system was designed to promote health coach efficiency and effectiveness by focusing outreach calls to patients needing further support based on ATSM responses. Health coaches provided education and engaged in collaborative goal-setting to form patient-centered action plans, a core process for self-management support by which patients set short-term goals to improve their self-management [20, 35].
SFHP used a health coach for responding to ATSM triggers, rather than a nurse practitioner care manager as was done in the IDEALL trial . Health coaches - SFHP employees without a medical background or postgraduate training - conducted phone calls under the supervision of a registered nurse care manager. SFHP registered nurse supervisors and UCSF study physicians taught health coaches about diabetes and trained them in behavior change counseling and overcoming barriers to health communication. UCSF staff also guided the development of written protocols and scripts to respond to potential ATSM call triggers, such as assessing hypoglycemia symptoms or causes.
Health coaches documented the content of their phone interactions, including patient-generated action plans and whether they were achieved, using an SFHP care management database system. SFHP health coaches also contacted the designated clinic contact and/or the primary care provider (PCP) using standardized templates for patients with pre-specified safety issues (such as a new medical symptom) or access concerns (such as need for refills or appointments). Non-urgent issues were communicated by email or fax, while urgent issues were also conveyed by phone.
Patients who indicated on their ATSM responses that they missed their diabetes, blood pressure, or cholesterol medications for 3-7 days in the previous week received calls from health coaches to troubleshoot barriers to adherence. These calls were only triggered by patient self-disclosure in response to ATSM queries, unlike adherence counseling calls triggered for SMARTSteps-PLUS participants described below.
The goal of the SMARTSteps-PLUS intervention was to detect and intervene for participants whose medication treatment was suboptimal, either because of non-adherence or potential missed opportunities to intensify their regimens. Because SFHP does not have prescribing authorization for its members, the intervention focused on enhancing medication regimens and adherence through collaborative goal-setting with patients and feedback to PCPs. Participants randomized to SMARTSteps-PLUS immediately received the ATSM intervention described above, as well as medication activation and intensification coaching triggered by refill non-adherence or suboptimal achievement of cardiometabolic treatment goals. Based on review of evidence-based guidelines , SFHP and UCSF collaborated to develop a protocol to improve medication adherence and promote patient-centered intensification by harnessing electronically available data from two additional sources: monthly SFHP pharmacy claims and clinical laboratory and blood pressure data from the electronic health records of the CHNSF.
The medication activation protocol targeted 3 groups of patients within the SMARTSteps-PLUS arm (see Figure 2):
Patients who indicated in response to weekly ATSM queries that they missed 3-7 days of medications in the last week.
Patients with a 15-day to 6-month gap in refilling specific cardiometabolic prescriptions: oral hypoglycemic, insulin, anti-hypertensives (including ACE inhibitors and angiotension-receptor blockers), cholesterol-lowering medications, or glucose testing strips (for patients receiving insulin or sulfonylureas), based on pharmacy claims data. We chose a lower limit of 15-days to account for potential delays in SFHP receipt of pharmacy claims and an upper limit of 6 months in case healthcare providers actively discontinued these medications.
Patients with suboptimal achievement of cardiometabolic goals :
Systolic blood pressure ≥ 130 mmHg at last recorded check or no measurement in clinical registry within preceding 6 months
Diastolic blood pressure ≥ 80 mmHg at last recorded check or no measurement in clinical registry within preceding 6 months
Hemoglobin A1c (HbA1c) > 7.0% on last measurement or no measurement in clinical registry within preceding 6 months
Low-density lipoprotein (LDL) > 100 mg/dL on last measurement or no measurement in clinical registry within preceding 12 months
Urinary albumin/creatinine > 30 mcg/mg for patients who were not prescribed or late to refill an ACE inhibitor or angiotensin-receptor blocker
An individual SMARTSteps-PLUS participant could fall into none, one, or more than one of these groups at any given time. SFHP queried the pharmacy claims and clinical registry data monthly to provide health coaches with a list of SMARTSteps-PLUS participants to be called.
For patients with evidence of non-adherence, SFHP health coaches were trained with specific written protocols and scripts to promote self-disclosure of medication non-adherence and troubleshoot barriers such as confusion about doses or frequency of medications, forgetfulness, concerns about side effects, or health beliefs (see Additional File 1). For patients who had not had cardiometabolic measurements within guideline-concordant timeframes, health coaches counseled patients about the reasons for these measures, encouraged them to talk with their providers, and notified PCPs of patients willing to obtain these tests. For patients with suboptimal achievement of treatment goals, health coaches were trained to counsel patients about their blood pressure or lab values and assess their willingness to discuss with their PCPs the possibility of taking more medication. Health coaches also encouraged patients to undergo potential lifestyle changes. Based on their discussions, health coaches engaged patients in collaborative goal-setting and action planning [20, 35]. Health coaches then notified the clinic contacts and/or PCPs about interactions with any patients who needed laboratory or blood pressure assessments, reported barriers to medication adherence, or indicated a willingness to discuss medication intensification.
The study design conforms to CONSORT statement recommendations for randomized trials of non-pharmacologic treatment . Randomization to intervention groups was conducted by SFHP before assessment of willingness to participate in UCSF evaluation interviews to permit analysis of primary outcomes. Intervention arm assignment was conducted using a 4 × 4 blocked randomized design, with the participant as the unit of randomization. This randomization was completed for each of the three study languages. Research assistants administering the evaluation interviews were blinded to group allocation. The intervention protocol is documented and filed on secure servers. Data generated during ATSM and health coach calls is stored in the ATSM and SFHP database systems, respectively. All analyses will be intent-to-treat.
Table 1 lists study outcomes.
For participants consenting to the UCSF evaluation trial, measures of all primary and secondary patient-reported outcomes were derived from structured interviews conducted in English, Spanish or Cantonese. For Spanish and Cantonese interviews, survey questions were translated and back-translated in their respective languages. Interviews were administered using computer-assisted telephone interview (CATI) software or paper survey, with responses later entered into the CATI-created database. Consenting participants completed baseline interviews within 2 weeks of being randomized, prior to the first ATSM call. SMARTSteps-ONLY and SMARTSteps-PLUS participants completed the 6-month follow-up interviews after the conclusion of their ATSM intervention, and wait-list participants completed the 6-month follow-up interviews before the start of their ATSM calls. Wait-list groups received an additional 6-month follow-up interview (12 months after enrollment) after completion of intervention period. Participants received a $50 gift card for a pharmacy store at the completion of each interview.
For all eligible SFHP members, we collected socio-demographic variables and cardiometabolic outcomes data using monthly downloads of SFHP claims data and CHNSF clinical registry data. At the conclusion of the project, we will obtain SFHP administrative and claims data for utilization outcomes.
Self-reported socio-demographic variables included age, gender, race/ethnicity, language, English proficiency, years since immigration to the U.S., marital status, educational attainment, employment status, annual household income, self-reported health literacy [49, 50], and duration of diabetes. We obtained SFHP administrative and CHNSF data regarding age and gender. Preferred language is captured in both SFHP and CHNSF data, and where this preference was discrepant, we used the language reported to SFHP enrollment workers.
Primary and Secondary Patient-Reported Outcomes
The primary outcome variables are physical and mental functional status (SF-12) and the number of days spent in bed due to illness . Secondary outcomes include diabetes self-efficacy and self-management behavior, as well as medication adherence in the preceding 7 days [52–54]. Secondary outcomes also include patient perspectives on the structure of their care, as measured by two instruments which have been translated and validated for our population in our prior work: Patient Assessment of Care for Chronic Conditions (PACIC) and interpersonal processes of care (IPC) [33, 55, 56].
These secondary outcomes will be analyzed on the larger set of SFHP participants randomized in the trial (N = 362), regardless of whether or not they participated in the evaluation interviews. Cardiometabolic outcomes include measures of HbA1C, LDL, and blood pressure obtained by clinical providers. Data will be derived from CHNSF electronic health record, the clinical registry database, and paper chart review. Values will be used at baseline only if obtained within one year before program enrollment. Follow-up laboratory and blood pressure values will be included only if obtained within 90 days before or after the 6-month and 12-month follow-up dates.
Care Delivery Processes and Utilization
Finally, for all eligible participants, we will use SFHP administrative claims data to examine measures of care delivery processes: proportion receiving blood pressure checks, laboratory testing, retinal examinations, influenza vaccination, and pneumococcal vaccination at guideline-concordant frequencies . We will also examine utilization, including the frequency of emergency department visits and hospitalizations.
Program Engagement and Satisfaction
Using data from the ATSM system, we will assess patient engagement as measured by exposure intensity (defined as the proportion of the 27 weeks of exposure, participants replied to the weekly calls) and the proportion of weekly calls that are determined to be complete (calls answered and responses to 100% of queries), partially complete (call answered and responses to < 50% of queries), and incomplete (call answered but responses to 0% of queries) . In addition, using data from the SFHP health coach database, we will analyze at the participant level the number of health coach calls that were triggered by ATSM responses; the number of calls attempted and/or completed by health coaches; and the number of action plans created and/or achieved, as reported by patients and documented in the database. We will conduct fidelity assessments to examine potential differential engagement across languages, clinics and study arms. Finally, we will assess satisfaction and perceived usefulness of the intervention as reported during evaluation interviews at 6- and 12-month follow-up.
We will assess the similarity of baseline characteristics between study arms using t-tests, chi-square tests, and Fisher's exact tests. We will control for any significant differences in baseline characteristics in analyses of all primary and secondary outcomes.
For each outcome, we will compare differences between study arms in 6-month changes using intent-to-treat analyses. Because this study is a quasi-experimental evaluation trial with a wait-list variant of a stepped wedge design, analyses of outcome changes will be conducted with two distinct methodologies. The first method will involve analyzing the data as if the trial was a standard intervention versus control study, comparing all participants who were randomized to the ATSM intervention (either SMARTSteps-ONLY or SMARTSteps-PLUS) versus all participants randomized to the wait-list control groups. We will use linear regression for continuous variables, logistic regression for dichotomous variables, and negative binomial models for the outcome variable of number of bed days because of its non-parametric data distribution. We will include as model covariates the baseline value of outcome variables and any socio-demographic or medical variables that may have differed by chance among the groups.
The second method will take advantage of the wait-list variant, stepped wedge design with study cross-over of participants from control to intervention . It will employ a repeated measures analysis using maximum likelihood to fit the models. Because discrete external events may coincide with the time that participants cross-over from control to intervention and thereby possibly confound the within-group comparisons from the wait-list control group, secular effects in the wait-list group will be modeled first to see if a specific spline to capture non-linearities analysis is needed or if a simple linear trend exists, simplifying the analysis. Other possible covariates in the model may be calendar time, actual time of intervention exposure, and predictors of dropping out of study.
To explore the added value of the medication activation arm, both the first and second methods above will also be used to compare SMARTSteps-ONLY versus SMARTSteps-PLUS in all primary and secondary outcomes.
Sample size calculations
Sample size calculations were based on comparison of primary outcomes for the combined intervention groups (SMARTSteps-ONLY and SMARTSteps-PLUS) with the combined wait-list control groups. Based on an eligible population of approximately 500 SFHP members, we anticipated recruitment of 260 participants to intervention and control with a 10% drop-out or loss to follow-up rate at 6 months. Using a two-tailed test, we calculated that we would have 80% power to detect a standardized effect size (SE) of 0.35 in the primary outcome of SF-12 scores, comparing the combined intervention groups with the combined wait-list groups. In the prior RCT, we observed standardized effect sizes in this range .
We estimated that over the course of the two-year observation period, approximately 20% of enrolled participants would have insufficient data for secondary cardiometabolic outcomes to be included in these analyses. Under this assumption, we would have 80% power to detect a difference in HbA1c of 0.51% (SE 0.28) when comparing wait-list vs. combined ATSM interventions participants. This effect size is intermediate between that estimated to be the effect on glycemic control of diabetes disease management programs that involve team changes (-0.3% HbA1c improvement) and those that involve medication intensification strategies (-0.6%) .
Figure 3 shows the CONSORT diagram for recruitment of participants into the SMARTSteps Project by SFHP and into the UCSF evaluation. Of 910 SFHP members originally assessed for eligibility, 548 were excluded: 220 did not meet inclusion criteria, 168 could not be contacted, and 160 declined participation. Of note, because SFHP membership was continually changing, SFHP categorized people as "not eligible for SFHP" if they had not been an active SFHP member for more than a month and were not yet contacted by SFHP for recruitment.
At close of evaluation trial, 362 patients were enrolled and randomized for the SMARTSteps program, with 278 completing baseline interviews for the evaluation trial. Table 2 shows the socio-demographic and medical characteristics for those who enrolled in SMARTSteps, SFHP members who declined enrollment, and those who were not contacted for recruitment (some of whom may have been found ineligible if they had been contacted to complete the eligibility screening survey). Compared with the members who declined, SMARTSteps participants were younger; more likely to be women, Hispanic/Latino, and non-English-speaking; and less likely to be white/Caucasian. Compared with non-contacted members, SMARTSteps participants had LDL values and were more likely to be non-English speaking and to have Healthy Workers insurance.
With SF-12 scores on 278 patients at baseline, assuming 10% loss to a 6-month follow-up sample of 250, we would have an 80% power to detect a mean difference of 2.85 in the SF-12 physical component score and 3.44 in the SF-12 mental component score.
Clinical registry data included available baseline HbA1c values for 348 total patients. With an assumption of 20% loss to follow-up, our study would have 80% power to detect a HbA1c difference of 0.37%.
The results of our recruitment process suggest that ATSM with health coach counseling may be a viable strategy for low-income managed care plans caring for linguistically and culturally diverse vulnerable persons with diabetes using population-based recruitment. In the IDEALL study, we found high levels of engagement, particularly with non-English speakers, and our early measures of reach suggest that this intervention model can be scaled up for larger populations [32, 33, 37].
The quasi-experimental design of the SMARTSteps evaluation trial offers a useful illustration of the advantages and challenges of a wait-list variant of a stepped wedge design. This design enabled all eligible SFHP members to receive ATSM as a benefit and increased the feasibility of scaling up the intervention through recruitment waves, while retaining elements of randomization that help minimize bias. However, this design requires careful evaluation of both intervention fidelity and potential confounders over time as well as investigation into the impact of extended follow-up time, potential time trends, and the impact of clustering in the data analysis strategy .
Translational evidence from safety net settings is needed to understand whether HIT applications such as ATSM can be effective in real-world settings and to inform how to harness HIT to support chronic disease self-management among vulnerable populations and improve patient-centered outcomes. In response to Medicaid and Medicare incentives arising from the 2009 Health Information Technology for Economic and Clinical Health Act, safety net settings are required to integrate electronic health records (EHRs) into the care environment and meet "meaningful use" criteria, including patient education and tracking of quality indicators [58–60]. In addition, federal and state policy has steered increasing numbers of vulnerable populations into capitated, managed care arrangements, with some promising results with respect to chronic disease outcomes and disparities reductions . Expanding HIT integration among Medicaid managed health plans, provider groups, and patients will increasingly be a critical avenue for improving the health of vulnerable populations.
The results of this evaluation trial, when completed, must be interpreted within the following limitations. First, SFHP's phone-based population recruitment targeted different languages over time due to enrollment worker availability, with greater emphasis on Cantonese-speaking populations earlier. Thus, potential differences in intervention impact by language may relate to changes in project implementation over time. Second, because SFHP does not have medication prescribing authority for its members or authority to document within participants' medical records, the interventions will not be fully-integrated into care delivery in the context of participants' medical homes; this may limit the effectiveness of the intervention, particularly the SMARTSteps-PLUS medication intensification arm. Finally, because data for cardiometabolic outcomes relies on collection through routine medical care rather than scheduled study visits, we may not be powered to detect a modest impact of the intervention on glycemic or blood pressure control.
Improving health communication through the use of tailored, proactive HIT is one scalable vehicle to improve chronic disease care for vulnerable populations [32, 33]. Focusing on improving the processes of care for vulnerable populations in under-resourced safety net settings through the application of evidence-based or evidence-informed HIT tools can have far-reaching implications for bridging the clinical care-public health divide, not only for diabetes, but for other chronic conditions and for preventive services . The SMARTSteps outcomes will provide evidence regarding the real-world implementation of ATSM within a Medicaid managed care plan and provide insight into translating and scaling up HIT interventions for linguistically and culturally diverse vulnerable populations with chronic disease.
automated telephone self-management
computer-assisted telephone interview
Community Health Network of San Francisco
health information technology
Interpersonal Processes of Care instrument
Patient Assessment of Care for Chronic Conditions instrument
primary care provider
Summary of Diabetes Self-Care Activities instrument
standardized effect size
San Francisco Health Plan
Self-Management Automated and Real-Time Telephonic Support
University of California, San Francisco
United States. Dept. of Health and Human Services, United States. Agency for Healthcare Research and Quality: 2010 National Healthcare Disparities Report. 2010, Rockville, MD.: U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality
Brown AF, Mangione CM, Saliba D, Sarkisian CA, California Healthcare Foundation/American Geriatrics Society Panel on Improving Care for Elders with Diabetes: Guidelines for improving the care of the older person with diabetes mellitus. J Am Geriatr Soc. 2003, 51 (5 Suppl Guidelines): S265-80.
Fernandez A, Schillinger D, Warton EM, Adler N, Moffet HH, Schenker Y, Salgado MV, Ahmed A, Karter AJ: Language barriers, physician-patient language concordance, and glycemic control among insured Latinos with diabetes: the Diabetes Study of Northern California (DISTANCE). J Gen Intern Med. 2011, 26 (2): 170-176. 10.1007/s11606-010-1507-6.
Cooper LA, Hill MN, Powe NR: Designing and evaluating interventions to eliminate racial and ethnic disparities in health care. J Gen Intern Med. 2002, 17 (6): 477-486. 10.1046/j.1525-1497.2002.10633.x.
Sarkar U, Handley MA, Gupta R, Tang A, Murphy E, Seligman HK, Shojania KG, Schillinger D: What happens between visits? Adverse and potential adverse events among a low-income, urban, ambulatory population with diabetes. Qual Saf Health Care. 2010, 19 (3): 223-228. 10.1136/qshc.2008.029116.
Schillinger D, Barton LR, Karter AJ, Wang F, Adler N: Does literacy mediate the relationship between education and health outcomes? A study of a low-income population with diabetes. Public Health Rep. 2006, 121 (3): 245-254.
Schillinger D: Misunderstanding prescription labels: the genie is out of the bottle. Ann Intern Med. 2006, 145 (12): 926-928.
Wolf MS, Davis TC, Shrank W, Rapp DN, Bass PF, Connor UM, Clayman M, Parker RM: To err is human: patient misinterpretations of prescription drug label instructions. Patient Educ Couns. 2007, 67 (3): 293-300. 10.1016/j.pec.2007.03.024.
Davis TC, Wolf MS, Bass PF, Thompson JA, Tilson HH, Neuberger M, Parker RM: Literacy and misunderstanding prescription drug labels. Ann Intern Med. 2006, 145 (12): 887-894.
Fang MC, Machtinger EL, Wang F, Schillinger D: Health literacy and anticoagulation-related outcomes among patients taking warfarin. J Gen Intern Med. 2006, 21 (8): 841-846. 10.1111/j.1525-1497.2006.00537.x.
Fernandez A, Schillinger D, Grumbach K, Rosenthal A, Stewart AL, Wang F, Perez-Stable EJ: Physician language ability and cultural competence. An exploratory study of communication with Spanish-speaking patients. J Gen Intern Med. 2004, 19 (2): 167-174. 10.1111/j.1525-1497.2004.30266.x.
Castro CM, Wilson C, Wang F, Schillinger D: Babel babble: physicians' use of unclarified medical jargon with patients. Am J Health Behav. 2007, 31 (Suppl 1): S85-95.
Sudore RL, Landefeld CS, Williams BA, Barnes DE, Lindquist K, Schillinger D: Use of a modified informed consent process among vulnerable patients: a descriptive study. J Gen Intern Med. 2006, 21 (8): 867-873. 10.1111/j.1525-1497.2006.00535.x.
Fisher EB, Brownson CA, O'Toole ML, Shetty G, Anwuri VV, Glasgow RE: Ecological approaches to self-management: the case of diabetes. Am J Public Health. 2005, 95 (9): 1523-1535. 10.2105/AJPH.2005.066084.
Institute of Medicine. Committee on Quality of Health Care in America: Crossing the quality chasm: A new health system for the 21st century. 2001, Washington, D.C.: National Academy Press
Naik AD, Palmer N, Petersen NJ, Street RL, Rao R, Suarez-Almazor M, Haidet P: Comparative effectiveness of goal setting in diabetes mellitus group clinics: randomized clinical trial. Arch Intern Med. 2011, 171 (5): 453-459. 10.1001/archinternmed.2011.70.
Sarkar U, Piette JD, Gonzales R, Lessler D, Chew LD, Reilly B, Johnson J, Brunt M, Huang J, Regenstein M, Schillinger D: Preferences for self-management support: findings from a survey of diabetes patients in safety-net health systems. Patient Educ Couns. 2008, 70 (1): 102-110. 10.1016/j.pec.2007.09.008.
Regenstein M, Huang J, Cummings L, Lessler D, Reilly B, Schillinger D: Caring for patients with diabetes in safety net hospitals and health systems (Publication No.826). 2005, New York: The Commonwealth Fund
Eakin EG, Bull SS, Glasgow RE, Mason M: Reaching those most in need: a review of diabetes self-management interventions in disadvantaged populations. Diabetes Metab Res Rev. 2002, 18 (1): 26-35. 10.1002/dmrr.266.
Fisher EB, Brownson CA, O'Toole ML, Shetty G, Anwuri VV, Glasgow RE: Ecological approaches to self-management: the case of diabetes. Am J Public Health. 2005, 95 (9): 1523-1535. 10.2105/AJPH.2005.066084.
Wagner EH, Austin BT, Von Korff M: Improving outcomes in chronic illness. Manag Care Q. 1996, 4 (2): 12-25.
Fiscella K, Geiger HJ: Health information technology and quality improvement for community health centers. Health Aff (Millwood). 2006, 25 (2): 405-412. 10.1377/hlthaff.25.2.405.
Institute of Medicine: Health literacy: A prescription to end confusion. 2004, Washington, D.C.: National Academies Press
Yach D, Hawkes C, Gould CL, Hofman KJ: The global burden of chronic diseases: overcoming impediments to prevention and control. JAMA. 2004, 291 (21): 2616-2622. 10.1001/jama.291.21.2616.
Varkey AB, Manwell LB, Williams ES, Ibrahim SA, Brown RL, Bobula JA, Horner-Ibler BA, Schwartz MD, Konrad TR, Wiltshire JC, Linzer M, MEMO Investigators: Separate and unequal: clinics where minority and nonminority patients receive primary care. Arch Intern Med. 2009, 169 (3): 243-250. 10.1001/archinternmed.2008.559.
Glasgow RE, Davis CL, Funnell MM, Beck A: Implementing practical interventions to support chronic illness self-management. Jt Comm J Qual Saf. 2003, 29 (11): 563-574.
Beach MC, Gary TL, Price EG, Robinson K, Gozu A, Palacio A, Smarth C, Jenckes M, Feuerstein C, Bass EB, Powe NR, Cooper LA: Improving health care quality for racial/ethnic minorities: a systematic review of the best evidence regarding provider and organization interventions. BMC Public Health. 2006, 6: 104-10.1186/1471-2458-6-104.
Chin MH, Cook S, Jin L, Drum ML, Harrison JF, Koppert J, Thiel F, Harrand AG, Schaefer CT, Takashima HT, Chiu SC: Barriers to providing diabetes care in community health centers. Diabetes Care. 2001, 24 (2): 268-274. 10.2337/diacare.24.2.268.
Bonell CP, Hargreaves J, Cousens S, Ross D, Hayes R, Petticrew M, Kirkwood BR: Alternatives to randomisation in the evaluation of public health interventions: design challenges and solutions. J Epidemiol Community Health. 2011, 65 (7): 582-587. 10.1136/jech.2008.082602.
Tunis SR, Stryer DB, Clancy CM: Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003, 290 (12): 1624-1632. 10.1001/jama.290.12.1624.
Green LW: Making research relevant: if it is an evidence-based practice, where's the practice-based evidence?. Fam Pract. 2008, 25 (Suppl 1): i20-4.
Schillinger D, Hammer H, Wang F, Palacios J, McLean I, Tang A, Youmans S, Handley M: Seeing in 3-D: examining the reach of diabetes self-management support strategies in a public health care system. Health Educ Behav. 2008, 35 (5): 664-682.
Schillinger D, Handley M, Wang F, Hammer H: Effects of self-management support on structure, process, and outcomes among vulnerable patients with diabetes: a three-arm practical clinical trial. Diabetes Care. 2009, 32 (4): 559-566. 10.2337/dc08-0787.
Machtinger EL, Wang F, Chen LL, Rodriguez M, Wu S, Schillinger D: A visual medication schedule to improve anticoagulation control: a randomized, controlled trial. Jt Comm J Qual Patient Saf. 2007, 33 (10): 625-635.
Lorig K: Action planning: a call to action. J Am Board Fam Med. 2006, 19 (3): 324-325. 10.3122/jabfm.19.3.324.
U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion: National Action Plan to Improve Health Literacy. 2010, Washingtonk D.C.: U.S. Department of Health and Human Services
Handley MA, Shumway M, Schillinger D: Cost-effectiveness of automated telephone self-management support with nurse care management among patients with diabetes. Ann Fam Med. 2008, 6 (6): 512-518. 10.1370/afm.889.
Schmittdiel JA, Uratsu CS, Karter AJ, Heisler M, Subramanian U, Mangione CM, Selby JV: Why don't diabetes patients achieve recommended risk factor targets? Poor adherence versus lack of treatment intensification. J Gen Intern Med. 2008, 23 (5): 588-594. 10.1007/s11606-008-0554-8.
Heisler M, Hogan MM, Hofer TP, Schmittdiel JA, Pladevall M, Kerr EA: When more is not better: treatment intensification among hypertensive patients with poor medication adherence. Circulation. 2008, 117 (22): 2884-2892. 10.1161/CIRCULATIONAHA.107.724104.
Handley MA, Schillinger D, Shiboski S: Quasi-experimental designs in practice-based research settings: Design and implementation considerations. J Am Board Fam Med. 2011, 24 (5): 589-596. 10.3122/jabfm.2011.05.110067.
Brown CA, Lilford RJ: The stepped wedge trial design: A systematic review. BMC Med Res Methodol. 2006, 6: 54-10.1186/1471-2288-6-54.
Hussey MA, Hughes JP: Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007, 28 (2): 182-191. 10.1016/j.cct.2006.05.007.
American Diabetes Association: Standards of medical care in diabetes--2009. Diabetes Care. 2009, 32 (Suppl 1): S13-61.
American Diabetes Association: Standards of medical care in diabetes--2010. Diabetes Care. 2010, 33 (Suppl 1): S11-61.
Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Intern Med. 2006, 21 (Suppl 2): S14-20.
Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: An exploration of how guideline developer capacity and guideline implementability influence implementation and adoption: study protocol. Implement Sci. 2009, 4: 36-10.1186/1748-5908-4-36.
National Committee for Quality Assurance: The State of Health Care Quality 2008. 2008, Washington, D.C.: National Committee for Quality Assurance
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, CONSORT Group: Methods and processes of the CONSORT Group: example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med. 2008, 148 (4): W60-6.
Sarkar U, Schillinger D, Lopez A, Sudore R: Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations. J Gen Intern Med. 2010, 26 (3): 265-271.
Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, Bradley KA, Nugent SM, Baines AD, Vanryn M: Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008, 23 (5): 561-566. 10.1007/s11606-008-0520-5.
Ware J, Kosinski M, Keller SD: A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34 (3): 220-233. 10.1097/00005650-199603000-00003.
Toobert DJ, Hampson SE, Glasgow RE: The Summary of Diabetes Self-Care Activities measure: results from 7 studies and a revised scale. Diabetes Care. 2000, 23 (7): 943-950. 10.2337/diacare.23.7.943.
Heisler M, Bouknight RR, Hayward RA, Smith DM, Kerr EA: The relative importance of physician communication, participatory decision making, and patient understanding in diabetes self-management. J Gen Intern Med. 2002, 17 (4): 243-252. 10.1046/j.1525-1497.2002.10905.x.
Fleming BB, Greenfield S, Engelgau MM, Pogach LM, Clauser SB, Parrott MA: The Diabetes Quality Improvement Project: moving science into health policy to gain an edge on the diabetes epidemic. Diabetes Care. 2001, 24 (10): 1815-1820. 10.2337/diacare.24.10.1815.
Glasgow RE, Whitesides H, Nelson CC, King DK: Use of the Patient Assessment of Chronic Illness Care (PACIC) with diabetic patients: Relationship to patient characteristics, receipt of care, and self-management. Diabetes Care. 2005, 28 (11): 2655-2661. 10.2337/diacare.28.11.2655.
Stewart AL, Napoles-Springer AM, Gregorich SE, Santoyo-Olsson J: Interpersonal processes of care survey: patient-reported measures for diverse groups. Health Serv Res. 2007, 42 (3 Pt 1): 1235-1256.
Shojania KG, Ranji SR, McDonald KM, Grimshaw JM, Sundaram V, Rushakoff RJ, Owens DK: Effects of quality improvement strategies for type 2 diabetes on glycemic control: A meta-regression analysis. JAMA. 2006, 296 (4): 427-440. 10.1001/jama.296.4.427.
Jha AK: Meaningful use of electronic health records: The road ahead. JAMA. 2010, 304 (15): 1709-1710. 10.1001/jama.2010.1497.
Hsiao CJ, Hing E, Socey TC, Cai B: Electronic medical record/electronic health record systems of office-based physicians: United States, 2009 and preliminary 2010 state estimates. 2010, Atlanta, GA: CDC National Center of Health Statistics
Krist AH, Woolf SH: A vision for patient-centered health information systems. JAMA. 2011, 305 (3): 300-301. 10.1001/jama.2010.2011.
Bindman AB, Chattopadhyay A, Osmond DH, Huen W, Bacchetti P: The impact of Medicaid managed care on hospitalizations for ambulatory care sensitive conditions. Health Serv Res. 2005, 40 (1): 19-38. 10.1111/j.1475-6773.2005.00340.x.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/12/22/prepub
Acknowledgements and Funding
The study is funded by the Agency for Healthcare Research and Quality grants R18HS017261 and 1R03HS020684-01; Centers for Disease Control and Prevention grant #5U58DP002007-03; National Institute of Diabetes and Digestive and Kidney Diseases 1P30DK092924-01 for The HMO Research Network-University of California, San Francisco Center for Diabetes Translational Research (CDTR); and the McKesson Foundation. No funders had any role in the study design; collection, analysis, and interpretation of data; writing of the manuscript; or decision to submit the manuscript for publication.
The salaries of the investigators (MH, JQ, and MR) working on this study were partially funded by the McKesson Foundation. No funders had any role in the study design; collection, analysis, and interpretation of data; writing of the manuscript; or decision to submit the manuscript for publication.
NR participated in the design and implementation of the study and drafted the manuscript. MH participated in the design and implementation of the study and helped to draft the manuscript. JQ participated in the implementation of the study, performed statistical analyses, and helped to draft the manuscript. US participated in the design and implementation of the study and helped to draft the manuscript. CS participated in the design and implementation of the study. KP participated in the design and implementation of the study. DS conceived of the study, oversaw the design and implementation of study, and helped to draft the manuscript. All authors read and approved the final manuscript.
Electronic supplementary material
Additional file 1: SMARTSteps-PLUS medication intensification scripts. This document offers an example of SMARTSteps-PLUS medication intensification scripts for health coach phone calls in a quasi-experimental evaluation trial of a language-concordant automated telephone diabetes self-management health plan intervention. (DOC 74 KB)
About this article
Cite this article
Ratanawongsa, N., Handley, M.A., Quan, J. et al. Quasi-experimental trial of diabetes Self-Management Automated and Real-Time Telephonic Support (SMARTSteps) in a Medicaid managed care plan: study protocol. BMC Health Serv Res 12, 22 (2012) doi:10.1186/1472-6963-12-22
- Health Information Technology
- Limited English Proficiency
- Pharmacy Claim
- Health Coach
- Step Wedge Design