Diabetes mellitus is among the leading chronic diseases in the US and internationally [1, 2]. Complications of diabetes can be reduced through appropriate medical care and behavior modification [3, 4]. Ensuring that patients with diabetes receive evidence based care to control disease and reduce the risk of complications is a significant challenge; much evidence exists of the gap between optimal care and the actual care patients receive [5, 6]. The chronic care model advocates an evidence based approach for disease management [7–9]. The model proposes several evidence based management practices organised in six interrelated elements supporting the implementation of evidence based medicine [10, 11].

It is essential to know which combinations of practices are most effective at improving diabetes care. Results from meta-analyses, randomized controlled trials, reviews and observational studies evaluating the impact of management practices on diabetes care are conflicting, presenting divergent results; important issues regarding the chronic care model and the practices embedded within it remain unresolved [12–16].

A systematic review assessing the effect of management practices in diabetes care concluded that professional and organizational practices, individually and in combination, improved diabetes care processes [17]. Adding patient education and or improving the role of a nurse led to improved care outcomes. Few studies assess the effects of individual management practices when several practices are applied at the same time and analysed with multivariate statistical models [14–16, 18–21].

Kerr has recommended that conflicting results among observational studies of care management could be addressed via a study design in which specific practices are carefully defined and the degree of practice implementation is consistently measured across diverse settings and compared to the outcomes delivered. The study reported here implements that approach [22].

We studied care management practices in a large U.S. integrated health care delivery system that has been recognized as a relatively high quality provider. It provides comprehensive care to several million members, including more than 500,000 adults with diabetes. Within this organisation, decentralized operating units result in variability in the way components of the chronic care model are implemented and in the relative emphasis on each component. Implementation varies between centers and for individual components within centers; consequently, local population care programs differ across the system.

The organisation has invested in standardized measurement of numerous quality indicators to enable comparisons over time and across operating units. Internal reporting shows variations in performance on process and outcome measures. Variability in population care practices and performance, together with standardized measurements, provide a ripe opportunity for learning about the impact of the chronic care model on diabetes care processes.

The purpose of this study was to identify management practices that affect glycemic and lipid monitoring in diabetes care in the context of several concurrent care management practices implemented at varying levels. We chose to use monitoring rates as our outcomes as they are solely a function of provider decision-making.

We interviewed one key informant at each of the 41 sites at the integrated health care delivery system. Most key informants were non-physician managers responsible for population based care or diabetes care; some were physician champions for diabetes care. Survey items elicited factual information rather than beliefs or opinions, and key informants were well positioned to provide the requested information.

Glycemic monitoring rate was defined as the percentage of members with diabetes who had one or more glycated hemoglobin (HbA1c) tests during one calendar year. Lipid monitoring rate was defined as the percentage of members with diabetes who had one or more low density lipoprotein-C (LDL-C) tests during one calendar year.

Person-level data on monitoring status were aggregated into strata defined by medical center (41 categories), age group (4 categories), gender, presence/absence of depression, and presence/absence of cardiovascular disease, and monitoring rates were calculated for all strata. Of the 1312 possible location-age-gender-comorbidity combinations, 1198 combinations were populated by one or more individuals. These 1198 strata were the units of observation for our analyses, with the outcome variables expressed as monitoring rates.

As we had only 41 medical centres and a large number of explanatory variables, we used stepwise regression analysis to obtain reasonably parsimonious models that were not overly parameterized. Care management practices were used as explanatory variables in a forward selection, stepwise logistic regression model, with random effects of medical centers and the observation level and dependent variables of glycemic and lipid monitoring rates. The medical center random effect captured excess variability between the 41 medical centers and played a major role in the statistical analysis in that it accounted for the medical center clustering effect of observations, hence providing the proper unbiased error estimates. The medical center random effect captured excess variability between the 41 medical centers and played a major role in the statistical analysis. The observation level random effect captured overdispersion in the data stemming from the fact that the effects of the independent variables and covariates do not completely explain the excess variation in outcomes [26]. The random effects associated with medical centres and observation levels were similar in both models, with rather high and comparable dispersion.

Each outcome was modeled separately, assessing the degree of association between various practices and the specific process measure. The models were fitted and compared by maximum likelihood methods using the "lmer" function of R. Even though we had a large number of explanatory variables and relatively few medical centers, our analysis did not demonstrate multicollinearity to an extent that would have made our models unstable.

The 41 practice sites provided care for 553,556 adults with diabetes, 51% of which were male. Among patients with diabetes, 16% were also diagnosed with coronary artery disease (CAD) and 13% with depression. Four percent of patients with diabetes were 18 to 34 years of age, 18% were 35 to 49 years old, 38% were 50 to 64 years old, and 40% were aged 65 years and up. The mean HbA1c was 7.2% (55 mmol/mol) and the mean LDL-C level was 105.1 mg/dl (2.72 mmol/L).

Two other practices affected monitoring rates. Guideline distribution and training increased the likelihood of glycemic monitoring; the odds ratio was 1.46 (p < 0.03). Action plans increased the likelihood of lipid monitoring; the odds ratio was 1.44 (p < 0.03).

In summary, the models for glycemic and lipid monitoring showed highly consistent effects; we interpret this finding as genuine because the implementation level of provider alerts differed across sites for these outcomes.

We assessed the individual effects of fifteen concurrently and variably implemented population care practices on two diabetes process of care measures. Provider alerts increased the likelihood of glycemic and lipid monitoring to a statistically significant degree. The effects of action plans and guideline distribution and training were significant at a level well below that of provider alerts.

Other studies support our finding that female gender is associated with lower quality of care for diabetes and/or cardiovascular disease [27–29]. Notably, we found that gender differences disappeared with increasing age. In our study, increasing age was markedly associated with higher monitoring rates, contradicting evidence that it is associated with poorer quality of care [30]. Overall, our data suggest that care might be more strongly affected by patient characteristics than by guideline recommendations.

Strengths of our study included the very large population managed at the care sites we studied and our ability to quantify the effect of practices taking implementation levels into account as recommended by Kerr [22]. Limitations to our study included the relatively narrow variation across sites; homogeneity of care management practices throughout the integrated health care delivery system stems from multiple factors, including clustering of sites within operationally consistent regions.

Other limitations are inherent in the observational study design, including the difficulty of measuring implementation levels and the large number of unmeasured factors and interactions that influence outcomes in real-world settings. Measuring implementation levels is a challenging endeavour about which relatively little has been written [14, 15]. More interviews with different health professionals with a variety of competencies and understandings of provided care might have improved the understanding of the implementation level of the management practices. However, because the interviews covered diverse aspects of care, many respondents gathered information from colleagues in preparation for the interview, so the single interviews typically integrate input from several individuals. We generally found that we could assess the degree to which structures and processes were in place to support each practice, but not their quality, highlighting the possibility of measurement error. To understand the practices in greater detail, we also conducted a series of case studies on diabetes care at four sites [31]. However, we were unable to complete a detailed assessment at all 41 sites.

Our results are best interpreted in the context of previous related studies. Provider alerts creating point-of-care reminders have been previously identified as an effective strategy for improving diabetes care [14, 16, 17, 32–34]. Varying effects of action plans and guideline distribution and training are shown in the literature [14, 16, 17, 35–38].

Another study using multivariate statistical modelling, The Translating Research into Action for Diabetes (TRIAD) study of approximately 9000 patients, examined the association between intensity of disease management by physicians and diabetes process and outcomes measures [25]. The TRIAD study identified an association between physician reminders and both lipid and glycemic monitoring rates: however, the same effect was noted for two other practices, structured care management and performance feedback, for which we did not find an association with monitoring rates.

In a multivariate study conducted in the U.S. Veterans Health Administration, four other practices affected a composite process measure of quality (monitoring tests and physical examinations): the level of support for guideline efforts, regional office leadership for guideline use, hospital use of guideline performance data, and hospital culture [16]. Other studies assessing the impact of management practices on diabetes quality of care and relying on multivariate statistical models have identified a variety of practices as important to quality [15, 16, 18–21]. Financial incentives have attracted much attention but did not have significant impact on the screening rates for HbA1c values nor for LDL-Cholesterol. Recent studies show mixed results of financial incentives regarding quality of care improvements [39–41].The physician financial incentives in place at the time of this study were small (less than 3% of total physician income). In sum, the literature is characterized by inconsistent findings regarding effective management practices in diabetes care.

Several factors may explain the lack of conclusive evidence. Comparability between studies is poor. Definitions of care management practices vary, and the Cochrane Collaboration suggests aligning these [32]. However, even across our study sites, we were not able to ensure consistency in the operational definitions of care management practices because they had been developed at the local level within the integrated health care delivery system over many years.

Another cause of conflicting results in the existing literature may be the effect of organisational and cultural contexts on the frontline delivery of diabetes care. These factors are very challenging to measure and analyze, and we did not attempt to do so.

Future research identifying effective management practices should build on the methods we used, including measurement of implementation levels and multivariate statistical models to assess the effects of multiple practices implemented in combination. Future efforts should also seek to align definitions of management practices, implementation levels, and outcome variables. The effects of patient-level covariates and organisational context should be taken into consideration.

The purpose of research attempting to identify specific practices most associated with better outcomes is to inform future quality improvement efforts; scarce resources would arguably be most efficiently allocated to those practices. This vein of research essentially attempts to "reverse engineer" critical performance drivers so they can be disseminated broadly or further amplified. However, conflicting results from observational care management studies suggest that this approach may not be the best way to identify or prioritize improvement opportunities; context dependencies and implementation details may preclude identifying performance drivers via observational studies. Quality improvement teams should consider other approaches. Two sound alternatives are based on "learning from improvement": using rapid-cycle improvement methods and/or identifying high-performing sites with similar organisational contexts from which structures and processes can be adopted [42].

The diabetes care management practice of provider alerts improved glycemic and lipid monitoring rates in patients with diabetes. Action plans and guideline distribution and training also affected monitoring rates, although to a lesser degree. The covariates of gender, age, CAD and depression also impacted monitoring rates to a statistically significant degree, individually and in combination.

Our findings contribute to a knowledge base that contains somewhat sparse and often conflicting results. Observational studies with uncontrolled variation on multiple factors may be inadequate for identifying the practices that could contribute most to ongoing improvement in population care, and quality improvement teams should also consider other methods for identifying improvement opportunities.

The study was reviewed and approved by the Institutional Review Boards for KP Northern California, KP Southern California, KP Northwest, KP Georgia, and KP Hawaii.