- Research article
- Open Access
- Open Peer Review
Developing a dashboard to help measure and achieve the triple aim: a population-based cohort study
BMC Health Services Researchvolume 14, Article number: 363 (2014)
Health system planners aim to pursue the three goals of Triple Aim: 1) reduce health care costs; 2) improve population health; and 3) improve the care experience. Moreover, they also need measures that can reliably predict future health care needs in order to manage effectively the health system performance. Yet few measures exist to assess Triple Aim and predict future needs at a health system level. The purpose of this study is to explore the novel application of a case-mix adjustment method in order to measure and help improve the Triple Aim of health system performance.
We applied a case-mix adjustment method to a population-based analysis to assess its usefulness as a measure of health system performance and Triple Aim. The study design was a retrospective, cohort study of adults from Ontario, Canada using administrative databases: individuals were assigned a predicted illness burden score using a case-mix adjustment system from diagnoses and health utilization data in 2008, and then followed forward to assess the actual health care utilization and costs in the following year (2009). We applied the Johns Hopkins Adjusted Clinical Group (ACG) Case-Mix System to categorize individuals into 60 levels of healthcare need, called ACGs. The outcomes were: 1) Number of individuals per ACG; 2) Total system costs per ACG; and 3) Mean cost per person per ACG, which together formed a health system “dashboard”.
We identified 11.4 million adults. 16.1% were aged 65 or older, 3.2 million (28%) did not use health care services that year, and 45,000 (0.4%) were in the highest acuity ACG category using 12 times more than an average adult. The sickest 1%, 5% and 15% of the population use about 10%, 30% and 50% of total health system costs respectively. The dashboard measures 2 dimensions of Triple Aim: 1) reduced costs: when total system costs per ACG or when average costs per person is reduced; and 2) improved population health: when more people move into healthier rather than sicker ACGs. It can help to achieve the third aim, improved care experience, when ACG utilization predictions are reported to providers to proactively develop care plans.
The dashboard, developed via case-mix methods, measures 2 of the Triple Aim goals and can help health system planners better manage their health delivery systems.
Health system planners and policymakers are responsible for managing health system performance, and thus review quality indicators as well as assessments of the current illness burden in the population so as to help predict the future need for health care services. They need to use existing data to help them first measure health system performance, and second improve the quality of the health system, such as where to implement particular quality improvement activities. The Institute of Healthcare Improvement, an organization that is a leader of quality improvement and performance measurement at the health system level [1–4], has espoused the seminal concept that high performing health systems ought to pursue the Triple Aim: to 1) reduce health care costs; 2) improve population health; and 3) improve the care experience . Within the U.K and U.S. especially, the Triple Aim has grown in popularity because new health care reforms, such as the NHS Outcomes Framework and the Affordable Care Act respectively, have broad goals of improving health system performance, such as improving quality, equity and access, but also include the three goals of the Triple Aim.
Within the past decade, many health system performance frameworks have been developed [6–23], comprised of various quality indicators, some of which relate to the Triple Aim goals. For example, quality indicators related to health system costs include health expenditures per capita , 30 day acute care readmissions , and preventable hospitalizations . Population health has been assessed through indicators such as life expectancy, disability-adjusted life year , obesity rates in children, and the percentage of adults who smoke . Care experience has been demonstrated by indicators such as the rate of adverse events, timely access to primary care, and wait times for surgery. On the one hand, the existing quality indicators have been useful to improve quality within particular populations because many of them are often disease-focused, setting-specific, or based on one episode of care. They have a well-defined denominator and thus a particular process of care or group of providers that can be held accountable to improve the rates. On the other hand, these measures are limited in other ways. They typically do not capture performance of the broader health care system, including care across different providers and settings throughout the continuum of care. As such current measures especially do not serve well patients with multiple comorbidities who represent the most expensive and medically-complex users of the system [24–26]. Thus it remains a challenge to measure the broader system performance, with respect to the Triple Aim.
Not only do planners want to measure Triple Aim and health system performance effectively, but want to do so in efficient and simple ways that can aid quality improvement. Health system planners and policymakers often use visual dashboards to integrate and compile important quality indicators and other key performance indicators into one place. An effective dashboard can help them to easily access and analyze important trends from the indicators, supporting timely decision-making and quality improvement. While other dashboards of health system performance exist, to our knowledge, no dashboards exist that measure the Triple Aim. Moreover, little research has explored how health system planners might use such as dashboard to aid quality improvement activities and help achieve the Triple Aim. The purpose of this study is to develop a visual dashboard, based on a case-mix adjustment method, and explore its applicability to measure and help improve health system performance, with respect to the Triple Aim.
We conducted a retrospective, population-based, cohort study of adults from Ontario, Canada using multiple administrative databases. To develop the dashboard generally, individuals were assigned a predicted illness burden score using a case-mix adjustment system from diagnoses and health utilization data occurring in fiscal year 2007–08, and then followed forward to assess the actual health care utilization and costs in the following year (fiscal year 2008–09).
Specifically, we identified a cohort of unique Ontarians with a valid Ontario Health Insurance Plan number, the universal provincial health insurance plan, who were alive on April 1, 2008 and deterministically linked with the other administrative databases [27–32]. We excluded children aged younger than 18 to simplify the analysis. Using the previous fiscal year’s administrative data (FY2007-08), we applied the Johns Hopkins Adjusted Clinical Group (ACG) Case-Mix System (version 7) in this cohort to categorize individuals into mutually exclusive levels of healthcare need, called ACGs. Then using the subsequent fiscal year’s administrative data (FY2008-09), we determined the actual health care utilization and costs for each individual during that year. In doing so, this methodology aims to demonstrate the value of the dashboard to predict future costs and utilization, based on previous utilization and illness burden.
In Ontario, Canada, because of the mainly single-payer nature of the health care financing system, the government collects a large amount of administrative health care data that can be used for research purposes. The provincial Ministry of Health and Long-Term Care provides funding for all Ontarians for hospital and physician services free at the point of care, with other services heavily subsidized. As such, Ontario’s Ministry of Health, in some ways, can be viewed as a large health insurer with approximately 13 M enrollees, and an annual budget of approximately $42B.
The Registered Persons Database was the source for valid health insurance number, age (at start of FY), and sex. The Discharge Abstract Database, maintained by the Canadian Institute of Health Information, contained data for all hospital in-patient admissions and length of stay. The National Ambulatory Care Reporting System contained data for all emergency department (ED) and hospital and clinic outpatient visits. Finally, the Ontario Health Insurance Plan Claims database contained information on physician claims and ancillary services (e.g. laboratory work, blood tests).
The ACG case-mix system
The Johns Hopkins ACG system is a diagnosis-based, case-mix method that uses administrative data to predict a population’s current and future healthcare need, healthcare use, and costs in one to two years. It captures the clinical complexity of multiple comorbidities without being disease-specific. ACGs have traditionally been used for risk-adjusted reimbursement and practitioner profiling in the US [33–37], Canada, and internationally [29, 38–44]. The method uses each person’s inpatient and outpatient diagnosis codes, age, and sex to assign individuals into 32 Ambulatory Diagnosis Groups (ADGs) — the building blocks of the ACGs. ADGs are unique in that they do not group by disease, but instead based on clinical similarities in need and expected resource use. Because people can have multiple chronic conditions, ADGs are not mutually exclusive. For example, ADG1 represents “time limited: minor conditions” (e.g. bell’s palsy or diaper rash), whereas ADG24 represents “injuries/adverse effects: major” (e.g. lower limb fractures or intracranial injury). Second, the methodology groups ADGs into 60 commonly occurring combinations, which are mutually exclusive ACG groups. Examples of ACG groups range from relatively small needs, such as “acute minor” (e.g. ear infection or cold), to the most expensive, “10+ ADG combinations, 4+ major ADGs” (e.g. multiple, unstable, chronic conditions with acute complications) .
At the individual-level, based on FY2007-08 administrative data, every adult was categorized into an ACG, and their subsequent year’s health care utilization and costs from FY2008-09 were determined. Our results were aggregated into sub-populations of ACG categories, and the all 60 ACGs collectively represent all adults in the province. Specifically, we analyzed and presented our data to create 3 figures, which we refer to as a health system “dashboard”. The dashboard’s 3 figures describe the:
Number of individuals in each ACG;
Total system costs for each ACG; and
Mean cost per person for each ACG.
We applied a costing methodology that has been validated previously that includes health utilization costs of only hospitalizations, ED visits, physician claims and ancillary services claims that were publicly financed by the Ontario government [45–47]. Thus, the dashboard represents the past illness burden, utilization, health system costs, as well as the predicted health care use of the entire population profile or of any particular ACG category. We created a dashboard for the overall province.
Besides ACG categories, health care need was also quantified using ACG utilization weights, calibrated to Ontario data . A utilization weight for each ACG represents the multiplier of expected resource use relative to the population’s mean expenditures. By definition, the average ACG weight was 1.0 for the entire population. Thus, an ACG weight of 2.0 represents a two-fold increase in expected healthcare utilization in the subsequent year. Note the dashboard is sorted from lowest to highest ACG weight, which corresponds to lowest to highest predicted healthcare need. Furthermore, we sorted our ACG categories into resource utilization quintiles based on the ACG weights. Higher quintiles were associated with higher expected utilization levels. All individuals in a given quintile were expected to use approximately the same amount of care. Non-users were excluded from the quintiles and assigned a zero value. This study was reviewed by the ethics committee of the Sunnybrook Health Science Centre and deemed exempt research, as it uses de-identified secondary data analysis.
Our provincial analysis of Ontario in FY2008-2009 identified 11.4 million adults. Nearly half of the cohort was female and 16.1% were aged 65 or older. (Table 1) Approximately 3.2 M (28%) did not use the healthcare system or had services with unclassified diagnoses (ACG = 5100); their average ACG weight was 0.17. Conversely, 45,383 adults (0.4%) were categorized into the highest average ACG weight of 12.61 (ACG = 5070). Figures 1A-1C comprise the provincial dashboard. Figure 1A displays the distribution of number of adults who are categorized in the various ACG categories. The second largest ACG category is adults older than 34 who are expected to have acute minor issue(s) (e.g. ear infections, sore throats, etc.), comprising 10.9% of the population (ACG = 4100). In the lower 4 resource utilization quintiles, the ACG weight is less than twice (2.00) the average adult.Figure 1B displays the total provincial health system costs in Canadian dollars for all adults within each of the ACG categories, totalling $15.8B of direct costs from hospital, fee-for-service physician and ancillary service billing, and ED services. Based on ACG weights, the sickest 1%, 5% and 15% of the adult population will use about 10%, 30% and 50% of total health system costs respectively. The largest 5 ACG categories with the most aggregate costs to the health system are: acute minor issues for adults >34 (ACG = 4100); 0–1 major ADG, age >34, and 6–9 other ADG combinations (ACG = 4910); 2 major ADG, age >34, and 6–9 other ADG combinations (ACG = 4920); 1 major ADG, age >44, and 4–5 other ADG combinations (ACG = 4420); and the 4+ major ADGs, age > 18, and 10+ other ADG combinations (ACG = 5070). Health system costs directly relate to both the number of adults within each group and the severity of the adults’ conditions. Hospital services comprise the largest percentage of cost within each ACG, followed by physician services. ED costs comprise a very small fraction of total costs within each ACG.Figure 1C displays the average cost per category (weight = 12.6) was $17,271. The average yearly cost per adult (ACG weight of 1) was $1,382. Similar to total system costs, hospital services comprise the largest percentage of costs.
Our study applied a case-mix and costing method to a large population-based sample to analyze the predicted health care needs, and then follow that population to calculate the actual utilization costs. This analysis was summarized as a health system dashboard of the sickest and highest cost users in the system, which acts as one measure of health system performance. Unsurprisingly, the sickest and most expensive adults had multiple major illnesses and multiple combinations of comorbidities. Similar to the US, a small percentage of our Ontarian study population (15%) use a disproportionate amount of the total health care system costs (48%).
The dashboard can help to partially measure the Triple Aims
We propose that the dashboard can measures 2 of the 3 dimensions of the Triple Aim, namely health system costs and population health. (Figure 2) First, the dashboard indicates lower system costs over time when the cost curve in Figure 2C shifts to the left (so average health costs/person are less) or when any ACG category in Figure 2B is shortened (so total system costs for an ACG category is reduced). Second, the dashboard indicates a healthier population over time when the healthcare utilization curve in Figure 2A shifts upwards (more people move upwards into ACG categories with less healthcare use) rather than downwards (more healthcare use). While past and predicted healthcare utilization is a proxy for population health, we recognize it does not truly capture all aspects of health, such as quality of life. Nonetheless, measurement of the Triple Aim, even partially, is important.
The dashboard can help policymakers to achieve all of the Triple Aims
In addition to measurement, we also propose that the dashboard’s information can support health system planners to manage their health delivery systems more effectively. The dashboard represents a profile of the illness burden and healthcare needs of a population at a given time, and is thus a useful tool to predict the need for future healthcare services. By using this data, planners and providers can proactively intervene with appropriate policy, programs, and clinical and social interventions for the highest risk patients or the patients who are most likely to benefit so that ideally the same population is healthier the next time the dashboard is measured. Thus, the dashboard would need to be reported regularly (e.g. quarterly or biannually) in a continual feedback loop to indicate changes over time. Using the dashboard in this way would be supported by evidence that show the effectiveness health system measures depends on the extent that it reflects the goals of the health system, the quality of the data, and the incentives for providers to scrutinize and act on the data . Below, we provide an example of how the dashboard could be used at various levels to support the achievement of the Triple Aim.
Reduce costs: First, to achieve lower system costs and move the cost curve left, (See Figure 2C) policymakers and providers could use the dashboard to support cost-effective improvements in care (e.g. new technologies, diagnostic equipment, or surgical techniques), such as in ACGs with many individuals even if they are not the most complex cases. The dashboard might also identify medical provider specialties that ought to be high priority for better alignment with financial incentives that focus on patient outcomes or bundles of care, rather than number of visits, as in fee-for-service. Second, policymakers and providers can act to reduce costs within individual ACG categories. For highest-needs or highest-costs individuals, they could support interventions that provide patients with additional care support, intensive case management, or early referral to palliative care as appropriate, all of which can avoid unnecessary hospitalizations and ED visits. For medium- and low-risk individuals, they could support interventions that maintain or improve health, such as self-care, healthy nutrition, exercise, and wellness programs, which can reduce costs within an ACG category. (See Figure 2B)
Improve population health: The dashboard can present data to providers at an individual or group practice level about their patient rosters to determine if their roster is getting healthier over time (See Figure 2A). Providers and policymakers could use the information to focus on particular health conditions, regions, or populations that could benefit from targeted interventions, and the size and scale required for such interventions. For the sickest population, they could refer them to interventions or supports that prevent the worsening of their condition (e.g. falls prevention), or help them manage complex conditions (e.g. intensive case management or nurse coaching). Just as importantly, for the medium and low-risk population, providers could encourage interventions that promote prevention and wellness, improve health, and reverse the disease progression, such as self management and exercise programs.
Improve patient care experience: The dashboard does not measure care experience directly. However, the dashboard’s information potentially can help providers improve care experience and overall health. For instance, the dashboard could report to providers the high-risk, medically complex patients they are the most responsible physician for. The providers could then work proactively, rather than reactively, to intervene and develop multidisciplinary care plans for higher need individuals to prevent worsening of the condition(s) rather than wait for them to arrive in the hospital or their office with an issue. Thus the dashboard can support integration with the broader health system and health care team to improve the care experience for the patient.
The dashboard marries both an economic analysis with a case-mix adjustment analysis to help user predict future costs in relation to predicted illness burden and health care needs. When the dashboard is compared over multiple time periods, it can provide a dynamic tool to help measure population health and health system performance. A key strength of the dashboard is that it can be adapted to profile multiple populations: it could be presented at a macro level (e.g. NHS regional teams or state Medicaid insurance program) for high-level planning and resource allocation, but could also be useful at a micro level (e.g. small health region, community, or small group practice) where sample sizes are small enough for health care providers to assume responsibility for patients in an ACG. Moreover, the dashboard can be adapted to present data by age (e.g. children), by disease or condition (e.g. cancer or heart disease), by geography, income, etc. Thus disease specific organizations, or community programs that focus on particular ages or needs, can also use a similar methodology to target high risk patients, as well as track impact and improvement on the population’s health. The dashboard relies on case-mix methods, which have the advantage of typically relying on existing administrative data already available. Moreover, large administrative data can produce population-based profiles, connect across multiple sectors, and compare across regions to allow for a broader health system assessment. Finally, the dashboard is not constrained by the ACG case-mix method. Other case-mix methodologies could be applied in a similar way to population-based samples.
The dashboard could be useful to policymakers, health system planners, and providers for measuring health system performance, especially in countries with largely single payer health systems, such as in Canada, UK, or other European Union countries. Many federally and regionally funded programs could report this dashboard indicator publically and regularly in order to manage region-wide initiatives and to facilitate quality improvement. In the US, the dashboard would be especially useful in “closed” group model systems or capitated managed care organizations, where they provide comprehensive care team is within the same health delivery system, so care information such as medical records can be shared easily. It would also be useful in open group models, because the dashboard is not setting specific and can support providers to better coordinate and integrate care across settings and providers.
While the dashboard might be unique and useful, it needs further development in applicability and accountability. We did not explore the ACG categories by provider practice group, by region, by children versus adult, or by disease type, though have planned other subsequent research to do this. The costing methodology could be more comprehensive and inclusive, and does not include privately obtained services. Cost and utilization by ACG does not determine quality of care. Case-mix measures are dependent on observed morbidity and provider diagnostic coding, which may be biased by unobserved morbidity and coding behaviours . Moreover, we have yet to explore the predictive nature of the dashboard for different outcomes from one year to the next. This paper showed a snapshot of one year, and did not explore the true dynamic nature of the dashboard over multiple time periods. Prior research has shown the ACGs alone are predictive of 40% of same year’s and 14% of next year’s total costs . Other versions of the ACG system include prescription drugs when assigning categories, which might improve predictive ability, though this was not done in this study. Further research that combines ACG data with other administrative data, such as clinical, health behavior, drug data, or survey data about quality of care, may increase its predictive ability and should be explored.
The application of case-mix methodology to administrative, population based data is not novel. In the public domain, many research studies applied ACGs to various populations for the purposes of case-mix adjustment and reimbursement, practice variation, provider profiling, and the management of disease populations [33–37, 50]. However, these studies do not aim to indicate health system performance with respect to the Triple Aim. In the proprietary domain, many managed care organizations and health insurers in the US, responsible for healthcare delivery to large populations, have likely adapted various case-mix methods to predict future expenditures and to measure health system performance, at least in the populations they serve. However, to our knowledge, the results of such analyses, especially the costing data, have not been made publicly available due to proprietary nature of the data. We believe this dashboard is a unique contribution in that it applies a standard case-mix approach—commonly used for confidential reimbursement, insurance rate setting, and practitioner profiling—in a way that can be used by health system planners to measure health system performance and inform quality improvement activities, whether organizations choose to make the data public or not. To measure more dimensions of the Triple Aim and health system performance more broadly, the dashboard should be integrated and used in combination with other quality measures.
In summary, the health system performance dashboard, developed via case-mix methods, can help policymakers, planners, and providers better manage their health delivery systems because it measures two of three of the Triple Aim goals at a system level. Moreover, when fed back appropriately to providers and policymakers, the health system performance dashboard has the potential to support appropriate resource allocation and related quality improvement in the system and ultimately help to achieve the Triple Aim.
Adjusted Clinical group
Ambulatory Diagnostic Grouper
Berwick DM, James B, Coye MJ: Connections between quality measurement and improvement. Med Care. 2003, 41: I30-I38.
Martin L, Nelson E, Lloyd R, Nolan T: Whole system measures. 2007, Massachusetts: Institute for Healthcare Improvement. IHH Innovation Series White Paper, Ref Type: Report
Leape LL, Kabcenell AI, Gandhi TK, Carver P, Nolan TW, Berwick DM: Reducing adverse drug events: lessons from a breakthrough series collaborative. Jt Comm J Qual Improv. 2000, 26: 321-331.
Wilson T, Berwick DM, Cleary PD: What do collaborative improvement projects do? Experience from seven countries. Jt Comm J Qual Saf. 2003, 29: 85-93.
Berwick DM, Nolan TW, Whittington J: The triple aim: care, health, and cost. Health Aff (Millwood). 2008, 27: 759-769. 10.1377/hlthaff.27.3.759.
Performance measurement for health system improvement: Experiences, challenges, and Prospects. 2009, New York: Cambridge University Press
Arah OA, Klazinga NS, Delnoij DM, ten Asbroek AH, Custers T: Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. Int J Qual Health Care. 2003, 15: 377-398. 10.1093/intqhc/mzg049.
Brien SE, Dixon E, Ghali WA: Measuring and reporting on quality in health care: a framework and road map for improving care. J Surg Oncol. 2009, 99: 462-466. 10.1002/jso.21188.
Etches V, Frank J, Di RE, Manuel D: Measuring population health: a review of indicators. Annu Rev Public Health. 2006, 27: 29-55. 10.1146/annurev.publhealth.27.021405.102141.
Evans DB, Edejer TT, Lauer J, Frenk J, Murray CJ: Measuring quality: from the system to the provider. Int J Qual Health Care. 2001, 13: 439-446. 10.1093/intqhc/13.6.439.
Galvin RS, McGlynn EA: Using performance measurement to drive improvement: a road map for change. Med Care. 2003, 41: I48-I60.
Handler A, Issel M, Turnock B: A conceptual framework to measure performance of the public health system. Am J Public Health. 2001, 91: 1235-1239. 10.2105/AJPH.91.8.1235.
Mulligan J, Appleby J, Harrison A: Measuring the performance of health systems. BMJ. 2000, 321: 191-192. 10.1136/bmj.321.7255.191.
Arah OA, Westert GP, Hurst J, Klazinga NS: A conceptual framework for the OECD Health Care Quality Indicators Project. Int J Qual Health Care. 2006, 18 (Suppl 1): 5-13.
Jencks SF, Cuerdon T, Burwen DR, Fleming B, Houck PM, Kussmaul AE, Nilasena DS, Ordin DL, Arday DR: Quality of medical care delivered to Medicare beneficiaries: A profile at state and national levels. JAMA. 2000, 284: 1670-1676. 10.1001/jama.284.13.1670.
McGlynn EA: Selecting common measures of quality and system performance. Med Care. 2003, 41: I39-I47.
OECD: Measuring up: Improving health system performance in OECD countries. 2002
Schoen C, Davis K, How SK, Schoenbaum SC: U.S. health system performance: a national scorecard. Health Aff (Millwood). 2006, 25: w457-w475. 10.1377/hlthaff.25.w457.
WHO: The WHO report 2000 - Health systems: Improving performance. 2000, Geneva, Switzerland: WHO, Ref Type: Report
WHO: Summary measures of population health. Concepts, ethics, measurements and applicatrions. 2009, Geneva, Switzerland: WHO, Ref Type: Report
Canadian Institute for Health Information: Health Indicators 2012. 2012, Ottawa, Ontario: CIHI, Ref Type: Report
Commonwealth Fund: Why not the best? Results from the national scorecard on U.S. health system performance. Edited by: McCarthy D, How SKH, Fryer A, Radley DC, Schoen C. 2011, Ref Type: Report
Institute of Medicine CoQoHCiA: Crossing the quality chasm: A new health system for the 21st Century. Edited by: Corrigan J, Donaldson MS, Kohn LT, Maguire SK, Pike KC. 2001, Washington, DC: National Academies Press, Institute of Medicine, Ref Type: Report
Reuben DB, Tinetti ME: Goal-oriented patient care–an alternative health outcomes paradigm. N Engl J Med. 2012, 366: 777-779. 10.1056/NEJMp1113631.
Tinetti ME, Fried TR, Boyd CM: Designing health care for the most common chronic condition–multimorbidity. JAMA. 2012, 307: 2493-2494.
Wolff JL, Starfield B, Anderson G: Prevalence, expenditures, and complications of multiple chronic conditions in the elderly. Arch Intern Med. 2002, 162: 2269-2276. 10.1001/archinte.162.20.2269.
Glazier RH, Agha MM, Moineddin R, Sibley LM: Universal health insurance and equity in primary care and specialist office visits: a population-based study. Ann Fam Med. 2009, 7: 396-405. 10.1370/afm.994.
Glazier RH, Klein-Geltink J, Kopp A, Sibley LM: Capitation and enhanced fee-for-service models for primary care reform: a population-based evaluation. CMAJ. 2009, 180: E72-E81. 10.1503/cmaj.081316.
Hollander MJ, Kadlec H, Hamdi R, Tessaro A: Increasing value for money in the Canadian healthcare system: new findings on the contribution of primary care services. Healthc Q. 2009, 12: 32-44. 10.12927/hcq.2013.20877.
Sibley LM, Moineddin R, Agha MM, Glazier RH: Risk adjustment using administrative data-based and survey-derived methods for explaining physician utilization. Med Care. 2010, 48: 175-182. 10.1097/MLR.0b013e3181c16102.
Sibley LM, Glazier RH: Evaluation of the equity of age-sex adjusted primary care capitation payments in Ontario, Canada. Health Policy. 2012, 104: 186-192. 10.1016/j.healthpol.2011.10.008.
Smith PM, Glazier RH, Sibley LM: The predictors of self-rated health and the relationship between self-rated health and health service needs are similar across socioeconomic groups in Canada. J Clin Epidemiol. 2010, 63: 412-421. 10.1016/j.jclinepi.2009.08.015.
Parente ST, Weiner JP, Garnick DW, Fowles J, Lawthers AG, Palmer RH: Profiling resource use by primary-care practices: managed Medicare implications. Health Care Financ Rev. 1996, 17: 23-42.
Powe NR, Weiner JP, Starfield B, Stuart M, Baker A, Steinwachs DM: Systemwide provider performance in a Medicaid program. Profiling the care of patients with chronic illnesses. Med Care. 1996, 34: 798-810. 10.1097/00005650-199608000-00007.
Salem-Schatz S, Moore G, Rucker M, Pearson SD: The case for case-mix adjustment in practice profiling, When good apples look bad. JAMA. 1994, 272: 871-874. 10.1001/jama.1994.03520110051028.
Stuart ME, Steinwachs DM: Patient-mix differences among ambulatory providers and their effects on utilization and payments for Maryland Medicaid users. Med Care. 1993, 31: 1119-1137. 10.1097/00005650-199312000-00005.
Weiner JP, Starfield BH, Powe NR, Stuart ME, Steinwachs DM: Ambulatory care practice variation within a Medicaid program. Health Serv Res. 1996, 30: 751-770.
Borkan J, Eaton CB, Novillo-Ortiz D, Rivero CP, Jadad AR: Renewing primary care: lessons learned from the Spanish health care system. Health Aff (Millwood). 2010, 29: 1432-1441. 10.1377/hlthaff.2010.0023.
Chang HY, Clark JM, Weiner JP: Morbidity trajectories as predictors of utilization: multi-year disease patterns in Taiwan's national health insurance program. Med Care. 2011, 49: 918-923. 10.1097/MLR.0b013e31821b344f.
Sicras-Mainar A, Velasco-Velasco S, Navarro-Artieda R, Prados-Torres A, Bolibar-Ribas B, Violan-Fors C: Adaptive capacity of the Adjusted Clinical Groups Case-Mix System to the cost of primary healthcare in Catalonia (Spain): a observational study. BMJ Open. 2012, 2 (3): doi:10.1136/bmjopen-2012-000941
Hanley GE, Morgan S, Reid RJ: Explaining prescription drug use and expenditures using the adjusted clinical groups case-mix system in the population of British Columbia, Canada. Med Care. 2010, 48: 402-408. 10.1097/MLR.0b013e3181ca3d5d.
Reid RJ, MacWilliam L, Verhulst L, Roos N, Atkinson M: Performance of the ACG case-mix system in two Canadian provinces. Med Care. 2001, 39: 86-99. 10.1097/00005650-200101000-00010.
Reid RJ, Roos NP, MacWilliam L, Frohlich N, Black C: Assessing population health care need using a claims-based ACG morbidity measure: a validation analysis in the Province of Manitoba. Health Serv Res. 2002, 37: 1345-1364. 10.1111/1475-6773.01029.
Reid RJ, Verhulst L, Forrest CB: Comparing apples with apples in clinical populations: applications of the adjusted Clinical Group System in British Columbia. Healthc Manage Forum. 2002, 15: 11-16. 10.1016/S0840-4704(10)60575-3.
Canadian Coordinating Office for Health Technology Assessment: Guidance document for the costing process, version 1.0. 1–24. 1996, Ottawa, ON: Canadian Agency for Drugs and Technologies in Health (CADTH), Ref Type: Report
Canadian Institute of Health Information: Resource Intensity Weights. Summary of Methodology 1995/96. 1–75. 1995, Ottawa, ON: Author, Ref Type: Report
Pink GH, Bolley HB: Physicians in health care management: 3, Case Mix Groups and Resource Intensity Weights: An overview for physicians. CMAJ. 1994, 150: 889-894.
Smith PC: Measuring health system performance. Eur J Health Econ. 2002, 3: 145-148. 10.1007/s10198-002-0138-1.
Kristensen T, Olsen KR, Schroll H, Thomsen JL, Halling A: Association between fee-for-service expenditures and morbidity burden in primary care. Eur J Health Econ. 2014, 15: 599-610. 10.1007/s10198-013-0499-7.
Kristensen T, Rose OK, Sortso C, Ejersted C, Thomsen JL, Halling A: Resources allocation and health care needs in diabetes care in Danish GP clinics. Health Policy. 2013, 113: 206-215. 10.1016/j.healthpol.2013.09.006.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/14/363/prepub
This research was supported through a collaboration between the Ministry of Health and Long-Term Care, Health Quality Ontario, and McMaster University’s Centre for Health Economics and Policy Analysis via the Applied Health Research Question program. We gratefully acknowledge Erik Hellsten and McMaster’s polinomicists for providing invaluable guidance on previous drafts.
The authors have no competing interests to declare.
Both authors were responsible for the conception and design of the study (HS and LS), data acquisition (HS and LS), data analysis (HS and LS), data interpretation (HS and LS), and the writing and review of the manuscript (HS and LS). Both authors have given final approval of the final manuscript and agree to be accountable for all aspects of the work.