While pharmacy-based dispensing records are becoming increasingly popular tools for studying patterns of medication adherence, it is important to understand the potential limitations of the resulting measures. By their very definition, some measures of adherence are more prone to an upward bias, while other potentially more accurate measures may rely on a stronger set of assumptions to be valid. In addition, such pharmacy-based measures of adherence have face validity as measures of actual medication usage only when measured over long periods of time, since in this case high levels of estimated adherence can only be achieved through repeated refills (which tacitly implies ongoing medication usage).
Although our study was conducted in the context of a randomized clinical trial, we believe our findings have applicability to any pharmacoepidemiologic study that might be conducted using electronic dispensing records. One key consideration is that many of the commonly used measures require medication use during the observation window in order to be calculated. Our study suggests that this requirement will clearly bias adherence values upwards, both by excluding the least adherent patients and by building in a minimum level of adherence in those who can be assessed. Increasingly, investigators will have access to EMRs that can be used to better define the population of individuals who are known users of the medication of interest, or at least have been prescribed such medications, thus allowing more accurate assessments of adherence for a target population. Further, EMR data can be used to identify whether such medication use has been discontinued by the patient’s provider. In this context it is therefore important to understand how many individuals are excluded from your adherence estimates by various definitions, and our results show that this number can be quite large. In the context of comparative studies, where nonadherence can be differential across the groups being compared, it is even more important to account for all subjects.
Of the measures we studied, those that required an initial dispensing for their calculation, particularly if not bounded above by 1 (i.e., CMA1-2), were prone to exhibit an upward bias relative to the other measures we considered. The four variants on the Proportion of Days Covered, (CMA3-6), generally performed similarly to one another. Adherence estimates based on all six of these measures exhibited an increasingly upward bias with shorter observation windows that did not flatten out until after about 9 months of observation (Table 3). This reflects the fact that the measurement window for all of these measures begins with a dispensing, and hence a certain amount of built-in implied adherence. The fact that the measurement windows for CMA1, CMA3, and CMA5 also all end with a dispensing event leads to a further upward bias, since they effectively require that one is actively using the medication throughout the measurement window (or at least at both the beginning and end of the window). However, use of CMA2, CMA4, and CMA6 carry the implicit assumption that one should be using the medication throughout the entire interval beginning with the last dispensing event and extending through the end of the observation window (i.e., tk to te). If such an assumption is valid, then these measures should yield more unbiased estimates of adherence than those from their CMA1, CMA3, and CMA5 counterparts. They have the further added advantage that they can be computed for more people since they only require a single dispensing event to be defined. This is particularly important since the missingness pattern will likely not be random (that is, nonadherent individuals will be more likely than adherent individuals to have missing data). Thus the upward bias in CMA1, CMA3, and CMA5 results both from a likely overestimation of adherence in those for whom it can be calculated, as well as from the fact that these same individuals are likely to be selectively more adherent than those for whom these indices cannot be calculated.
If viewed merely as measures of medication acquisition, rather than medication taking, some of the problems discussed above become moot and CMA1 or CMA2 may be the preferred measures to use. Indeed there may be some theoretical interest in those with very high values for CMA1 or CMA2. However, values of CMA1 or CMA2 greater than 1 can also substantially skew mean rates of adherence upwards for a population and in many if not most instances may not reflect actual adherence behavior. To the best of our knowledge based on a limited amount of data checking, the majority of these extreme values reflect actual dispensings rather than, for example, administrative errors in the data. Whether they reflect excessive use by one individual, medication sharing, vacation supplies, or medication wastage we don’t know. Presumably these extreme values reflect some combination of all of these possibilities. In the end, however, we believe that most researchers who use these measures think of and talk about them as measures of adherence, and for that reason we would argue that CMA1 and CMA2 should not be used. Nonetheless, further studies are needed to better understand patient factors associated with very high rates of dispensing, and whether such excessive dispensings are associated with adverse health outcomes. Reasons for this apparent over-adherence have been attributed to changes in directions not noted in the pharmacy record, intentional variable dosing, and stockpiling .
In practice, at least for our dataset, we found only minimal differences between CMA3/CMA4 and their CMG-based counterparts CMA5/CMA6. Given that the latter are computationally more intensive to calculate, this might argue for the use of the former measures (which are relatively easy to calculate). The main advantage of using the CMG-based measures is their ability to describe gaps in medication use, although this requires the rather strong assumption that the medication is used exactly as directed. Nonetheless we feel that the added benefit of studying gaps in usage conceptually has a lot of appeal and is worth consideration when evaluating the use of either CMA3 or CMA4 versus CMA5 or CMA6.
Each of CMA1 through CMA6 can be calculated solely on the basis of pharmacy dispensing records available during the observation window. The use of CMA7 and CMA8 requires additional knowledge. This could simply be information about medication use prior to the start of the formal observation window of interest or, as in our case, this information supplemented by diagnostic data from an EMR. Due to the growing use of EMRs and the expectation that they will only become more prevalent over time, these requirements should not pose a serious problem to the future use of these measures. However, the availability of such information, while necessary, is not sufficient to justify the use of CMA7 or CMA8. One must further be able to justify the assumption that participants should be taking the medication throughout the observation window. The validity of this assumption is likely to be more true for some medications than for others. In the case of asthma, for instance, ICS are considered first-line therapy for patients with persistent disease. Hence the presumption that a patient, once prescribed them, should continue using them is more likely to be true than not true (although physicians may discontinue use if they are deemed ineffective for a given patient). The presence of stop orders, if available in the EMR, could be used to further refine the calculation of such measures, although our experience is that clinicians are not good about documenting them.
The fact that CMA7 and CMA8 do not require a dispensing during the observation window in order to be calculated should, if the assumptions underlying their use are met, cause them to lead to the most valid measures of adherence. However, given that the assumptions for their use are inevitably not met for some individuals, in practice population-based estimates of adherence based on these measures are probably biased downwards from truth. These observations are consistent with the trends observed in our data, which showed both the lowest estimates of adherence and the least change in estimated adherence with varying length of the observation window. Even if the assumptions underlying CMA7 and CMA8 do lead to a downward bias in the estimates, this needs to be weighed against the fact that these measures can be calculated for everyone. Despite the fact that everyone had a pre-existing diagnosis of asthma and an order or dispensing for ICS in the baseline year, CMA1/3/5 could not be calculated for 32 % of this cohort, and even the less restrictive CMA2/4/6 measures could not be calculated for 17 % of the cohort.
Although we did not evaluate them as part our analyses, under the assumptions for use of CMA7 and CMA8 one could also modify the definitions of CMA4 and CMA6 to include the entire observation window in the denominator. The modified CMA4 index in particular, which would correspond to Hess et al.’s  definition of the Medication Possession Ratio, could be much more easily computed than any of the CMG-based measures and it is not unreasonable to hypothesize that it might be fairly comparable in its measurement properties to CMA7 and CMA8. It would also by definition be definable for all subjects, and not just those with one or more dispensing events during the observation window.
Our motivation for defining CMA8 is probably unique to the context of randomized clinical trials, where the concept of implied initial adherence at the start of an adherence intervention that could not be related to treatment allocation is a relevant consideration for analysis.
Our results are consistent with, and extend, the results of previous investigations that have compared the performance of competing measures of adherence in the same dataset. Hess et al.  compared 11 measures, though per their descriptions of them it is not clear that these were all mathematically distinct measures. They concluded that the equivalent of what we term CMA1-CMA6 provided essentially the same adherence values, although those measures that were capped at a maximum value of 1 (or 100 %), similar to our CMA3-CMA6 measures, produced adherence estimates that were slightly lower than those that were not. Participants in their study had a mean (SD) observation window of 350 (16) days, and they did not report on the impact of using shorter observation windows. A separate review of pharmacy-based measures of medication adherence, however, noted that the assessment of adherence over short intervals is likely to be imprecise and suggested that the observation window should be long enough to span the expected days’ supply from at least three dispensing events . This agrees well with our own observation (Table 3) that the extreme early bias in these measures takes about 9 months to flatten out (the average ICS days’ supply for this population was 2–3 months). Of course adherence often falls naturally over time, particularly among new users  and that is another rationale for prolonged observation of refill behavior. Vink et al.  compared the equivalents of our CMA3 and CMA7 when assessed over one year and concluded that the latter had a significantly better area under the curve for classifying a measure of adherence based on chart review.
Several reports have noted the lack of standardization in terminology in the published adherence literature, noting both that the same term (e.g., Medication Possession Ratio) can mean different things in different papers and conversely that multiple distinct terms have been used to describe the same measure [1, 2, 7, 9]. These reports also note that comparisons with published studies are further complicated by the fact that the precise methodology used for defining a given measure is not always provided. Our approach has been to explicitly define a series of measures and give their rationales and assumptions for valid use, while avoiding assigning a formal nomenclature to them. We have, however, tried to relate our terms to those used by Hess et al. .
In the end, the choice of which adherence measure to use for any given study must be based on the richness of data available to the investigator, the chronicity of the disease for which the medication is being used, the availability of other therapeutic options, knowledge about standards of practice, and the question being addressed by the study. No single measure is likely to be optimal for all occasions. Still, given that the assumptions for use of more sophisticated measures, such as CMA7 or CMA8, are met, we believe our findings suggest that these may be more appropriate (i.e., less biased and more universally estimable) alternatives than simpler measures such as the various derivatives of the MPR.
The study has two main limitations. First, none of the measures described here really speak to the issue of primary nonadherence (failure to obtain the first fill of a medication ordered by their clinician), and instead focus on adherence among known medication users. Thus from the perspective of overall population-based measures of adherence to prescribed medications, all of these measures will tend to have a further upward bias. In addition, we made no attempt to adjust for hospitalization days in our adherence estimates. For populations for which this is a meaningful consideration, it is common to remove hospital days from the denominators of many of these statistics. However, for the purpose of comparing relative properties of the various measures, any bias this introduces is likely to be comparable across the measures so that relative comparisons should be largely unaffected.