Ethical review

The research project was reviewed and approved by the Human Research Ethics Committee of the University of Adelaide.

Design

The Burden of Oral Disease Study was conducted as a cross-sectional study using a mailed survey approach. Dentists were provided with a practitioner logbook in the dentist questionnaire to record for the first 1 to 5 consecutive adult patients (depending on study group assignment of dentist) of a random clinical day the diagnosis of the oral disease or condition treated and treatment they performed. At the conclusion of treatment each practitioner (except those in the study group that had no patient questionnaires to distribute) passed on a survey kit to their sampled patient(s) containing a patient questionnaire, cover letter and explanation sheet. Sampled patients completing the patient questionnaire recorded basic socio-demographic characteristics and data concerning the nature, severity and duration of their symptoms. The primary rationale for this 2-stage sampling methodology was to allow linkage of dentist-assessed oral health status to patient perceptions of quality of life. The patient questionnaires were identified using the practitioner identification number allowing linkage between the practitioner logbook data and patient questionnaire data, but maintaining the anonymity of each patient to the investigators.

Instrument development

A pilot study was conducted which collected five patients per dentist in order to establish the feasibility of the 2-stage methodology. Since the optimum number of patients to sample from dentists was not known, dentists in the main study were randomised into six equal-sized groups in order to assess the sample size-related efficiency and response properties of recording data on from 1 to 5 patients and distributing between 0 to 5 patient questionnaires.

Sampling and data collection

A 2-stage sampling design was used where stage 1 involved sampling dentists and stage 2 involved sampling of patients within selected dentists. Dentists were randomly sampled from the South Australian Dental Register. The Dental Register was used as a sampling frame as it provides a comprehensive listing of all persons registered to work as dentists in the State of South Australia, and is therefore representative of the target population of dentists. Sampled dentists were randomised into one of seven equal-sized study groups to assess the optimum number of patients to sample from dentists and sent a mailed self-complete dentist questionnaire along with up to five self-complete patient questionnaires depending on the study group. Note that dentists in the group that had no patient questionnaires to distribute recorded details of 5 patients in their dentist questionnaire, while dentists in all other groups recorded the same number of patients in their dentist questionnaire as they distributed patient questionnaires.

Data were collected during 2001–02 with a primary approach letter sent initially to each dentist, followed a week later by the survey materials, with a reminder card two weeks later, and up to four follow-up mailings of survey materials to dentists who had not yet responded in order to ensure higher response rates [16].

The emphasis of the project was to obtain precise estimates of the component measures of the burden of oral disease. These are typically expressed as percentages, such as the percentage of persons or percentage of time experiencing symptoms of a given degree of severity. Taking a parameter size of 10% as a reference estimate for any given measure, in order to achieve a level of precision of 20% or less relative standard error, a minimum target sample of n = 225 patients was required. This would provide an acceptable level of precision for estimates as low as 10% in size, and better precision for any estimates larger than 10% in size.

Data items

Dentists recorded the details of the dental conditions that patients had, and patients recorded their experience of those dental conditions. In the patient questionnaire, patients were asked if the dental conditions had caused problems in each of six health state dimensions using the European Quality of Life indicator or EuroQol (EQ-5D+) instrument [8]. The six health state dimensions were: mobility (e.g, walking about), self-care (e.g, washing, dressing), usual activities (e.g., work, study, housework, family or leisure), pain/discomfort, anxiety/depression and cognition (e.g, memory, concentration, coherence, IQ). The EuroQol measures each of these six dimensions according to a 3-level response grading from 1 = no problems, 2 = some/moderate problems and 3 = extreme problems. Patients were also asked to rate their experience of dental problems in the last year using the OHIP-14 [7], which uses 14 items to capture measures of the seven dimensions of functional limitation, physical pain, psychological discomfort, physical disability, psychological disability, social disability and handicap. For each of the 14 OHIP questions subjects were asked how frequently they had experienced impact in the preceding 12 months using a Likert-type response scale re-coded as a Guttman scale 4 = very often, 3 = fairly often, 2 = occasionally, 1 = hardly ever and 0 = never.

Measures

The main output measure was calculated by converting EuroQol item responses to health state values, where each set of responses on the standard 5-item instrument was matched to a health state value where 0 = death and 1.0 = perfect health by an algorithm derived from modelling values [17] using health state preferences elicited from a general population [8]. The responses to the OHIP-14 instrument were coded into categories of 'Never', 'Hardly ever' and 'Occasionally/Fairly often/Very often' with each category converted into indicator variables with values of one if there was a response in that category or with values of zero if there was no response in that category. Age was entered in years and sex was coded as one for males and zero for females.

Data analysis

The characteristics of responding patients were compared descriptively with published data on dental patients and the Australian population. A random sample of two thirds of the respondents was used to construct models of health state values (see Figure 1 for an outline of the sampling and analysis). Two tobit regression models were constructed using the EQ-5D based health state value as the dependent variable with the independent variables of OHIP-14 items, sex and age. Tobit regression was used to account for censoring of the characteristically bounded nature of health state values that can result in biased and inconsistent estimates using ordinary least-squares regression [18–20]. One model used the OHIP-14 items as categorical variables with the category 'Never' as the reference category, while the other model used the OHIP-14 item responses as continuous variables that were coded from 0 to 4. Cases with missing data on the dependent or independent variables were excluded from the analysis. Non-significant terms were retained for comparability across models, and their potential value in controlling for confounding [21]. The remaining one third sample of the respondents was used to test the models by comparing fitted versus actual values using forecast errors. Forecast errors were calculated by subtracting fitted values from actual health state values and dividing by the mean actual health state value to convert absolute forecast errors into relative forecast errors as a percentage of the actual sample mean health state value. Model building strategies also included testing model fit after inclusion of additional terms such as age-squared to model non-constant age effects and interactions between OHIP-14 items, and examining correlations of independent variables for possible collinearity. The design effect of clustering of patients within sampled dentists was calculated and used as a weight to adjust the reported confidence intervals.

Response

Characteristics of patients

Distribution of variables

Regression models

Note that alternative versions of the models were attempted using an age-squared term and also exploring interactions between OHIP items, but as they did not substantially alter pseudo R-squared values or forecast errors, the results for these additional models are not presented. Examination of correlations among the independent variables indicated that OHIP items were positively correlated with rho ranging between 0.19 and 0.84. The only correlation above 0.75 occurred between the items 'Felt self-conscious' and 'Been embarrassed'. However dropping one of these items had no effect on the direction of associations and only minimal effect on the pseudo R-squared value or the significance and magnitude of the regression coefficient, hence the full set of items were retained in the analysis.

Response and representativeness

Response rates to the survey were adequate for both the dentist and patient questionnaires [25]. Comparison of respondents against estimates for private general practice and the Australian population indicated a slightly higher percentage of female patients compared to the population consistent with higher reported visiting rates by females [24], but both place of birth and time since last visit was similar. While dental insurance was higher, the percentage of check-up visits was lower among patients indicating a higher percentage of dental problems for patients compared to the population.

The use of data from a self-selected typical day was used to provide representative estimates. A report has shown that there was no significant difference in service rates in all 10 main areas of service between data collected over a 10-day sampling period compared with estimates based on one typical day nominated from the 10-day sampling period by the responding dentists [22].

Instruments

Previous analysis of the EuroQol and OHIP-14 instruments found that both the generic and specific measures showed evidence of discriminant validity in detecting associations with visit characteristics and main dental condition being treated among dental patients. There was little difference by type of measure used with simple counts, additive scores and scale scores demonstrating discriminative ability in both bivariate and multivariate analyses [26]. The generic and specific instruments showed a degree of overlap in dimensions, particularly for pain [27]. The partial separation in the domains of both instruments confirms that generic and specific measures can be used in combination to capture different elements of quality of life – with both instruments covering symptom experience of pain but EuroQol tapping daily activities such as self-care and usual activities and OHIP tapping oral health-specific aspects of oral functional limitation and physical disability as well as psychological and social aspects of disability and handicap. There are, however, plausible potential links between the two descriptive systems. For example, mobility could in some persons be influenced by oral health problems, such as severe toothache that results in their seeking bed rest or limiting their movements. It is also worth noting that in this study the EQ-5D was asked in relation to dental problems, hence potential effects of co-morbidity on health state value would not be confounding the relationship between EQ-5D health states and OHIP-14 items.

As reported for population surveys [10], the distribution of the EuroQol was skewed among dental patients with a minority reporting problems on any one dimension [26]. The effect of skewness is to produce a ceiling effect where most of the responses are clustered at one extreme [28]. This ceiling effect was less marked for the pain/discomfort and anxiety/depression dimensions, but the large numbers of respondents reporting no problems may make the instrument less appropriate for studies of milder conditions [2]. Despite the simplicity of the EuroQol in terms of dimensions and response categories, there is growing evidence of its construct validity [10]. Similarly, the OHIP has also been noted as displaying ceiling effects even among dental patients [29], indicating limitations in both the generic and specific instruments as descriptive measures. Floor effects are just like ceiling effects but they are found at the opposite end of the scale. Floor effects, with high percentages reporting problems, have not been reported to be of the same extent for the EuroQol compared to other generic measures such as SF-6D or HUI3 making it more suitable for generating preference-based index values for use in economic evaluation when the conditions studied are more severe [2, 30].

Health state value algorithm

The present study provides an algorithm that transforms OHIP-14 scores into estimates of health state values. While both the models performed similarly, the lower forecast errors for the continuous model indicate that this may be preferred over the categorical model. This algorithm can then be used in QALY or DALY analyses from databases that contain OHIP-14 scores, but not health state values. While, the preferred method would be to derive health state valuations from a population sample, the mapping algorithm facilitates cost-effectiveness and burden of disease studies through proxy health state values that can be derived from the numerous oral health studies that have collected data using the OHIP instrument.

The proportion of variance explained by the algorithm from a previously reported mapping of a profile measure on an index was between 35% – 55% of the variance in HUI3 explained using SF-12 items [3]. In this study the values of pseudo R-squared were higher. However, values of pseudo R-squared are based on likelihood statistics from a model containing the independent variables versus a model containing a constant term only, rather than a comparison of fitted to observed values as obtained from linear regression models (31). Previous mapping studies have cautioned that the use of such mapped utility values would not be appropriate for use at the individual level and instead should be applied to analyses performed at the group level [3]. In the present study, the fitted mean was higher than the observed mean for the group of patients as a whole, and when stratified into different levels of health state value. Hence, an individual's predicted health state value may be an over- or under-estimate of the true health state if it were observed (i.e., directly measured), and as a group, health states would be slightly over-estimated. Such variation may be acceptable at a group level where aggregate health state values are of interest. However, some caution should be applied in interpreting findings as health states will tend to be slightly over-estimated (i.e., healthier than if directly measured) and in the case of disability weights for burden of disease studies the over-estimation of health states equates to lower or conservative estimates of disability weights (i.e., less disability than if directly measured). As participants in this study were dental patients who may have more oral disease than the general population the over-estimation of health states may be exacerbated when applied to general populations. The consistency of the over-estimation of health state values suggests that no systematic variation or bias would result from the application of the algorithm to sub-groups with different underlying health states.