Development and validation of the Diabetes Numeracy Test (DNT)
© Huizinga et al; licensee BioMed Central Ltd. 2008
Received: 15 May 2007
Accepted: 01 May 2008
Published: 01 May 2008
Low literacy and numeracy skills are common. Adequate numeracy skills are crucial in the management of diabetes. Diabetes patients use numeracy skills to interpret glucose meters, administer medications, follow dietary guidelines and other tasks. Existing literacy scales may not be adequate to assess numeracy skills. This paper describes the development and psychometric properties of the Diabetes Numeracy Test (DNT), the first scale to specifically measure numeracy skills used in diabetes.
The items of the DNT were developed by an expert panel and refined using cognitive response interviews with potential respondents. The final version of the DNT (43 items) and other relevant measures were administered to a convenience sample of 398 patients with diabetes. Internal reliability was determined by the Kuder-Richardson coefficient (KR-20). An a priori hypothetical model was developed to determine construct validity. A shortened 15-item version, the DNT15, was created through split sample analysis.
The DNT had excellent internal reliability (KR-20 = 0.95). The DNT was significantly correlated (p < 0.05) with education, income, literacy and math skills, and diabetes knowledge, supporting excellent construct validity. The mean score on the DNT was 61% and took an average of 33 minutes to complete. The DNT15 also had good internal reliability (KR-20 = 0.90 and 0.89). In split sample analysis, correlations of the DNT-15 with the full DNT in both sub-samples was high (rho = 0.96 and 0.97, respectively).
The DNT is a reliable and valid measure of diabetes related numeracy skills. An equally adequate but more time-efficient version of the DNT, the DNT15, can be used for research and clinical purposes to evaluate diabetes related numeracy.
In 1992, the National Adult Literacy Survey (NALS) estimated that over 90 million adult Americans have inadequate literacy and numeracy skills . More recently, the National Assessment of Adult Literacy (NAAL) showed that 36% of American adults had basic or below basic health-related literacy and quantitative skills . In that survey, 22% of adults demonstrated below basic quantitative skills and could only perform simple, concrete, single-step math operations, such as addition. Another 33% demonstrated basic quantitative skills, solving single-step math problems where the operation was either given or easily inferred. Patients with low literacy can have difficulty following medical instructions, understanding health information and performing self-management tasks . Low literacy is associated with worse disease knowledge and poorer clinical outcomes in patients with diabetes [4, 5]. The broad construct of literacy consists of several components: print literacy, oral literacy and quantitative skills . Many of the recent studies evaluating the relationship between literacy and health have focused on print literacy with little attention given to quantitative or numeracy related skills. While common for patients with low literacy to have inadequate quantitative skills, patients with adequate literacy may still have difficulty with numeracy.
Numeracy is an important component of literacy that can be simply defined as the ability to understand and use numbers in everyday life [6–10]. Numeracy does not refer solely to basic mathematical skills (arithmetic, calculations, fractions, algebra and geometry) but also to one's ability to understand time, currency, measurement, graphic representations, logic, hierarchies and probability. To apply numeracy to a specific problem or task, one must know which numerical skill(s) to apply, deduce and interpret the result(s) and then appropriately apply the result(s) to the situation [7, 11]. For a patient with diabetes, numeracy is needed to interpret blood glucose meter data, properly administer medications and follow nutritional recommendations. For example, a patient with diabetes may need numeracy to calculate their carbohydrate intake and adjust their insulin based on carbohydrates and/or current blood glucose level. Poor numeracy skills in patients with diabetes could lead to suboptimal glycemic control, increased hypoglycemic episodes or widely varying glucose values.
Scales have been developed to measure literacy in the health care setting [12–14]. One of these scales, the TOFHLA, did include 17 items to measure general quantitative skills . These items primarily focused on reading prescriptions and other materials that tested simple mathematical skills such as understanding dates and timing of medication dosage. These items were highly dependent on reading ability and, not surprisingly, they are highly correlated to reading ability. These numeracy items are not included in the shortened form, the S-TOFHLA . The TOFHLA was not designed to measure the range of numeracy skills needed in diabetes management and does not give a provider clinically useful information in the care of a patient with diabetes .
There are no current scales designed specifically to assess diabetes related numeracy skills required to perform daily diabetes self-management. Numeracy skills may be particularly critical in the management of diabetes and the currently available health literacy scales may not adequately identify patients with low numeracy. In order to better understand numeracy in diabetes, the Diabetes Numeracy Test (DNT) was developed. The DNT was specifically designed to evaluate the wide range of numeracy skills used by patients with diabetes.
Research design and methods
The first phase of development included item generation by a group of experts in diabetes, literacy and numeracy. This group included diabetologists, certified diabetes educators, primary care providers, registered dietitians, behavioral researchers in diabetes, and literacy and numeracy experts. The initial item pool contained 70 items that addressed all mathematical skills required in the daily management of diabetes. Those 70 items covered five diabetes self-management areas (nutrition, exercise, glucose monitoring, oral medication and insulin use) and the general mathematical skills required in the management of diabetes including addition, subtraction, multiplication, division, fractions/decimals, numerical hierarchy, inference skills and multi-step calculations. The 70 items were administered to 40 individuals without diabetes to assess understandability. To eliminate redundancy, the expert panel reduced the measure to 45 items that represented the five self-management areas. The panel agreed that the 45 item scale had adequate content validity to address the range of numeracy skills required in the management of diabetes.
The objective of the second phase of development of the DNT was to address the clarity of items for patients with diabetes. Ten cognitive response interviews were conducted with patients with diabetes to assess each item. Interviewees were asked specific questions about each item to assess the understandability of the wording. If an item was unclear, the interviewee was told the purpose of the item and then encouraged to suggest an alternate format or wording. In response to the interviews, the scale was reformatted and slightly reduced to the final 43 items .
Description of DNT Items.
Diabetes Care Domain
Blood Glucose Monitoring
Oral Medication Use
In the final phase of the study, the scale was shortened to a more clinically useful 15 items and then verified through random split sample analysis. To perform the split sample analysis, the sample data were randomly split into two smaller sub-samples. Sub-sample 1 was used for the development of the shortened scale and sub-sample 2 was used for confirmation of the results. In order to determine the items to include in the DNT15, principal components analysis was performed on sub-sample 1. Items with > 0.6 loading on the primary component in sub-sample 1 were included. In addition, those items with > 80% mean correct score were discarded. Three items with high face validity, as determined by practicing diabetologists and diabetes educators, were added back in to bring the total number of items to 15. Reliability was evaluated by internal consistency (Kuder-Richardson 20) and validity was assessed through correlation testing using Spearman's correlations between the DNT15 and the full DNT and comparing the DNT15 to the a priori construct validity model for both sub-samples.
Participants in this study were recruited from four sites: general medicine clinics at two academic health centers, a diabetes clinic at an academic health center and an endocrinology clinic at a VA health center. Participants were paid $20 to participate. A convenience sample of 398 participants was recruited at clinic visits. Inclusion criteria included diagnosis of type 1 or 2 diabetes, age 18 to 80 years and English speaking. Potential participants were excluded if their corrected visual acuity was > 20/50 using a Rosenbaum Pocket Vision Screener or if they had a diagnosis of significant dementia, psychosis or blindness.
Trained research assistants collected patient characteristics, blood glucose monitoring frequency data from glucose meters, and administered a series of surveys to the patients. Measures included a validated literacy measure (REALM), a validated general numeracy scale (WRAT-3R) and a validated diabetes knowledge test (DKT) [12, 17, 18].
Internal consistency was determined by the Kuder-Richardson 20 coefficient of reliability. Spearman's rank order correlations were performed for the a priori model testing. Principal component analysis with oblique rotation was performed to determine the presence of multiple factors and item loadings on the factors. For development of the DNT15, Spearman's correlations were used in the random split sample analysis. Stata 9.2 (StataCorp, College Station, TX) was used for all statistical analysis.
All work was approved by the institutional review boards of Vanderbilt University and the University of North Carolina.
Patient Characteristics (n = 398).
Mean ± SD or n (%)
54.2 ± 13
Annual Family Income
VA diabetes center
Academic center primary care
Academic diabetes clinic
High School or GED
College or more
Literacy status, REALM
≤ 6th grade
≥ 9th grade
Numeracy status, WRAT
≤ 6th grade
≥ 9th grade
0.69 ± 0.18
33.6 ± 8.1
Type of diabetes
On insulin, n = 240
Adjusts for blood glucose
Adjusts for carbohydrate intake
Blood glucose monitoring
≤ 1 time per day 2–3 times per day
133 (48) 185 (46)
≥ 4 times per day
Received prior diabetes education
Hemoglobin A1c, (%)
7.6 ± 1.7
Duration of diabetes, years
11 ± 9
DNT correlations with a priori model
Spearman's correlation (rho)
DNT15 and sub-sample analysis: Spearman's correlations.
Sub-sample 1 (rho)
Sub-sample 2 (rho)
Psychometric analysis of the 43 item version of the DNT shows that it has good reliability and validity in testing numeracy skills in patients with diabetes. Of the six a priori construct hypotheses, five demonstrated significant correlations in the expected directions. The DNT was correlated with diabetes knowledge, education, socioeconomic status, literacy and general numeracy. Insulin use was not correlated with the DNT scores as initially predicted. This is likely due to the fact that many patients on insulin may be placed on long-acting insulin in which one or two doses may be administered with no adjustment for blood glucose or carbohydrate intake. When complexity of the insulin regimen was taken into account, more complex regimens including adjustment for carbohydrate intake and blood glucose level were significantly associated with the DNT.
The shortened version, the DNT15, also showed good reliability and construct validity. The DNT15 was designed to retain the items that discriminated diabetes related numeracy skills while also keeping the items that would be most useful to a diabetes educator or clinician. The five diabetes self-care areas are retained in the DNT15, including three items on nutrition, one item about exercise, three items regarding blood glucose monitoring, one item on oral medications and seven items about insulin administration. A diabetes educator or clinician may use the DNT15 to help target education or guide therapy.
Patients with low literacy also showed low numeracy skills on both the WRAT and the DNT. However, there were several patients with literacy skills above the ninth grade level who had low numeracy skills. Although our population was, on average, highly educated, the mean score on the DNT was only 61% correct. This suggests that numeracy should be evaluated separately from literacy. Patients with low literacy need special instructions and interventions, and patients with inadequate numeracy skills may also require modified counseling and education to improve health outcomes.
This study has several limitations. The DNT was initially developed without input from patients to determine what numeracy skills patients viewed as important. We used a convenience sample of patients recruited from academic and VA clinics, and it is possible that patients who did not participate in the study may have had different literacy and numeracy skills than those that participated. The DNT has been tested in English speaking participants only. The DNT was, in part, validated against a commonly used literacy assessment tool (the REALM), but most patients in the study scored at the highest level that the REALM assesses. A more refined assessment of reading ability at the higher levels may have been more useful.
The DNT and DNT-15 are primarily research tools for understanding the role of diabetes specific numeracy in the management of diabetes. The clinical utility of these scales will be the subject of future research. The treatment of diabetes requires the application of many literacy and numeracy skills. As the disease progresses, the complexity of the regimen may also progress. Patients need ongoing education to appropriately treat their diabetes. However, little is known about the benefits of targeting education to a patient's level of diabetes specific numeracy to improve health outcomes. Other studies have identified the role of literacy and low-literacy techniques in the improvement of health outcomes in diabetes and congestive heart failure [19, 20]. Patients with low numeracy may benefit from similar interventions that address numeracy, particularly in the setting of diabetes management. The DNT and the DNT15 can provide measurement of diabetes specific numeracy and provide more information about the role of disease specific numeracy in future studies. More studies are needed to further understand the role of numeracy tailored interventions in the management of diabetes.
However, there are also clinical implications that can be learned from this study. We learned that the framing of instructions was very important in predicting patient performance. For example, study participants had a difficult time with the multi-step math required to calculate a correction dosage of insulin when instructions were presented as a sequence of sentences. This item was included to mirror clinical practice regarding how patients are currently instructed to take their insulin. Patients clearly had a much easier time calculating the insulin dosage when the material was presented in an easy to read table. This example provides an important lesson for health care providers and educators in effective communication styles for all clinical care recommendations.
Numeracy, a component of literacy, is important in the daily management of many chronic diseases. Even patients with good literacy skills may have marginal numeracy skills. The full DNT and the shortened version, the DNT15, are reliable and valid scales that may be useful in identifying patients that may have deficits in diabetes related numeracy skills.
This study identified several potential problem areas for patients. These areas included food label interpretation and carbohydrate calculations, understanding of when to refill medications, understanding of insulin measurement and insulin adjustment based on blood glucose and/or carbohydrate intake, and application of medication titration instructions. Framing of the instructions impacted the understandability and may impact health outcomes. Simplification of the instructions greatly improved the participant's ability to answer the questions correctly.
To help determine the clinical utility of these scales, we have designed a new educational intervention for patients with diabetes and low literacy and numeracy skills. This new intervention is based on simple instruction, pictorial representation and color-coding for ease of use, and other accommodations to help patients, particularly those with poor literacy and numeracy skills. This intervention is currently being evaluated in a prospective randomized control trial.
This research is funded with support from the American Diabetes Association (Novo Nordisk Clinical Research Award), the Pfizer Clear Health Communication Initiative, and the Vanderbilt Diabetes Research and Training Center (NIDDK 5P60DK020593). Dr. Rothman is also currently supported by an NIDDK Career Development Award (NIDDK 5K23DK065294).
- Kirsch I, Jungeblut A, Jenkins L, Kolstad A: Adult Literacy in America: A First Look at the Findings of the National Adult Literacy Survey. NCES 93275. 1993, National Center for Education Statistics, US Department of EducationGoogle Scholar
- Kutner M, Greenberg E, Jin Y, Paulsen C: The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. 2006, National Center for Education StatisticsGoogle Scholar
- Schillinger D, Grumbach K, Piette J, Wang F, Osmond D, Daher C, Palacios J, Sullivan GD, Bindman AB: Association of Health Literacy With Diabetes Outcomes. JAMA. 2002, 288 (4): 475-482. 10.1001/jama.288.4.475.View ArticlePubMedGoogle Scholar
- Rothman R, Malone R, Bryant B, Horlen C, DeWalt D, Pignone M: The Relationship Between Literacy and Glycemic Control in a Diabetes Disease-Management Program. The Diabetes Educator. 2004, 30 (2): 263-273. 10.1177/014572170403000219.View ArticlePubMedGoogle Scholar
- Rothman RL, Malone R, Bryant B, Wolfe C, Padgett P, DeWalt DA, Weinberger M, Pignone M: The Spoken Knowledge in Low Literacy in Diabetes Scale: A Diabetes Knowledge Scale for Vulnerable Patients. The Diabetes Educator. 2005, 31 (2): 215-224. 10.1177/0145721705275002.View ArticlePubMedGoogle Scholar
- Nielsen-Bohlman L, Institute of Medicine (U.S.), Committee on Health Literacy, NetLibrary Inc: Health literacy a prescription to end confusion. 2004, National Academies Press, Washington, D.C, xix: 345.Google Scholar
- Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE: A definition and operational framework for health numeracy. American journal of preventive medicine. 2005, 29 (4): 375-376. 10.1016/j.amepre.2005.06.012.View ArticlePubMedGoogle Scholar
- Rothman RL, Housam R, Weiss H, Davis D, Gregory R, Gebretsadik T, Shintani A, Elasy TA: Patient understanding of food labels: the role of literacy and numeracy. Am J Prev Med. 2006, 31 (5): 391-398. 10.1016/j.amepre.2006.07.025.View ArticlePubMedGoogle Scholar
- Ancker JS, Kaufman D: Rethinking Health Numeracy: A Multidisciplinary Literature Review. J Am Med Inform Assoc. 2007Google Scholar
- Rothman R, Montori V, Cherrington A, Pignone M: Perspective: The Role of Numeracy in Health Care. Journal of health communication.
- Montori VM, Rothman RL: Weakness in numbers. The challenge of numeracy in health care. J Gen Intern Med. 2005, 20 (11): 1071-1072. 10.1111/j.1525-1497.2005.051498.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, Crouch MA: Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993, 25 (6): 391-395.PubMedGoogle Scholar
- Williams MV, Parker RM, Baker DW, Parikh NS, Pitkin K, Coates WC, Nurss JR: Inadequate functional health literacy among patients at two public hospitals. Jama. 1995, 274 (21): 1677-1682. 10.1001/jama.274.21.1677.View ArticlePubMedGoogle Scholar
- Parker RM, Baker DW, Williams MV, Nurss JR: The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med. 1995, 10 (10): 537-541. 10.1007/BF02640361.View ArticlePubMedGoogle Scholar
- Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J: Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999, 38 (1): 33-42. 10.1016/S0738-3991(98)00116-5.View ArticlePubMedGoogle Scholar
- Vanderbilt Diabetes Research and Training Center Assessment Scales and Educational Materials. [http://vanderbiltdrtc.org/preventionandcontrol/tools.php]
- Wilkinson G: The Wide Range Achievement Test – Third Edition (WRAT-3). 1993, Wilmington, DE: Wide Range, ThirdGoogle Scholar
- Fitzgerald J, Funnell M, Hess G, Barr P, Anderson R, Hiss R, Davis W: The reliability and validity of a brief diabetes knowledge test. Diabetes Care. 1998, 21 (5): 706-710. 10.2337/diacare.21.5.706.View ArticlePubMedGoogle Scholar
- Rothman RL, DeWalt DA, Malone R, Bryant B, Shintani A, Crigler B, Weinberger M, Pignone M: Influence of Patient Literacy on the Effectiveness of a Primary Care-Based Diabetes Disease Management Program. JAMA. 2004, 292 (14): 1711-1716. 10.1001/jama.292.14.1711.View ArticlePubMedGoogle Scholar
- Dewalt DA, Malone RM, Bryant ME, Kosnar MC, Corr KE, Rothman RL, Sueta CA, Pignone MP: A heart failure self-management program for patients of all literacy levels: A randomized, controlled trial [ISRCTN11535170]. BMC Health Serv Res. 2006, 6 (1): 30-10.1186/1472-6963-6-30.View ArticlePubMedPubMed CentralGoogle Scholar
- Bennett I, Switzer J, Aguirre A, Evans K, Barg F: 'Breaking it down': patient-clinician communication and prenatal care among African American women of low and higher literacy. Ann Fam Med. 2006, 4 (4): 334-340. 10.1370/afm.548.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/96/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.