- Research article
- Open Access
- Open Peer Review
Emergency department documentation templates: variability in template selection and association with physical examination and test ordering in dizziness presentations
BMC Health Services Researchvolume 11, Article number: 65 (2011)
Clinical documentation systems, such as templates, have been associated with process utilization. The T-System emergency department (ED) templates are widely used but lacking are analyses of the templates association with processes. This system is also unique because of the many different template options available, and thus the selection of the template may also be important. We aimed to describe the selection of templates in ED dizziness presentations and to investigate the association between items on templates and process utilization.
Dizziness visits were captured from a population-based study of EDs that use documentation templates. Two relevant process outcomes were assessed: head computerized tomography (CT) scan and nystagmus examination. Multivariable logistic regression was used to estimate the probability of each outcome for patients who did or did not receive a relevant-item template. Propensity scores were also used to adjust for selection effects.
The final cohort was 1,485 visits. Thirty-one different templates were used. Use of a template with a head CT item was associated with an increase in the adjusted probability of head CT utilization from 12.2% (95% CI, 8.9%-16.6%) to 29.3% (95% CI, 26.0%-32.9%). The adjusted probability of documentation of a nystagmus assessment increased from 12.0% (95%CI, 8.8%-16.2%) when a nystagmus-item template was not used to 95.0% (95% CI, 92.8%-96.6%) when a nystagmus-item template was used. The associations remained significant after propensity score adjustments.
Providers use many different templates in dizziness presentations. Important differences exist in the various templates and the template that is used likely impacts process utilization, even though selection may be arbitrary. The optimal design and selection of templates may offer a feasible and effective opportunity to improve care delivery.
Clinical documentation systems have been shown to influence processes of care in many different settings[1–7]. One of the most common places that clinical documentations systems are used is the emergency department (ED). ED providers frequently use complaint-specific paper templates to document evaluation and management services. One company, T-system, Inc., makes a set of templates (e.g., "Dizziness" template, "Headache" template, "Acute Chest Pain" template) that are used by physicians to document the care encounter, and the company reports that more than 40% of EDs in the United States use these templates.
Although the templates are widely used, we are not aware of prior research assessing the association of the templates with processes of care. Prior research on the templates has focused on the association of the templates with time and billing outcomes[9, 10]. Understanding and awareness of the effects of this particular system is important since it is widely used. The purpose of the system is primarily to aid in efficiency and billing. But any item placed on a documentation form has the potential to unintentionally function as a "checklist" item. In other words, seeing the item on the template could trigger the corresponding action which might not have been performed otherwise.
Unique to the T-System templates are the numerous template options that providers have to choose from. In fact, there are more than 60 different templates (e.g., "Dizziness" template, "Headache" template, "Acute Chest Pain" template) that are used by physicians to document the care encounter. To our knowledge, prior research has not reported on the variability in use of individual templates within a population of a symptom presentation like dizziness. Though there is a "Dizziness" template, there are also other templates that could reasonably be selected in primary dizziness presentations (e.g., "Nausea, Vomiting, Diarrhea" template, "General Adult" template, "Neuro Deficit" template) and other templates that could be selected if dizziness was one of several symptoms, which is a common scenario. If there is wide variation in template selection and also important differences in the templates, then future refinements of the system would need to consider selection of the template as an important factor in optimizing its effects.
In a population-based study of ED dizziness visits, we discovered that all participating EDs use the T-System complaint-specific templates as the principal form of physician documentation. Because of considerable heterogeneity in the type of template used within this group and important differences in the items on different templates, we had an opportunity to describe variability in the selection of templates and also to test a hypothesis that the template type used to document care is associated with the documentation of processes of care. We selected two processes of care to study the potential association: head computerized tomography (CT) scan utilization and nystagmus assessment documentation. These processes were chosen because they are both important topics in dizziness evaluations and because items for these processes only appear on a subset of the templates. Head CT scan use is an important topic because it is widely regarded as overused in the assessment of dizziness patients[11, 12]. Nystagmus assessment is an important topic because nystagmus is the hallmark indicator of vestibular dysfunction and thus can be the crucial element in making a diagnosis[13, 14]. Dizziness presentations are very common and heterogeneous, and important differences are likely to exist between dizziness patients that receive different complaint-specific templates. To control for this important source of selection effects, propensity score matching was used. If templates do impact care delivered, then the optimal design and selection of templates may be a feasible way of optimizing care.
Study setting and Data Collection
The Dizziness Evaluation and Treatment in Corpus Christi, Texas (DETECT) Project is a population-based, emergency department, dizziness surveillance study in Nueces County, Texas. The ultimate aim of the study is to define the impact of dizziness in terms of numbers and healthcare utilization, and to search for opportunities to optimize care. The county is served by six adult care EDs. The study was approved by the relevant institutional review boards (i.e., The University of Michigan and the participating EDs in Corpus Christi) and granted a HIPPA waiver of informed consent. Prospective active case ascertainment is utilized to review recent ED presentations. Dizziness visits are identified by a trained abstractor who screens ED logs for any of the following reason for visit terms: dizziness, imbalance, or vertigo. The abstractor underwent training procedures and certification in the collection of data and in data entry, and was blinded to the current study question. On-going quality assurance mechanisms for data collection are in place. For the purposes of the current study, patient visits were identified from January 15, 2008 through January 14, 2009.
The study population included all visits for dizziness to one of the Nueces County EDs. Exclusion criteria were age <18 years, principal residency outside of Nueces County, institutionalized individuals, trauma presentations, and patients leaving before being seen by a health care provider.
ED template type exposure and Outcomes
All EDs participating in this study use complaint-specific templates produced by T-System, Inc., as the primary record of the physician encounter. The T-System templates are a set of 60 different chief complaint-specific paper templates. Most templates are two pages in length and each template was developed to address presentation components relevant to the specific chief complaint. Utilizing the pre-printed items on the template, characteristics of the patient presentation and assessments rendered (i.e., evaluation and management) are documented by ED providers during the course of care using circles, checks, and backslashes. Each section of the template also has blank space where physicians can hand write additional information. For the purposes of this study we coded each of the complaint-specific template types used in this study as a head CT-item template and/or a nystagmus-item template, if the template type contained relevant items pre-printed on the form (Table 1). We excluded visits when a template was not used to document the encounter (i.e., the encounter was instead documented with a handwritten or dictated note) or when the template was of poor quality such that information could not be abstracted.
The primary outcomes considered in this study were documentation of head computerized tomography (CT) and nystagmus assessment. A head CT was determined to have been performed if a head CT result was recorded on the template or a head CT report accompanied the ED record. Visits were considered as having documentation of a nystagmus assessment if the completed template indicated the assessment was performed. To count as performed, the template needed to either have the relevant item checked/circled/slashed (if it was a pre-printed item) or have the item and results of the assessment handwritten on to the template. When these criteria were not met, the process was counted as not performed.
Covariates were included that were thought to be associated with the use of template types, as well as variables thought to be associated with the utilization of the processes of interest. These included sociodemographic variables (age, gender, race-ethnicity, insurance status), hospital, type of dizziness symptom, dizziness presentation type, number of medical symptoms (inclusive of dizziness symptoms, pain, palpitations, fatigue, generalized weakness, nausea, vomiting, shortness of breath), number of other neurologic signs or symptoms, number of stroke risk factors, and admission status. A clinical item (i.e., symptom, exam sign, stroke risk factor) was considered to be absent if there was no mention of it being present. The type of dizziness symptom was categorized using a hierarchy established a priori whereby mention of vertigo in the template was selected over other dizziness symptoms, and subsequently the hierarchy followed as such: imbalance, non-vestibular types of dizziness (i.e., lightheadedness, fainting, or psychological dizziness), and dizziness not otherwise specified. The dizziness presentation type was also categorized using a common categorization scheme because of the differences in potential etiologies relative to the presentation characteristics. The presentation categories were determined by the abstractor who searched the template for key descriptors about the characteristics of the presentation and classified each case using predefined criteria. Five categories of dizziness presentations were used: acute (i.e., ≤ 7 days from onset) constant dizziness, recurrent spontaneous attacks of dizziness, recurrent positionally triggered attacks of dizziness, subacute or chronic (>7 days from onset) constant dizziness, and dizziness as an accompaniment symptom. Presentations were categorized as "dizziness as an accompaniment" when dizziness was not the principal symptom. As a data reduction method for the project, visits categorized with dizziness as an accompaniment presentation were no longer fully abstracted beginning on September 12, 2008. Thus visits categorized with dizziness as an accompaniment on or after that date were not included in the current analysis.
Logistic regression models were used to compare the odds of documentation of each process among patient visits having a template that contained the relevant pre-printed items compared to patient visits that did not have a template with relevant pre-printed items, controlling for covariates. Interaction terms of template type and each other covariate were tested. In the head CT models, a strong interaction of the template type with the dizziness presentation type was found so the interaction terms were retained in the final model. The c-statistic of each model demonstrated good to excellent discrimination (nystagmus model c-statistic, 0.9704; head CT model c-statistic, 0.8093) and Hosmer-Lemeshow tables showed excellent calibration for each model. The logistic regression models were used to generate the predicted probability of having the relevant process documented or performed by template type while holding all other covariates at their means.
Repeated visits were approached by limiting the population to only the first visit. A secondary analysis was performed including all visits and controlling for repeated visit effects (multivariable generalized estimating equation model) but results did not differ.
Propensity score analysis was performed to further assess the comparability of the treatment groups. Standard methods for propensity score development were used and a detailed description of these methods is provided in Additional file 1. All analyses were performed using STATA 10.0 (StataCorp, College Station, Texas).
Out of 1,593 visits for dizziness, there were 1,488 unique patients. After excluding three visits that had a dictated or hand-written note rather than a template, the final cohort was comprised of 1,485 visits. No visits were excluded because of inability to abstract information from the template. The median age of the cohort was 49.7 years (range, 18-96 years), and 974 (65.6%) were female. Table 2 shows the characteristics of the population.
A total of 31 different templates were used in this population. The most common template used was the 'Dizziness' template (53% visits), followed by 'Nausea, Vomiting, and Diarrhea' template (6.5%), 'General Adult' template (5.7%), 'Headache' template (5.3%), and 'Abdominal Pain' template (4.1%). A head CT was performed in 28.5% of visits, and a nystagmus assessment was documented in 59.8% of visits.
A head CT was performed in 38.8% (402/1036) of the visits that received a template with a head CT item, compared with only 11.1% (50/449) of those not receiving a template with a head CT item. Even after adjusting for the covariates in the multivariable model, the difference remained substantial with the probability of head CT performance of 29.3% (95% CI, 26.0%-32.9%) when a template with a head CT item is used compared to only 12.2% (95% CI, 8.9%-16.6%) when template without a head CT item is used. Testing for interaction effects found a significant interaction of the template type with the dizziness presentation type, such that the overall association of the template type with head CT performance was mostly driven by the "dizziness as an accompaniment" presentation type. In the dizziness as an accompaniment presentation group the probability of receiving a head CT was 38.5% (95% CI, 29.7%-48.2%) if a template with a head CT item was used compared with 8.4% (95% CI, 5.2%-13.3%) if a template without a head CT item was used. The other presentation categories demonstrated trends favoring an association of head CT performance with head CT template use (Table 3), and a separate analysis conducted after excluding the "dizziness as an accompaniment" patients (i.e., population of all dizziness presentations except those with dizziness as an accompaniment) found that the probability of receiving a head CT was 31.8% (95% CI, 28.5%-35.4%) if the patient received a template with a head CT item versus 23.4% (95% CI, 15.7%-33.3%) if the patient did not, with all other covariates held at their means. Propensity score adjustments led to slight changes in the effects but no substantial changes in the overall inferences (Table 3). The absolute differences in the adjusted probability of receiving the head CT are shown in Figure 1.
A nystagmus assessment was documented in 95.6% (834/872) of visits having a nystagmus item template and only 8.8% (54/613) of visits without a nystagmus item template. Even after adjusting for all covariates in the model (all covariates set to their mean), the probability of documentation of nystagmus remained high (95.0%, 95% CI, 92.8%-96.6%) when a nystagmus item template was used and low (12.0%, 95%CI, 8.8%-16.2%) when the template used did not contain a nystagmus item (Figure 2). After matching visits by propensity score, the visits having a nystagmus template remained substantially associated with documentation of a nystagmus assessment. The absolute difference in the probability of receiving a nystagmus assessment calculated by the propensity score analysis was 67.8% (95% CI, 54.6%-81.1%) when a nystagmus-item template was used compared to when a nystagmus-item template was not used.
The optimal design and selection of templates may offer a feasible and effective opportunity to improve efficiency and reduce variability in healthcare. The finding that systems of clinical documentation are associated with clinical care is not new[1–7]. Across a wide spectrum of clinical topics and study designs, these associations have been reported. The current study contributes to the evidence base on the relationship of documentation systems to clinical care for a couple of reasons. First, the template system studied, T-System, Inc., is already widely used (40% of EDs). Thus, it is important to explore any unintended effects of the system since the real world impact could be substantial. Another unique feature of this study is that the documentation system has a large number of template options. For each patient, a provider must choose from more than 60 different chief-complaint focused templates. As a result, we had the opportunity to study variability in template choice within a large population of similar symptom presentations.
Template systems could be an important factor in the efforts to optimize CT scan use because we found that a head CT template item is associated with head CT scan utilization in this population of dizziness presentations. Though the association was concentrated in the presentation of "dizziness as an accompaniment" group, the trends found in other groups may also be relevant considering that there are more than 2.5 million annual ED dizziness presentations in the United States  and 40% of the EDs use these templates. Finding the association of template item with CT use is particularly provocative and timely because a strong sentiment exists that CT studies are overused in dizziness presentations[12, 16–18]. Head CT scans may be particularly overused in dizziness presentations because nearly 30% of dizziness visits in the ED receive one even though serious central nervous system causes are uncommon, the sensitivity of CT is dismal for the most common central cause (ischemic stroke), and the test is associated with important harms including cost, radiation exposure, and time in the ED[12, 20, 21]. We don't know what the optimal rate of head CT use should be for dizziness presentations, but even a small reduction in the nearly 30% use rate could have an important impact on reducing the net "harms" of the test with very little or no trade off in lost net benefit. Changes in template design should be feasible because these changes should be spared from the barriers to change that impact the many other non-clinical factors related to the use of imaging studies[16, 20, 22].
The optimal design and use of templates might also help to optimize the bedside assessment of the patient with dizziness. Nystagmus assessment documentation had a strong association with whether the nystagmus item was on the template or not. Nystagmus assessments are an important aspect of the clinical evaluation of dizziness patients because the findings can be the critical component in discriminating among causes of dizziness[13, 14]. Prior research demonstrates the shortcomings of differentiating causes of dizziness by the patient's description of the symptom and even by extensive diagnostic testing[23, 24]. Nystagmus, however, is the hallmark sign of a vestibular disorder[13, 14]. The presence or absence of nystagmus and its pattern are key to determining the most likely etiology or localization in dizziness presentations. Some patterns of nystagmus (e.g., uni-directional horizontal nystagmus) are highly indicative of a benign peripheral vestibular disorder in the appropriate clinical context, whereas other patterns (e.g., bi-directional or pure vertical nystagmus) are highly indicative of a central disorder. No nystagmus in an acute dizziness presentation would make any vestibular involvement unlikely. As a result the nystagmus assessment plays an important role in the diagnosis and resulting management decisions. Because the current results suggest a strong association of template-item type with which patients have a nystagmus assessment, optimal template examination items may be a way to reduce variability of important examination assessments. An important limitation to the findings of the examination assessments is that we cannot confirm that documented exam components were actually performed.
The associations between template items and processes suggest that template items might function like a "checklist" item even though this is not the intention of the item placement. The primary intention of the template system is to improve time, efficiency, and billing accuracy. But an item placed on a template could have the unintended effect of functioning like a checklist item by reminding the provider to consider the process. In some cases this may be an unintended positive effect, whereas in other cases it may be an unintended negative effect.
We found that many different template types were used in this population. The large number of template types used suggests that the template selection may be more arbitrary than systematic. Though the "Dizziness" template was by far the most commonly used, more than 30 different template types were used throughout the whole population. The propensity scores were well balanced on many different important variables (about 95% of the visits were in the area of common support in the propensity score analysis) indicating substantial similarities in the characteristics of visits receiving different template types. We do not know why so many different templates were used. Some possible factors include the non-specific nature of the "dizziness" label and the fact that dizziness frequently co-occurs with other symptoms. When important differences exist in the content of the various templates, then the selection of the template by the physician may be just as important as the design of the template by the company. This in turn leads to the important question: What happens if the wrong template is picked? The results of this study suggest that the patient would be more likely to receive processes unique to the "wrong" template and less likely to receive processes unique to the "right" template. These differences could influence the efficiency and effectiveness of care.
What contributes to the template selection is not clear. Further research is necessary to understand this factor. In some cases, the template may be selected by the triage staff or based on triage staff documentation. Some physicians may have a preference for certain templates and select the template based on its content relative to what they already plan to do.
We did not find an "ideal" template for dizziness presentations in this population, and thus it would be difficult to judge the appropriateness of the template selection. For example, the "Dizziness" template was only one of two templates that had nystagmus as a pre-printed item but this template also had a head CT item. Thus a dizziness presentation receiving the "Dizziness" template was more likely to receive a nystagmus assessment (unintended benefit), but also was more likely to receive a head CT (unintended consequence). All of the template options that did not have a head CT item also did not have a nystagmus assessment item. Ultimately, these issues have a large potential to be resolved by using an electronic template that can adjust upcoming items (e.g., exam components, management options) based on details entered about the clinical presentation, thus reducing the effect of selection of a paper template.
This study was performed in a representative community and the results may not be generalizable to other settings. The analysis was not adjusted for physician level variability because physician identifiers were not collected. Physician level information should be considered in future studies of this type. This study was limited as it was based on medical record review. It remains possible that clinical information was obtained by treating physicians but was not documented. Nevertheless, prior research has demonstrated acceptable concordance between documentation in the medical record and actual performance, as assessed by direction observation or videotapes[26, 27]. Because there are no consensus guidelines on the use of head CT in dizziness presentations, the appropriateness of use of the test could not be assessed. No randomized controlled trials have proven the benefit of a nystagmus assessment on patient outcomes, though the same could be said for most individual bedside assessment components. Ultimately, the cause and effect of template-items on care delivered can only be determined by randomized controlled trials which would be an important future step in efforts to optimize care. Future studies would also be important to assess whether unintended template effects contribute to either outcome benefits or outcome consequences.
This study found that many different complaint-specific templates are used by providers to document care of ED dizziness presentations. In addition, we found an association of template type used to document care with the processes of care delivered. The template type was associated with head CT utilization. Documentation of a nystagmus examination was strongly associated with whether a nystagmus item was included as a pre-printed item on the template used to document care. Optimal template item design and selection is likely to be an important area of future research since items impact care and selection of templates may be arbitrary. The more widespread adoption of electronic medical records will increase the use of template-like documentation systems enabling a potentially large opportunity to tie documentation and ordering of tests to efficient and effective algorithms of care. An electronic template also has the potential to reduce selection effects. The optimal design and selection of templates may offer a feasible and effective opportunity to improve care delivery.
Weyer SM, Konrad N, Esola D, Goodwin MA, Stange KC, Flocke SA: Features of medical records in community practices and their association with preventive service delivery. Med Care. 2005, 43 (1): 28-33.
Hahn KA, Ferrante JM, Crosson JC, Hudson SV, Crabtree BF: Diabetes flow sheet use associated with guideline adherence. Ann Fam Med. 2008, 6 (3): 235-238. 10.1370/afm.812.
Fielstein EM, Brown SH, McBrine CS, Clark TK, Hardenbrook SP, Speroff T: The effect of standardized, computer-guided templates on quality of VA disability exams. AMIA Annu Symp Proc. 2006, 249-253.
Laflamme MR, Dexter PR, Graham MF, Hui SL, McDonald CJ: Efficiency, comprehensiveness and cost-effectiveness when comparing dictation and electronic templates for operative reports. AMIA Annu Symp Proc. 2005, 425-429.
Parikh JA, Yermilov I, Jain S, McGory ML, Ko CY, Maggard MA: How much do standardized forms improve the documentation of quality of care?. J Surg Res. 2007, 143 (1): 158-163. 10.1016/j.jss.2007.03.040.
Kanegaye JT, Cheng JC, McCaslin RI, Trocinski D, Silva PD: Improved documentation of wound care with a structured encounter form in the pediatric emergency department. Ambul Pediatr. 2005, 5 (4): 253-257. 10.1367/A04-196R.1.
Hysong SJ, Best RG, Pugh JA: Clinical practice guideline implementation strategy patterns in Veterans Affairs primary care clinics. Health Serv Res. 2007, 42 (1 Pt 1): 84-103. 10.1111/j.1475-6773.2006.00610.x.
T-System, Inc. [http://www.tsystem.com]
Marill KA, Gauharou ES, Nelson BK, Peterson MA, Curtis RL, Gonzalez MR: Prospective, randomized trial of template-assisted versus undirected written recording of physician records in the emergency department. Ann Emerg Med. 1999, 33 (5): 500-509. 10.1016/S0196-0644(99)70336-7.
Mulvehill S, Schneider G, Cullen CM, Roaten S, Foster B, Porter A: Template-guided versus undirected written medical documentation: a prospective, randomized trial in a family medicine residency clinic. J Am Board Fam Pract. 2005, 18 (6): 464-469. 10.3122/jabfm.18.6.464.
Kerber KA, Meurer WJ, West BT, Fendrick AM: Dizziness presentations in U.S. emergency departments, 1995-2004. Acad Emerg Med. 2008, 15 (8): 744-750. 10.1111/j.1553-2712.2008.00189.x.
Kerber KA, Schweigler L, West BT, Fendrick AM, Morgenstern LB: Value of CT scans in ED dizziness visits: Analysis from a nationally-representative sample. Am J Emerg Med. 2009.
Baloh RW: Vertigo. Lancet. 1998, 352 (9143): 1841-1846. 10.1016/S0140-6736(98)05430-0.
Hotson JR, Baloh RW: Acute vestibular syndrome. N Engl J Med. 1998, 339 (10): 680-685. 10.1056/NEJM199809033391007.
Rosenbaum PR, Rubin DB: The central role of the propensity score in observational studies for casual effects. Biometrika. 1983, 70 (1): 41-55. 10.1093/biomet/70.1.41.
Government Accountability Office: Medicare Part B Imaging Services: Rapid Spending Growth and Shift to Physician Offices Indicate Need for CMS to Consider Additional Management Practices. Volume Publication GAO-08-452. 2008, washington, DC: Government Accountability Office
Deyo RA: Imaging idolatry: the uneasy intersection of patient satisfaction, quality of care, and overuse. Arch Intern Med. 2009, 169 (10): 921-923. 10.1001/archinternmed.2009.124.
Polensek SH, Tusa R: Unnecessary diagnostic tests often obtained for benign paroxysmal positional vertigo. Med Sci Monit. 2009, 15 (7): MT89-94.
Chalela JA, Kidwell CS, Nentwich LM, Luby M, Butman JA, Demchuk AM, Hill MD, Patronas N, Latour L, Warach S: Magnetic resonance imaging and computed tomography in emergency assessment of patients with suspected acute stroke: a prospective comparison. Lancet. 2007, 369 (9558): 293-298. 10.1016/S0140-6736(07)60151-2.
Iglehart JK: Health insurers and medical-imaging policy--a work in progress. N Engl J Med. 2009, 360 (10): 1030-1037. 10.1056/NEJMhpr0808703.
Sodickson A, Baeyens PF, Andriole KP, Prevedello LM, Nawfel RD, Hanson R, Khorasani R: Recurrent CT, cumulative radiation exposure, and associated radiation-induced cancer risks from CT of adults. Radiology. 2009, 251 (1): 175-184. 10.1148/radiol.2511081296.
Birbeck GL, Gifford DR, Song J, Belin TR, Mittman BS, Vickrey BG: Do malpractice concerns, payment mechanisms, and attitudes influence test-ordering decisions?. Neurology. 2004, 62 (1): 119-121.
Newman-Toker DE, Cannon LM, Stofferahn ME, Rothman RE, Hsieh YH, Zee DS: Imprecision in patient reports of dizziness symptom quality: a cross-sectional study conducted in an acute care setting. Mayo Clin Proc. 2007, 82 (11): 1329-1340. 10.4065/82.11.1329.
Colledge NR, Barr-Hamilton RM, Lewis SJ, Sellar RJ, Wilson JA: Evaluation of investigations to diagnose the cause of dizziness in elderly people: a community based controlled study. Bmj. 1996, 313 (7060): 788-792.
Hales BM, Pronovost PJ: The checklist--a tool for error management and performance improvement. J Crit Care. 2006, 21 (3): 231-235. 10.1016/j.jcrc.2006.06.002.
Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, Jaen CR: How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998, 36 (6): 851-867. 10.1097/00005650-199806000-00009.
McDermott MF, Lenhardt RO, Catrambone CD, Walter J, Weiss KB: Adequacy of medical chart review to characterize emergency care for asthma: findings from the Illinois Emergency Department Asthma Collaborative. Acad Emerg Med. 2006, 13 (3): 345-348. 10.1111/j.1553-2712.2006.tb01705.x.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/11/65/prepub
This work was supported by grant K23 RR024009 (KAK) from the National Center for Research Resources of the National Institutes of Health.
The authors declare that they have no competing interests.
KAK, LBM, TPH, AMF, and WJM made substantial contributions to the study concept and design. KAK and LBM collected the data. KAK and TPH performed the statistical analysis. KAK, TPH, AMF, and LBM made substantial contributions to the interpretation of data. KAK wrote the manuscript. KAK, LBM, TPH, AMF, and WJM made substantial contributions to the critical revisions of the manuscript. All authors read and approved the final manuscript.