- Study protocol
- Open Access
- Open Peer Review
Decision making preferences in the medical encounter – a factorial survey design
© Müller-Engelmann et al; licensee BioMed Central Ltd. 2008
- Received: 22 September 2008
- Accepted: 17 December 2008
- Published: 17 December 2008
Up to now it has not been systematically investigated in which kind of clinical situations a consultation style based on shared decision making (SDM) is preferred by patients and physicians. We suggest the factorial survey design to address this problem.
This method, which so far has hardly been used in health service research, allows to vary relevant factors describing clinical situations as variables systematically in an experimental random design and to investigate their importance in large samples.
To identify situational factors for the survey we first performed a literature search which was followed by a qualitative interview study with patients, physicians and health care experts. As a result, 7 factors (e.g. "Reason for consultation" and "Number of therapeutic options") with 2 to 3 levels (e.g. "One therapeutic option" and "More than one therapeutic option") will be included in the study. For the survey the factor levels will be randomly combined to short stories describing different treatment situations.
A randomized sample of all possible short stories will be given to at least 300 subjects (100 GPs, 100 patients and 100 members of self-help groups) who will be asked to rate how the decision should be made. Main outcome measure is the preference for participation in the decision making process in the given clinical situation.
Data analysis will estimate the effects of the factors on the rating and also examine differences between groups.
The results will reveal the effects of situational variations on participation preferences. Thus, our findings will contribute to the understanding of normative values in the medical decision making process and will improve future implementation of SDM and decision aids.
- Therapeutic Option
- Share Decision Making
- Situational Factor
- Medical Encounter
- Insurance Physician
Shared decision making
Currently we are witnessing a change in the traditional physician-patient-relationship. Shared decision making (SDM) has become increasingly popular in the last decade . Many patients no longer rely only on their physician's opinion about the best treatment. They have gained self-confidence and many of them want to participate in decisions or even be the only responsible decision maker [2–4].
there are at least two involved: the physician and the patient
both are incorporated into the process of treatment decision making
sharing information is the basic condition of SDM
both agree with the decision made
Research on SDM
Reviews demonstrate multiple favorable effects of SDM and related decision aids. SDM seems to increase patients' satisfaction, quality of life, treatment compliance, knowledge about the disease and the therapeutic options, perceived control of the disease and symptom control [6–9]. However, a recent review suggests that SDM might be useful only under certain circumstances, i.e. related to chronic illness and mental health problems .
Other studies analysed the effects of certain patient characteristics on his/her preferences either to participate or not to participate in the medical decision making process. In regard to socio-demographic variables, research mainly focused on age, educational level and gender. Most of the studies demonstrate a tendency for women [3, 10], younger [11–13] and more highly educated people [4, 14, 15] to prefer more participation. Even so, there are studies which showed opposing results for gender  and a lack of association with educational level . Increasingly, health professionals agree that all patients should be offered SDM irrespective of age, gender or formal education.
When should SDM be used?
Whereas there have been many studies on patients' characteristics and participation preferences, the role of the treatment situation has hardly been investigated. There might be a difference whether there are severe side effects to a treatment or not, whether there are various options or only one, whether the condition to be treated is life threatening or not, on the degree to what extend SDM is regarded as appropriate or not. Up to now, most studies dealing with situational factors related to SDM preferences only focused on one dimension of the treatment situation. Furthermore, there are only a few studies addressing the normative question when SDM should be applied. Mc Kinstrey  confronted patients with video vignettes of consultations about five different themes (serious acute, minor acute, chronic, mental health and lifestyle), using either a shared decision or a directive consultation style. In this study, patients were more likely to prefer SDM in situations dealing with psychological problems or lifestyle. Similarly, Deber et al.  showed, in another vignette study, that in situations involving a potential life threatening disease there was a greater preference for giving control to the physician than in situations involving a minor disease or quality of life. If more than one treatment alternative exists, patients generally want to be involved in treatment decisions according to Guadagnoli and Ward .
What has been less considered is that real treatment situations are characterized by several dimensions and the complex interactions of these dimensions. Against this background, Whitney  developed a model addressing the normative question when SDM should be used. He characterized treatment situations by the level of importance of the medical decision for the patient and by the level of the physician's certainty about the efficacy of the therapeutic options. According to this model, decisions that are of high importance for the patient in combination with a low physician's certainty should be made by the patient, whereas the physician should make decisions of high treatment certainty with low importance for the patient. The remaining decisions should be shared.
However, this model does not include dimensions named before like the number of therapeutic options and side effects of therapeutic options. Up to know it is still not clear, which dimensions of treatment situations are important in answering the question when SDM is preferred. As a first step we suggest a factorial survey of patients and physicians.
Analysing values in complex decisions: the role of the factorial survey
The method of the factorial survey was developed by Rossi and Anderson  in the eighties to study opinions, attitudes and decisions related to complex situations. It combines the strength of a classical experiment with the advantage of a sample survey design. Like an experimental design, it allows to manipulate several variables. At the same time a large number of subjects can participate without undue expense .
In a factorial survey, realistic vignettes, i.e. short narratives of situations or actors, are presented to the subjects. It differs from other vignette surveys in that the characteristics (factors) are varied in a systematic way . Subjects rate the vignettes on a scale depending on the research question, e.g. indicating what kind of decision one should make in that specific situation. Each respondent receives a set of stories with varying combination of characteristics.
The aim of the statistical analysis is to explore the importance of each factor and interactions between different factors. Furthermore, differences in rating between respondents or groups of respondents can be analysed  including demographic variables .
Developed in the social sciences, factorial surveys have up to now been applied in many different fields proving their versatility and utility as a research method. A wide range of topics has been covered, e.g. consumer preferences , how people define alcohol abuse  and how couples decide to move .
Lauder  claims that factorial surveys help to understand the factors that underlie clinical judgments of health care professionals. Yet, there are only a few published examples in health services research. Currently, factorial surveys were occasionally undertaken in nursing science [21, 27–29]. Furthermore, the method was used to investigate the reasons that make a medical error unacceptable , the factors influencing the reaction to medical errors  and how physicians decide to prescribe benzodiazepines for nervousness and insomnia .
We consider the factorial survey to be a feasible method to analyse the variables underlying the decision making process in the medical encounter. It will help to reveal the importance of different situational factors, determining when patients and physicians prefer SDM. Furthermore, it might serve as a tool to analyse differences between patients and physicians in this domain.
Choosing factors in a systematic way
While factors for factorial surveys are often chosen somewhat haphazardly based on literature and individual researchers' perceptions, we undertook a qualitative study to identify systematically the input for our main survey .
We developed an interview guideline based on published literature on situational factors related to SDM. 12 physicians, 15 patients and 14 experts in the health care system were interviewed. Among the latter there was a representative of the German Federal Ministry of Education and Research, the national Commissioner for Patients' Affairs, a speaker of The National Association of Statutory Health Insurance Physicians and a member of The Agency for Quality in Medicine.
In this qualitative study we covered three key areas: characteristics of the disease, characteristics of therapeutic options and characteristics of the patient. For each topic participants were asked to mention factors influencing the decision making process with regard to the person who should make the decision. In a second step, interviewees were asked about their opinion on the importance of the factors mentioned in the literature. The interviews were recorded, transcribed and quantitatively and qualitatively analysed using an inductive coding system.
In total we found 32 factors, 9 related to the disease i.e. severity, 8 related to the therapeutic options i.e. whether there is only one or more than one therapeutic option and 15 related to the patient i.e. the wish of the patient to participate.
Factors and levels included in the study
Reason for the consultation
• Mild disease
• Severe disease
Time of negative consequences
• Immediate negative
• Negative in the future
Time until treatment should be started
• Treatment should be started immediately
• For the beginning of treatment there is no time pressure
Number of therapeutic options
• One therapeutic option
• More than one therapeutic option
Adverse effects of the treatment
• Treatment(s) is (are) well tolerated
• Treatment(s) could have severe adverse effects
• Treatments are partly well tolerated, may partly have severe adverse effects
Scientific evidence for the effectiveness of the treatment
• Good evidence for effect
• No evidence for Effect
• Good evidence for one/some treatment option/s, no evidence for other treatment option/s
Wish of the patient to participate
• The patient wishes to participate
• The patient does not wish to participate
To construct vignettes, for each factor specific levels have to be defined. Few levels are to be preferred in order to keep the number of possible vignettes small . Therefore we did not include more than 3 levels for each factor. In factors like "time until treatment should be started" we introduced only two levels, i.e. "immediate treatment necessary" vs. "no time pressure". Table 1 gives an overview of all factors and their levels finally included in our study.
The principle of a factorial survey is to combine all levels of one factor with all levels of the other factors. Therefore, for writing a vignette, one level of a factor is randomly combined with one level of all other factors, resulting in a description of a specific situation.
Example of a survey vignette
A patient consults his/her physician because of a mild disease that doesn't affect him right now but could lead to negative consequences in the future. There is no time pressure for beginning the treatment. There are various therapeutic options. Treatments are partly well tolerated and may partly have severe adverse effects. There is good evidence for all therapies. The patient in general wishes to participate in medical decisions.
Example of a semi-concrete vignette
A patient consults his doctor because of mild elevated blood pressure that doesn't affect him right now but could lead to negative consequences in the future. There is no time pressure for beginning the treatment. The available therapeutic options are: a pill that could have adverse effects or a change of lifestyle that is well tolerated. There is good evidence for all therapeutic options. The patient in general wishes to participate in medical decisions.
To find out whether the abstract (see table 2) or the semi-concrete version (see table 3) will be better accepted, better understood and easier to rate for physicians as well as for patients, we will conduct a cognitive pretest based on think aloud techniques and cognitive probes [34, 35].
The 5-point scale for rating preference for participation in treatment decision making (adapted form Sutherland et al.)
Please check the statement that best describes what you belief would be ideal:
• The doctor should make the decision using all that is known about the treatment
• The doctor should make the decision but strongly consider the patients opinion
• The doctor and the patient should make the decision together on an equal basis
• The patient should make the decision but strongly consider the doctor's opinion
• The patient should make the decision using all he knows or learns about the treatment
There is no consensus on how many factors can be included in a factorial survey . Beck and Opp  conclude that subjects don't have any difficulties with vignettes up to 6 dimensions, whereas Taylor  points out that more than twenty are possible. Nevertheless, most of the studies on factorial surveys have not used more than 8 factors [29, 30, 33]. Considering the capacity of the human's working memory, having difficulties to capture more than 6 to 9 information units at the same time , it makes sense to limit the number of factors included in the vignettes. Therefore, in our main study we will include only the 7 factors mentioned above.
Each participant will receive 11 vignettes for rating. Generally speaking, the number of vignettes for each participant depends on the time available, on the difficulty and complexity of the vignettes and on statistical considerations . Previous studies used various numbers of vignettes given to each subject, ranging from only 3  up to 40  and even more in older investigations . Beck and Opp  suggest that when there is no antecedent survey of a specific issue and therefore no information on how many vignettes could be tolerated, a number of 10 to 20 vignettes is reasonable. Furthermore, the number of vignettes should be higher than the number of factors and 10 vignettes seems to be the statistically best solution (C. Sauer, University of Duisburg-Essen, personal communication). Under these considerations and due to the complexity of our seven-factor survey, 11 vignettes per participant seem a reasonable choice to us. However, the ideal number of vignettes given to each subject is an additional aspect that will be tested in the pretest.
The vignettes that will be included in the study will be randomly chosen from all possible vignettes. For vignettes selection, two statistical methods are mentioned in the literature . "Random designs" randomly select the desired number of vignettes from all possible vignettes. The disadvantage of this method is that main and interaction effects are confounded in the statistical analysis . In "quota design" a planned selection of the vignettes is suggested to reduce this source of error. Unfortunately, quota designs in complex survey studies including many factors and different numbers of factor levels are computationally demanding and reliable statistical solutions are not available. Furthermore, quota designs are not applicable, if factor combinations have to be excluded from the study for logical reasons like in our design . We therefore chose the random selection method.
Supported by a computer program 40 sets of vignettes will be randomly drawn with replacement from the pool of possible vignettes. The number of 40 results form a calculation based on the total number of vignettes, the expected sample size and the number of vignettes that will be given to each participant . All 40 sets will be available for all three study groups (see "Participants"). Distribution will be at random within groups.
To our knowledge there are no well established power analysis methods for hierarchical models in factorial survey study designs. Thus, we decided to use conservative ANOVA for fixed effects, special, main effects and interactions . The sample size was calculated assuming an effect size of 0.25. Defining the single vignette as the unit of analysis, we estimated that a minimum number of 438 observations would be required to detect an effect of 0.25 with a power of 1-β = 0.95. Since one subject has to rate 11 vignettes, 40 subjects would be necessary. As we don't have estimations for intracluster correlation coefficient (ICC) and in order to compensate for drop outs and missing data, we aim at a sample size of 100 persons.
This results in a total number of 300 participants who will be included in the study: 100 general practitioners (GPs), 100 patients and 100 members of self-help groups.
We decided to include GPs because in Germany the general practitioner is the first person in contact with the patient. He is the one who guides the patient through the health care system. Usually, medical treatment is coordinated in a GP's surgery. As a consequence of the special relationship between GP and patient, most important decisions in the medical encounter are made in this context. For the study we will recruit members of our regional GP network as well as attendees of national conferences.
We plan to distinguish between the "usual" patient seen in general practice and members of self-help groups because we expect the latter to be more active in the decision making process. Patients will be recruited in surgeries. Members of self-help groups will also be contacted resorting to self-help group nets.
Patients are included, if they are 18 years and older, able to read German and able to give informed consent. We try to cover different educational levels and age groups, balanced for gender. Patients requiring emergency care are excluded.
Material given to the participants
Before receiving the vignettes, all participants will be given written information about the study and will have to sign an informed consent form, approved by the local research ethics committee, University of Marburg. Participants will then fill in the survey questionnaire. After asking for socio-demographic data and giving instructions 11 vignettes will be presented with the response format described above. If the participants have any questions a member of the research team will be present.
For our analysis we will use mixed models to estimate the effects of factors and interactions. Factors will be coded as dummy variables. In addition, we will examine differences between the three groups (physicians, practice patients and members of self help groups). Furthermore, we will investigate the influence of participants' demographic characteristics.
Linear models based on Ordinary Least Squares (OLS) techniques, e.g. analysis of variance or multiple regression, cannot be used for analysis since they would assume independence of all judgements . However, responses in factorial surveys are clustered in subjects . Therefore, hierarchical models based on maximum likelihood estimators seem to be more appropriate for this design. Hox et al.  demonstrated that choice between OLS-estimation and hierarchical/mixed models is critical since there are large calculational differences between the two solutions.
The findings of this factorial survey study will reflect normative values related to medical decision making. They will reveal the effect of situational variations on consultation style preferences.
The results might help physicians to adapt their consultation style not only to individual patients but also to individual treatment situations. Future implementation of SDM and decision aids might be more successful than in the past when communication patterns are tailored to specific situations.
Furthermore, we hope to encourage researchers of the effects of SDM to incorporate situational factors into their study designs.
We believe that the factorial survey will become a much more frequently used study design. A pure top-down approach to meso- and macro-level-decisions in health care is becoming less acceptable. In most instances many stakeholders are involved ranging from patients to providers, payers and governments, to commercial manufacturers of drugs and devices. Factorial survey designs can make an invaluable contribution, since opinions, interests and values of multiple stakeholders are systematically assessed to inform policy decisions in health care.
This study is supported by the German Federal Ministry of Education and Research (ID: 01GK0701). The research protocol was approved by the local research ethics committee, University of Marburg.
- Joosten EA, DeFuentes-Merillas L, de Weert GH, Sensky T, Staak van der CP, de Jong CA: Systematic review of the effects of shared decision-making on patient satisfaction, treatment adherence and health status. Psychother Psychosom. 2008, 77: 219-226. 10.1159/000126073.View ArticlePubMedGoogle Scholar
- Floer B, Schnee M, Bocken J, Streich W, Kunstmann W, Isfort J, et al: [Shared decision making]. Dtsch Med Wochenschr. 2004, 129: 2343-2347. 10.1055/s-2004-835265.View ArticlePubMedGoogle Scholar
- Levinson W, Kao A, Kuby A, Thisted RA: Not all patients want to participate in decision making. A national study of public preferences. J Gen Intern Med. 2005, 20: 531-535. 10.1111/j.1525-1497.2005.04101.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Strull WM, Lo B, Charles G: Do patients want to participate in medical decision making?. JAMA. 1984, 252: 2990-2994. 10.1001/jama.252.21.2990.View ArticlePubMedGoogle Scholar
- Charles C, Gafni A, Whelan T: Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med. 1997, 44: 681-692. 10.1016/S0277-9536(96)00221-3.View ArticlePubMedGoogle Scholar
- Griffin SJ, Kinmonth AL, Veltman MW, Gillard S, Grant J, Stewart M: Effect on health-related outcomes of interventions to alter the interaction between patients and practitioners: a systematic review of trials. Ann Fam Med. 2004, 2: 595-608. 10.1370/afm.142.View ArticlePubMedPubMed CentralGoogle Scholar
- O'Connor AM, Stacey D, Rovner D, Holmes-Rovner M, Tetroe J, Llewellyn-Thomas H, et al: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2001, CD001431.Google Scholar
- Scheibler F, Janssen C, Pfaff H: [Shared decision making: an overview of international research literature]. Soz Praventivmed. 2003, 48: 11-23.View ArticlePubMedGoogle Scholar
- Stewart M, Brown JB, Weston WW, McWhinney IR, McWilliam C, Freeman TR: Patient-centered medicine: transforming the clinical method. 2003, Abingdon: Radcliffe Med. PressGoogle Scholar
- Vick S, Scott A: Agency in health care. Examining patients' preferences for attributes of the doctor-patient relationship. J Health Econ. 1998, 17: 587-605. 10.1016/S0167-6296(97)00035-0.View ArticlePubMedGoogle Scholar
- Arora NK, McHorney CA: Patient preferences for medical decision making: who really wants to participate?. Med Care. 2000, 38: 335-341. 10.1097/00005650-200003000-00010.View ArticlePubMedGoogle Scholar
- Degner LF, Sloan JA: Decision making during serious illness: what role do patients really want to play?. J Clin Epidemiol. 1992, 45: 941-950. 10.1016/0895-4356(92)90110-9.View ArticlePubMedGoogle Scholar
- Frosch DL, Kaplan RM: Shared decision making in clinical medicine: past research and future directions. Am J Prev Med. 1999, 17: 285-294. 10.1016/S0749-3797(99)00097-5.View ArticlePubMedGoogle Scholar
- Elkin EB, Kim SH, Casper ES, Kissane DW, Schrag D: Desire for information and involvement in treatment decisions: elderly cancer patients' preferences and their physicians' perceptions. J Clin Oncol. 2007, 25: 5275-5280. 10.1200/JCO.2007.11.1922.View ArticlePubMedGoogle Scholar
- Janz NK, Wren PA, Copeland LA, Lowery JC, Goldfarb SL, Wilkins EG: Patient-physician concordance: preferences, perceptions, and factors influencing the breast cancer surgical decision. J Clin Oncol. 2004, 22: 3091-3098. 10.1200/JCO.2004.09.069.View ArticlePubMedGoogle Scholar
- McKinstry B: Do patients wish to be involved in decision making in the consultation? A cross sectional survey with video vignettes. BMJ. 2000, 321: 867-871. 10.1136/bmj.321.7265.867.View ArticlePubMedPubMed CentralGoogle Scholar
- Deber RB, Kraetschmer N, Irvine J: What role do patients wish to play in treatment decision making?. Arch Intern Med. 1996, 156: 1414-1420. 10.1001/archinte.156.13.1414.View ArticlePubMedGoogle Scholar
- Guadagnoli E, Ward P: Patient participation in decision-making. Soc Sci Med. 1998, 47: 329-339. 10.1016/S0277-9536(98)00059-8.View ArticlePubMedGoogle Scholar
- Whitney SN: A new model of medical decisions: exploring the limits of shared decision making. Med Decis Making. 2003, 23: 275-280. 10.1177/0272989X03256006.View ArticlePubMedGoogle Scholar
- Rossi HR, Anderson AB: The factorial survey approach. An Introduction. Measuring social judgments. The factorial survey approach. Edited by: Rossi PH, Nock SL. 1982, Beverly Hills/London/New Deli: Sage Publications, 15-67.Google Scholar
- Lauder W: Factorial survey methods: A valuable but under-utillised resarch method in nursing research. Nursing Times Research. 2002, 7: 35-43.View ArticleGoogle Scholar
- Steiner PM, Atzmüller C: Experimentelle Vignettendesigns in faktoriellen Surveys. Kölner Zeitschrift für Soziologie und Sozialpsychologie. 2006, 58: 117-146. 10.1007/s11575-006-0006-9.View ArticleGoogle Scholar
- Taylor BJ: Factorial surveys:unsing vignettes to study professional judgement. Birtish Journal of Social Work. 2005, 36: 1187-1207. 10.1093/bjsw/bch345.View ArticleGoogle Scholar
- Feitelson E: Consumer preferences and willigness-to-pay for water related residences in non-urban settings – a vignette analysis. Regional Studies. 2008, 26: 49-68. 10.1080/00343409212331346771.View ArticleGoogle Scholar
- O'Brian LJ, Rossi HR, Tessler RC: Wow much is too much? Popular definitions of alcohol abuse. The factorial survey approach. An Introduction. Edited by: Rossi PH, Nock SL. 1982, Beverly Hills/London/New Deli: Sage Publications, 235-252.Google Scholar
- Auspung K, Abraham M: Die Umzugsentscheidung von Paaren als Verhandlungsproblem. Eine quasiexperimentelle Überprüfung des Bargainig-Modells. Kölner Zeitschrift für Soziologie und Sozialpsychologie. 2007, 59: 271-293. 10.1007/s11577-007-0029-7.View ArticleGoogle Scholar
- Lauder W, Scott PA, Whyte A: Nurses' judgements of self-neglect: a factorial survey. Int J Nurs Stud. 2001, 38: 601-608. 10.1016/S0020-7489(00)00108-5.View ArticlePubMedGoogle Scholar
- Ludwick R: Clinical decision making: recognition of confusion and application of restraints. Orthop Nurs. 1999, 18: 65-72.PubMedGoogle Scholar
- Ludwick R, Zeller RA: The factorial survey: an experimental method to replicate real world problems. Nurs Res. 2001, 50: 129-133. 10.1097/00006199-200103000-00009.View ArticlePubMedGoogle Scholar
- Schwappach DL, Koeck CM: What makes an error unacceptable? A factorial survey on the disclosure of medical errors. Int J Qual Health Care. 2004, 16: 317-326. 10.1093/intqhc/mzh058.View ArticlePubMedGoogle Scholar
- Mazor KM, Fischer MA, Haley HL, Hatem D, Rogers HJ, Quirk ME: Factors influencing preceptors' responses to medical errors: a factorial survey. Acad Med. 2005, 80: S88-S92. 10.1097/00001888-200510001-00024.View ArticlePubMedGoogle Scholar
- Brown RL, Brown RL, Saunders LA, Castelaz CA, Papasouliotis O: Physicians' decisions to prescribe benzodiazepines for nervousness and insomnia. J Gen Intern Med. 1997, 12: 44-52. 10.1007/s11606-006-0006-2.View ArticlePubMedPubMed CentralGoogle Scholar
- Beck M, Opp K-D: Der faktorielle Survey und die Messung von Normen. Kölne Zeitschrift für Soziologie und Sozialpsychologie. 2001, 52: 283-306. 10.1007/s11577-001-0040-3.View ArticleGoogle Scholar
- Collins D: Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. 2003, 12: 229-238. 10.1023/A:1023254226592.View ArticlePubMedGoogle Scholar
- Kurz K, Prüfer P, Rexroth M: Zur Validität von Fragen in standardisierten Erhebungen. Ergebnisse des Einsatzes eines kognitiven Pretestinterviews. ZUMA-Nachrichten. 1999, 44: 83-107.Google Scholar
- Sutherland HJ, Llewellyn-Thomas HA, Lockwood GA, Tritchler DL, Till JE: Cancer patients: their desire for information and participation in treatment decisions. J R Soc Med. 1989, 82: 260-263.PubMedPubMed CentralGoogle Scholar
- Hussy W: Denken und Problemlösen. 1998, Stuttgart/Berlin/Köln: KohlhammerGoogle Scholar
- Jasso G: Whom shall we welcome? Elite judgemnts of the criterai for the selection of Immigrants. American Sociological Review. 1988, 53: 919-932. 10.2307/2095900.View ArticleGoogle Scholar
- Jasso G, Rossi HR: Distributive justice and earned income. American Sociological Review. 1977, 42: 639-651. 10.2307/2094561.View ArticleGoogle Scholar
- Dülmer H: Experimental plans in factorial surveys: Random or quota design. Sociological methods and research. 2007, 35: 382-409. 10.1177/0049124106292367.View ArticleGoogle Scholar
- Faul F, Erdfelder E, Lang AG, Buchner A: G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007, 39: 175-191.View ArticlePubMedGoogle Scholar
- Hox JJ, Kreft IGG, Hermkens PLJ: The analysis of factorial surveys. Sociological methods and research. 1991, 19: 493-509. 10.1177/0049124191019004003.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/260/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.