The representation of patient experience and satisfaction in physician rating sites. A criteria-based analysis of English- and German-language sites
© Reimann and Strech; licensee BioMed Central Ltd. 2010
Received: 10 September 2010
Accepted: 7 December 2010
Published: 7 December 2010
Information on patient experience and satisfaction with individual physicians could play an important role for performance measures, improved health care and health literacy. Physician rating sites (PRSs) bear the potential to be a widely available source for this kind of information. However, patient experience and satisfaction are complex constructs operationalized by multiple dimensions. The way in which PRSs allow users to express and rate patient experience and satisfaction could likely influence the image of doctors in society and the self-understanding of both doctors and patients. This study examines the extent to which PRSs currently represent the constructs of patient experience and satisfaction.
First, a systematic review of research instruments for measuring patient experience and satisfaction was conducted. The content of these instruments was analyzed qualitatively to create a comprehensive set of dimensions for patient experience and patient satisfaction. Second, PRSs were searched for systematically in English-language and German-language search engines of Google and Yahoo. Finally, we classified every structured question asked by the different PRS using the set of dimensions of patient experience and satisfaction.
The qualitative content analysis of the measurement instruments produced 13 dimensions of patient experience and satisfaction. We identified a total of 21 PRSs. No PRSs represented all 13 dimensions of patient satisfaction and experience with its structured questions. The 3 most trafficked English-language PRS represent between 5 and 6 dimensions and the 3 most trafficked German language PRSs between 8 and 11 dimensions The dimensions for patient experience and satisfaction most frequently represented in PRSs included diversely operationalized ones such as professional competence and doctor-patient relationship/support. However, other less complex but nevertheless important dimensions such as communication skills and information/advice were rarely represented, especially in English-language PRSs.
Concerning the potential impact of PRSs on health systems, further research is needed to show which of the current operationalizations of patient experience and satisfaction presented in our study are establishing themselves in PRSs. Independently of this factual development, the question also arises whether and to what extent health policy can and should influence the operationalization of patient experience and satisfaction in PRSs. Here, the challenge would be to produce a set of dimensions capable of consensus from among the wide range of operationalizations found by this study.
For questions about health and sickness in general, a large share of the information relevant to decision-making is publicly available through digital media (particularly the internet). Anyone can learn about the characteristics of certain illnesses (cause, symptoms, diagnostic criteria) and of the available medical measures (effectiveness, dosage, side effects) (e.g. [1, 2]). This information is more practically relevant if it facilitates the primary goals and ethical principles of medicine [3, 4]: (i) the welfare of patients, (ii) the respect for and promotion of patient autonomy and health literacy, and (iii) social justice.
Alongside information on illnesses and medical measures, it could also be relevant to the primary goals of medicine to acknowledge patients' satisfaction with physicians and provide information on this [5–7]. However, health services research has recently criticised the concept of patient satisfaction because of its inherent sources of bias . Patients, for example, can describe high levels of satisfaction at the same time as describing experiences that are suboptimal. Thus in light of the limitations of patient satisfaction research, a recent trend in health services research has favored questions about patients' experiences . Despite the great amount of active research and scientific publications in the field of patients' experience and satisfaction [7–9] there is only very little publicly available information on patients' experience and satisfaction concerning individual doctors or hospitals .
Physician rating sites (PRSs) are a new web-based source for peer-to-peer information on individual physicians. PRSs provide information about a physician's address, opening hours and certifications . Next to this factual information, a major objective of PRSs is to collect and present information about patients' experience and satisfaction with individual physicians and their practices. PRSs could, therefore, improve informed provider choice [11, 12] and are similar to other peer-to-peer consumer information websites that allow the rating and discussion of the experience and satisfaction with hotels, restaurants or technological devices. In contrast to the rather expert-driven approaches to information on patients' experience and satisfaction, such as the CAHPS Clinician and Group Survey , PRSs are a consumer-driven alternative.
So far, very little research has been done on PRSs [10, 13]. In contrast to this lack of research, there is controversial discussion about the impact of PRSs in several health systems. The British National Health Service (NHS) and the leading sickness fund within the German statutory health insurance have encouraged patients to review their physicians and hospitals [14, 15]. American and German physician organizations, including the American Medical Association (AMA) and the German Medical Association (BÄK), have been rather opposed to the development of PRSs, arguing, for example, that the identity of patients cannot be confirmed, reviews will be excessively negative, and physicians' responses will be hampered by confidentiality issues . To our knowledge, patient organizations were rather silent about their views on positive and negative consequences of PRSs.
Besides the technical and judicial questions surrounding PRSs, future discussion and decision making on PRSs also have to consider the content and rating options of PRSs. The practical relevance of information on patient experience and satisfaction - with respect to the achievement of the primary goals of medicine - depends significantly on how the complex construct of patient experience and satisfaction is represented in PRSs.
Our study examines, qualitatively and quantitatively, the extent to which PRSs currently represent patients' experiences and the construct of patient satisfaction as measured by research instruments.
The framework of dimensions representing patients' experience and satisfaction
An examination of the extent to which PRSs represent patients' experience and the construct of patient satisfaction requires a comprehensive catalogue of dimensions. Given the variety of instruments for measuring patients' experiences and satisfaction, which to some extent vary significantly in their exclusion and inclusion of individual dimensions, we decided to compile one single framework of dimensions on the basis of a systematic review of reviews of relevant measurement instruments. We conducted a systematic search in MEDLINE using the following search algorithm: ("Health Surveys"[Mesh] OR "Questionnaires"[Mesh]) AND ("Patient Satisfaction"[Mesh] OR "Patient Experience") AND "Review "[Publication Type]. We included all reviews that reviewed instruments for measuring patients' experience and satisfaction with resident or clinical doctors in general. We obtained the original versions of the potentially relevant measurement instruments mentioned in the reviews. We included all measurement instruments that (i) provided at least one dimension to measure patients' experience or satisfaction in general and (ii) reported information on reliability (re-test reliability, internal consistency) and validity. Measurement instruments on illness- or setting-specific patient satisfaction were excluded, for example [17, 18]. All bibliographies of the reviews and measurement instruments we included were examined for further relevant measurement instruments. We extracted dimensions for patients' experience and satisfaction from the instruments we included through a qualitative content analysis . After an initial reading, both authors individually and jointly organized all mentioned dimensions into conceptual relationships and higher order dimensions. After discussion both authors were able to reach a consensus about all discrepancies in reviewing and analyzing the instruments. The resulting set of 13 dimensions arranged into three subgroups produced the framework for the analysis of the PRSs.
Systematic search for physician rating sites
Operators and ranking of websites
Physician Rating Sites
Sites linking in
Owned by MDx Medical, Inc with headquarters in Lyndhurst, NJ. Comprised of a team of dedicated experts in the fields of healthcare, marketing, and database management.
Just average patients who happen to know how to make websites; not affiliated with any medical organizations.
Founded by Internet industry veterans from WebMD and Valicert; funded by Bessemer Venture Partners, Trinity Ventures, and Partech International.
Operated by the Medical Quality Enhancement Corp., founded by Steven R. Feldman, M.D., Ph.D.
No information on owner or sponsor found
No information on owner or sponsor found
DoctorTree.org; Encino, CA
No information on owner or sponsor found
Team of doctors and programmers who created FindaDoc to make the daunting and sometimes frustrating process of finding the best doctor as simple as possible
Owner and operator: Women's College Hospital
Built and operated by people that are free from ties to medical businesses, medical associations, hospitals, clinics, doctors, and even free from patient advocacy groups.
No information on owner or sponsor found
imedo GmbH, Berlin
jameda GmbH, München; Tomorrow FOCUS AG and FOCUS Magazine Publisher
Andingo Capital GmbH, Berlin; Funding by the German Ministry for Education and Research (BMBF) and the German Ministry for Economy and Technology (BMWi)
Comventure GmbH, Ludwigshafen
medführer Online-publisher, GmbH in cooperation with University of Trier
Helpster GmbH, Munich
ArztData GmbH, Hamburg
Endverbraucher Ltd. & Co., Fürth
Mediaseed GmbH, Stuttgart
Structured content analysis of physician rating sites
We used the 13 framework dimensions to classify each rating option or structured question provided by the different PRSs. Multiple classifications for one rating option were possible.
Instruments for assessing patients' experience and satisfaction
ABIM-10, American Board of Internal Medicine
CSQ, Consultation Satisfaction Questionnaire
CSS, Consumer Satisfaction Survey
"dialogue", consultation satisfaction questionnaire
National Clinical Audit Centre 
GPAS, General Practice Assessment Survey
National Primary Care Research a. Development Centre 
IPQ, Improving Practice Questionnaire
KPF, Kölner Patientenfragebogen
McKinley et al.
MISS-21, Medical Interview Satisfaction Scale
OPEQ, Outpatient Experiences Questionnaire
PDRQ-9, Patient-Doctor-Relationship Questionnaire
Van der Feltz-Cornelis 
PEQ, Patient Experience Questionnaire
PSQ, Patient Satisfaction Questionnaire
REPERES-60, Recherche Evaluative sur la Performance des Réseaux de Santé
SOSQ, Seattle Outpatient Satisfaction Questionnaire
ZAP - Zufriedenheit in der ambulanten Versorgung - Qualität aus Patientensicht
Characteristics of encounter between doctor and patient
Doctor-patient-relationship and Support
Information and advice
Bedside manner (caring)/Spending enough time with me
Accuracy in diagnosing a problem/following up as needed after my visit
Helpfulness (Is the doctor approachable and nice? Is he rude, arrogant, or just plain mean? Does he have a good bed-side manner?)
Personal skills: Friendly and approachable
Takes time to answer questions. (Do you feel rushed while talking? Does the doctor show concern for you? Do you have to repeat your case history every time?
Knowledge and skill: Diagnoses problems accurately, recommends best treatment. (Does your doctor have up-to-date medical skills? Can you get recommendation to a specialist easily?)
friendliness and caring attitude/The extent that the doctor includes you in decisions about your care and treatment/How well MD follows-up on any problems or concerns you have
thoroughness of exam or check-up/Ability to get all of the care for your health problem or illness at this clinic.
How well all questions were answered/instructions on how to take care of your illness or health condition
Listening to you
Knowledge and professionalism
Bedside manner (Do they make you feel comfortable)
Knowledge of medicine
Personal attention during visit/shows caring & compassion
Ability to communicate
Willingness to make referrals/quality of referrals
Explanation/Coordination of medications
Bedside manner (attitude and conduct of a physician in the presence of a patient)
Medical knowledge (how knowledgeable the doctor is in his or her field)
Privacy - upholding the need for a confidential and private atmosphere
Interpersonal factors - was the doctor competent in your estimation
Interpersonal actors - did you feel well-advised by the doctor? (Information on preventative exams, preventative options, diagnosis, prognosis, therapy, written information, education)
Friendliness (the doctor's conduct, openness, was the doctor empathetic and considerate)
Did the doctor listen to you enough? Were your questions answered thoroughly and patiently?
Relation of trust - did you feel yourself in good hands with this doctor?
Was the doctor's diagnosis later confirmed? Did he follow up with the appropriate treatment?
Education about illness and treatment/education about illness and treatment (were the explanations understandable to you?)
Satisfaction with the understanding, empathy, humaneness, being taken seriously, treated as a person; being involved in the decisions (satisfaction with support and advice)
Do you trust this care professional?
Satisfaction with professional competence (thorough and diligent medical treatment, readiness to refer to other specialists)
Satisfaction with information and advice
Doctor's friendliness, timeframe
Type and extent of examination
Doctor's explanations, explanation of the results
Doctor-patient relation (did the doctor take enough time; how understandable were your instructions after leaving the office)
Medical secretary (satisfaction with the atmosphere of privacy)
Medical services (satisfaction with the professional abilities of the doctor)
Medical services (satisfaction with the information and advice, involvement in the treatment)
Doctor (patient involvement in choice of alternative therapies, friendliness, readiness to help, respectfulness)
Doctor (maintenance of the atmosphere of privacy)
Doctor (thoroughness of examination, professional competence)
Doctor (comprehensibility of explanation)
I find the relation between my treatment and the ensuing costs to be very good
All of my questions were answered by the doctor and his/her team to my satisfaction
Organizational aspects of medical practice
Medical and technical facilities
Office organization, Waiting time
Ease in getting an appointment
Waiting time during a visit
Courtesy and professionalism of office staff
How long does the doctor keep you waiting?
How is the service and helpfulness of the doctor's staff?
Availability - Appointments are available easily and on a timely basis. (Is there good after-hours support? Can the doctor be reached over phone? How about availability by email?)
Punctuality - Is reasonably punctual with appointments. (How is the wait time in the office? Has the doctor cancelled appointments?)
Office Staff - Office staff is courteous and professional. Do your phone messages get communicated to the doctor? Is the staff sensitive to you and your condition?
Ability to get all of the care for your health problem or illness at this clinic.
The ability to see the health care provider you wanted to see at this clinic/Getting the advice or help you needed after office hours.
Patient convenience such as ample parking and location of office
Getting your test results back in a timely manner./About how many days did you have to wait to get an appointment to see the doctor?/After arriving at the office, how many minutes did you wait before seeing the doctor/About how many minutes did the doctor spend with you in your most recent visit
The friendliness and courtesy of the office staff.
Punctuality/approximate time spent in waiting room
Time spent in waiting room
Cleanliness of office
Waiting room time/Returns calls in a timely fashion/Accuracy of billing
Professionalism of staff
Office Setting (This is a measurement of the appearance and cleanliness, of a doctor's facilities)
Wait Time (How long you have to wait to get an appointment and how long you have to wait in the doctor's office.)
Office Staff (How well the office staff meets your needs as a patient)
Time you waited to get an appointment after requesting one
Cost/Time you waited in the waiting room
The Office staff/The nurses
Punctuality, post appointment action
The secretary/administrative assistant
Office facilities: no computers
Availability of appointments: how quickly did you get an appointment?
Office organization, information on the organizational structure of the office
Punctuality, how long you had to wait in the waiting room
Staff, how friendly and well-organized were the office staff in making appointments and in the office
Waiting times for an appointment (optional), availability by telephone (optional), public accessibility, parking spaces (optional)
Office facilities (optional); Entertainment in the waiting room (optional)
Waiting times in the office (optional)
Attentiveness and friendliness of the entire personnel (optional)
Satisfaction with the waiting time for an appointment, how long did you wait?
Satisfaction with the atmosphere
Satisfaction with the waiting times, how long did you wait; house visits
Satisfaction with the friendliness of the office staff
Office facilities (up-to-date)
Office organization (making appointments)
How did you find the office facilities (well-kept waiting room, tidy examination room)
Office organization (office waiting time, attended to during the waiting time)
Appointment (were you satisfied with the time needed to get an appointment with this doctor), getting there (parking spaces, directions, accessibility), doctor-patient relation (is the doctor available by telephone)
Reception (satisfaction with the diversions in the office); Office premises (service, ambiance, cleanliness and hygiene)
Reception (satisfaction with the organization and waiting times); Medical secretary (access to patient files)
Reception (friendly greeting); Medical secretary (professional competence and friendliness of the team)
Personal assessment (how satisfied were you with how you were treated - thoroughness, cleanliness)
Personal assessment (how satisfied were you with the waiting times)
Personal assessment (how satisfied were you with the friendliness)
Organisation and service (availability by telephone, available office hours, availability of appointment, accessibility with public transportation, parking)
Appearance (equipment, hygiene, signs, clear arrangement)
Organization and service (waiting time in the office)
Personnel (friendliness, readiness to help, respetfulness, clarity of information, preservation of privacy)
The appointments were always kept
The premises and the facilities of the office left a very good impression on me
There were hardly waiting times
I was always treated with friendliness and consideration by the office staff
Overarching assessment categories
Success of outcome
Willingness to recommend the doctor
Overall, what is your opinion of this doctor?
How did his treatments work for you?
Your treatment success
Your patient satisfaction score rating for this visit:
Overall quality of care
Would you recommend this doctor?
9 Find a doc.com
Patient satisfaction (This is your overall level of satisfaction with the doctor and his or her treatment)
recommend this doctor to a family member or close friend
How would you rate this doctor and the office overall?
Were you better after the treatment?
Satisfaction with the treatment
Assessment of the quality of treatment in general
Would you recommend this doctor?
Readiness to recommend the doctor
Medical services (satisfaction with the results of the treatment)
Medical assessment (How do you subjectively judge the success of the treatment?)
Recommendation (to your best friend)
I find the result of the treatment to be very good
I would feel very good recommending this doctor to friends and relatives
A total of 12 English-language and 9 German-language PRSs were identified (see tables 3, 4, and 5) [last checked in April 2010]. No PRS represented all 13 dimensions of patients' experience and satisfaction with its structured questions. The number of dimensions for patients' experience and satisfaction represented in PRSs range from 3 (checkMD.com, doctortree.org) to 11 (Jameda.de, Docinsider.de). For data about the traffic of PRSs see table 1. While the 3 most trafficked English-language sites (vitals.com, RateMD.com, vimo.com) only represent between 5 and 6 dimensions (see tables 3, 4, and 5) the 3 most trafficked German-language sites (Imedo.de, Jameda.de, Docinsider.de) represent between 8 and 11 dimensions.
Among the dimensions most frequently represented in PRSs are the less complex dimensions such as office organization/waiting time (in 21 PRSs) and office staff (in 17 of 21 PRSs). On the other hand quite complex and thus diversely operationalized dimensions of patient experience and satisfaction such as professional competence (in 18 of 21 PRSs) and doctor-patient relationship (in 16 of 21 PRSs) are also often found on these sites. However, other dimensions that allow users to rate important aspects of the encounter between doctor and patient were rarely represented. For example, only 4 of 12 English and 1 of 9 German PRSs provided structured questions about physicians' communication skills. Also only 3 of 12 English PRSs but 8 of 9 German PRSs provided structured questions concerning how well physicians offer information and advice.
The following sections show how the dimensions of patients' experience and satisfaction are represented both by the research instruments and in the PRSs. We will begin by qualitatively describing the operationalization of each dimension by the research instruments. This operationalization formed the basis for the classification of rating options in the PRSs that we will present subsequently. To facilitate reading we separated the 13 dimensions of patients' experience and satisfaction into 3 subgroups (see tables 3, 4, and 5) that aim to group together (i) characteristics of the encounter between doctor and patient (e.g. doctor-patient relationship, communication) in table 3 (ii) organizational aspects of the medical practice (e.g. equipment, personnel, organization) in table 4 and (iii) overarching assessment categories (e.g. general satisfaction) in table 5. Numbers in parentheses indicate the corresponding PRSs specified in tables 3, 4 and 5.
Characteristics of the encounter between doctor and patient
Patients' experience and satisfaction research instruments include the personal skills of the doctor such as patience, taking the patient seriously, being friendly, caring, trustworthy, diligent, empathetic and humane. Additional characteristics, such as seeing the patient as an equal partner or as a person and not as a number, are often classified under patient-centeredness. Additionally the quality of the doctor-patient relationship has been operationalized by means of doctor behavior such as showing interest in the patient, maintaining a respectful manner, and allowing the patient to ask necessary questions as well as supporting the patient through his or her mental, social and bureaucratic difficulties.
PRS: 16 PRSs (76%) allow for an explicit assessment of certain aspects of the doctor-patient relation as described above (1-10, 12, 14-17, 19). Three English-language PRSs use the term "bedside manner" as a catch-all term for a particular behavior on the part of the doctor, but explain it differently - one more descriptively as the "attitude and conduct of a physician in the presence of a patient" (9) and another in clear normative terms as "do they (physicians) make you feel comfortable" (7). Other criteria that we correlated with the dimension of doctor-patient relation were helpfulness (2, 10, 12), approachability and friendliness (2-5, 14-16, 19), spending enough time with the patient (1, 17), shows caring or concern (8, 12), and shows compassion (8).
Patients' experience and satisfaction research instruments include the ability to listen, to communicate diagnoses to the patient appropriately, to inform the patient about decisions and to (transparently) involve the patient in the process of examination and treatment. Even if there is some concrete overlap between the dimensions of communication and doctor-patient relation, we find it possible and productive to maintain a distinction. The Association of American Medical Colleges (AAMC) also emphasizes this distinction between communication skills and interpersonal skills , whereby the communicative skills include above all effective listening, appropriate questioning and the provision of information. Interpersonal qualities are seen more in the establishing of a trusting relation .
PRS: Five sites (24%) asked explicitly about this dimension as the "ability to communicate" (8), "understanding" (12), or "listening to you" (6). The criteria of "takes time to answer questions", "feel rushed while talking" (3) and the question of sufficient conversational time (14) also represent the doctor's communicative skills.
Patients' experience and satisfaction research instruments include questions about whether the patient feels trust in the particular doctor, whether an atmosphere of privacy obtains or whether the patient can open up to entrust the doctor with all of the necessary information.
PRS: Six sites (29%) ask about this dimension. The explanations hardly differ at all; they usually concern the preservation of the sense of privacy (13, 17, 19) and then ask quite unspecifically about whether the patient trusts the particular doctor, for example with the question of whether one feels oneself in good hands (9, 14) or "Do you trust this care professional?" (15).
Patients' experience and satisfaction research instruments include the doctors' knowledge of his or her own (professional) limitations (e.g. prompt referral, collaboration with other doctors, admitting his or her own errors) and the aspect of diligence (e.g. investigating all possible causes, correct diagnosis, thorough examination, responsibility). This dimension also includes knowledge of all aspects of the medical therapy and the appropriateness of the treatment (e.g. medications without or with only minor side effects, no excess treatment, individualized medical treatment, no doubled examinations, awareness of price).
PRS: 18 sites (86%) ask for an assessment of this dimension (1-10, 12-17, 19, 20). The questions range from unspecified aspects such as "knowledgeable" or "Knowledge and Professionalism" (2, 5-7, 9, 10, 12, 17, 19) to more comprehensive descriptions of professional competence such as "Was the doctor's diagnosis later confirmed? Did he follow up with the appropriate treatment?" (14) or "Does your doctor have up-to-date medical skills? Can you get a recommendation to a specialist easily?" (3).
Information and advice
Patients' experience and satisfaction research instruments include the content and scope of what is communicated. Hence this dimension particularly concerns how well and how transparently the patient is informed during the medical treatment. This includes the information given (e.g. understandable information about the causes and process of the illness and the side effects of diagnosis and treatment) and advice (e.g. information on self-help groups or nutritional advice).
PRS: Eleven sites (52%) covered this dimension in their assessment (4, 5, 8, 13-17, 19-21). Two of the sites only named the dimension as "informative" (5) and "advice" (21), whereas the other assessment templates provided the user with more thorough explanations, for example "Information on preventative examinations, preventative options, diagnosis, therapy, written information, education" (13) or "All of my questions were answered by the doctor and his or her team to my complete satisfaction. I felt that I was advised very well." (20).
Organizational aspects of the medical practice
Medical and technical facilities
Patients' experience and satisfaction research instruments include the technical facilities of the physician's practice (or the hospital).
PRS: Four of the sites (19%) include this dimension in the assessment (4, 11, 13, 16). This dimension is represented by questions such as "How did you find the technical equipment of the medical practice (technically up-to-date)?" (16) or "Ability to get all of the care for your health problem or illness at this clinic" (4).
Patients' experience and satisfaction research instruments include the doctor's availability by telephone outside of visiting hours, the accessibility of the office or clinic (e.g. wheel-chair accessible, directions by public transportation are available, parking nearby, etc.) as well as the possibility of house visits. An acceptable waiting period for an appointment and an arrangement of appointments suitable to the patient are also included.
PRS: More than half of the sites (12; 57%) asked about aspects of accessibility/availability (1, 3, 4, 8, 11, 13-17, 19, 20). Often only one representative criterion was named such as "availability" or "arrangement of appointments" (1, 8, 16, 20). More concrete questions concerned the possibility of house visits, the waiting time for an appointment, the possibility of reaching the doctor by telephone outside of visiting hours or per email, the allocation of appointments, and available parking.
Patients' experience and satisfaction research instruments include the atmosphere in the office or clinic (team spirit, attractive waiting room, cleanliness), separate changing rooms, or play-areas for children.
PRS: This dimension was included by 12 sites (57%). It was represented by "cleanliness of the office" (8), "entertainment in the waiting room" (14) and "office setting" (9) or more specifically by the hygiene or atmosphere of the office or the parking situation (4, 13, 15-21).
Patients' experience and satisfaction research instruments include time management (brief waiting periods within the office, good office organization), customer-orientation (emphasis on the patient, no interrupting phone calls, equal service for all patients, shorter waiting times for emergencies) and service (priority for mothers with children, referral given on the same day, copies of all test results).
PRS: This dimension is included in all of the rating sites. There are hardly any differences in the weight given individual aspects of office organization: waiting times and promptness are the primary concerns. One site goes into this point in greater detail: "Getting your test results back in a timely manner./After arriving at the office, how many minutes did you wait before seeing the doctor?/About how many minutes did the doctor spend with you in your most recent visit?" (4). On English-language websites this category also sometimes includes an estimate of the costs of the treatment (8-11).
Patients' experience and satisfaction research instruments include the competence and friendliness of the staff as well as their interaction within the team.
PRS: 17 of the sites (81%) cover this aspect. Several sites ask about this without any further explanation, particularly concerning the friendliness, helpfulness and professionalness of the staff (1-6, 8, 11-15, 17-20). Four sites provide examples of how the friendliness of the staff could be demonstrated, e.g.: "How friendly and well-organized were the office personnel in the office and in making appointments" (13), "Base your opinion on your experience making an appointment, how you are greeted for an office visit, and how well the office staff meets your needs as a patient" (9).
Overarching assessment categories
Success of outcome
Patients' experience and satisfaction research instruments include assessments of effectiveness (e.g. help by the doctor, effective medications), the suitability of the therapy relative to the diagnosis, the disappearance of symptoms, the amelioration of the complaint, and the increase in functional ability and quality of life.
PRS: Nine of the sites (43%) ask explicitly about the success of the treatment, for example with questions such as: "Were you better after the treatment?" (14), "How would you subjectively assess the success of the treatment?" (18), or "How did his treatments work for you?" (2). All 9 PRSs (2, 4, 10, 12, 14, 15, 17, 18, 20) assess this dimension by asking about the subjective treatment success; there are no detailed descriptions of the reduction of specific symptoms, ability to go back to work, increase in quality of life, etc.
Patients' experience and satisfaction research instruments include the aggregate assessment as a general summary of all previous assessments.
PRS: 13 of the sites asked about general satisfaction (62%), for example using terms such as "overall satisfaction" (1, 6, 8, 9, 14, 15) or by asking for a total rating/assessment (4, 10, 11, 16, 18, 19, 21). No site said anything in greater detail about this total assessment. The visual presentation of the assessment of a doctor (ranging from check-marks to stars, plus points, and other symbols) or the option of writing free commentary might implicitly represent general satisfaction. On this broader definition the dimension general satisfaction would be represented on all sites.
Willingness to recommend the doctor
Patients' experience and satisfaction research instruments include patients' willingness to recommend a doctor and generally describe this dimension as: would the patient be able to recommend this doctor (or this clinic) to friends and relatives?
PRS: Only six sites (29%) explicitly posed this question to their evaluating users (8, 9, 15, 16, 19, 20). This dimension is similar to the representation of general satisfaction: one would expect a high level of satisfaction to correspond with a high willingness to recommend the doctor, especially as this is precisely the point of the sites: to allow people to share their experiences with those looking for a suitable doctor and to offer their own experiences to others who might find it useful and helpful.
Possibility of free commentary
86% of the sites (18) allow for users to write free commentary (except 5, 13, 17). One site only offers assessment by commentary without indicating or asking about any other dimensions, and was thus excluded from the criteria-based analysis. "The operators of this site assume that "rating" doctors doesn't give you the kind of information you need. [...] You can't make a "best fit" healthcare choice when all you have is a number between 0 and four" ("Our Philosophy" ).
The 21 PRSs examined here show a clearly heterogeneous representation of the different dimensions of patient experience and satisfaction as viewed from a quantitative and qualitative perspective. Our quantitative findings show that the most trafficked English-language PRSs currently only represent between 5 and 6 of the 13 dimensions that can play a role in assessing patient experience and satisfaction. However, the most trafficked German-language PRSs represent between 8 and 11 of the 13 dimensions.
Findings from the qualitative analysis demonstrate that the most frequently represented dimensions in PRSs include diversely operationalized ones such as professional competence and doctor-patient relationship. To assess professional competence, for example, PRSs provide heterogeneous questions such as: "Does your doctor have up-to-date medical skills", "Was the doctor's diagnosis later confirmed? Did he follow up with the appropriate treatment?", "Thoroughness of examination" and others..
The PRSs identified using our search dimensions are the sites that a potential user would run into in his or her search. Nevertheless, we cannot exclude the possibility of other PRSs not discovered by our search algorithm. However, we do not aim at completeness; instead we interpret the sites examined here as a faithful representation of existing sites and assume that other sites not examined will deviate minimally.
We only examined the structured questions asked by the PRSs and not the actual patient reviews. Therefore we did not include the content of open and narrative feedback sections which may represent additional specifications of the 13 dimensions for patients' experience and satisfaction.
At the current state of research, we cannot give evidence-based recommendations about which and how many dimensions of patients' experience and satisfaction need to be represented in PRSs to best improve the primary goals of medicine [3, 4]. There may be plenty of arguments against the representation of all dimensions in PRSs. However, PRSs should not be confused with other well-investigated expert initiatives that measure physician performance and quality of care [6, 7, 28]. For example, a rather broad representation of dimensions of patients' experience and satisfaction could have a positive impact on the interactive functions of PRSs and thus on the three levels of patients' health literacy [29, 30]: Aside from the direct recommendations of individual physicians PRSs bear the potential to improve users' critical reflection about what aspects could and should be considered prior to choosing a doctor. It seems possible, therefore, that PRSs increase the users' health literacy at the interactive level, since it is precisely the exchange of information that is considered the essential point of these sites. Concerning the critical level of health literacy, one could also expect (in the best case) a similarly positive development. In particular the possibility of evaluating a specific dimension as more or less helpful and the possibility of free narrative commentary could train users to individually filter out what is important - possibly leading the patients to pay more attention to the doctors' competence in shared decision-making and less attention to practices' atmosphere. However, the major precondition for PRSs to achieve these objectives would be to provide possibilities for open narrative commentaries next to structured rating options and to facilitate peer-to-peer communication. Needless to say, PRSs also bear the potential to negatively influence the three levels of health literacy.
Our findings provide the starting point for dealing with another potential impact of PRSs. It also seems plausible to assume that in the future complex constructs such as 'quality of care', 'physician performance', or 'patient satisfaction/experience' will still not be operationalized through consensus on a single gold standard, nor that such a gold standard will be used in all PRSs. Thus the consumer-driven, peer-to-peer operationalization of dimensions such as professional competence and doctor-patient relationship in PRSs could significantly shape the social image of the doctor and the self-understanding on the part of both doctors and patients. An important question in the research accompanying PRSs, therefore, would be: which of the current operationalizations of patients' experiences and satisfaction in general and of professional competence and doctor-patient relation in particular are establishing themselves in PRSs?
Independently of the factual development examined here, the question also arises whether and to what extent health policy can and should influence the operationalization and assessment of patients' experience and satisfaction in consumer-driven PRSs (true Web 2.0 application). One could imagine certifications from professional societies and public institutions entailing not just formal and legal standards but also specifications for a suitable representation and operationalization of patients' experience and satisfaction. Here the concrete difficulty would be to work out a set of dimensions capable of consensus from among the wide range of operationalizations shown by this study. From an ethical viewpoint such certifications would be more legitimate if they arose through a transparent process allowing the participation by various stakeholders (including consumer representatives). This process would have to include discussions of which characteristics of doctors and medical practices are crucial in order to appropriately inform consumers in accordance with the primary goals of medicine (see above).
The practical relevance of all issues we discussed in the previous paragraphs only increases with a more widespread use of PRSs. At present, the use of PRSs is rather limited. Recent work on the state of PRSs found that reviews were scarce, and when present, most were positive concerning the overall satisfaction with the specific physician . For 300 physicians, only 66 written patient narratives across 33 sites could be identified . However, there are many reasons that argue in favour of a more widespread use of PRSs in the near future. For example, the so-called Facebook or MySpace generation has been especially socialized with the internet. When this generation reaches the age in which they are increasingly interested in health questions and thus doctors, it is likely that the internet will also play a significant role in their decisions (see among other things the rapidly growing number of participants in patient communities ).
In the last decade several national health systems have adopted public reporting instruments allowing consumers to make explicit comparisons between the performances of health care providers or health plans in order to make an informed choice . While the aims of public reporting approaches - increasing public accountability, supporting consumer choice and finally improving quality of care - are comparable with the aims of PRSs, they currently lack patient experience and satisfaction information with respect to individual physicians. PRSs might be interesting esspecially for countries that do not follow gatekeeper models and therefore have free physician choice (e.g. Germany).
In light of the mentioned trend towards an increasing practical relevance of PRSs, these new websites bear the potential to influence the public health and health literacy. Future ethical and policy analyses should take these various influences into account and explicitly weigh them before drawing any conclusions on the form and content of PRSs .
We thank the reviewer Marjan Faber for many helpful comments that we acknowledged in revising an earlier version of this paper.
- NIH: National Institute for Health and Clinical Excellence (NICE). [http://www.nice.org.uk]
- Cochrane Collaboration. [http://www.cochrane.org]
- Beauchamp TL, Childress JF: Principles of Biomedical Ethics. 2008, New York, Oxford: Oxford University Press
- ABIM Foundation, ACP-ASIM Foundation, and European Federation of Internal Medicine: Medical professionalism in the new millennium: a physicians' charter. Lancet. 2002, 359 (9305): 520-2. 10.1016/S0140-6736(02)07684-5.View Article
- Elwyn G, Buetow S, Hibbard J, et al: Measuring quality through performance. Respecting the subjective: quality measurement from the patient's perspective. BMJ. 2007, 1021-2. 10.1136/bmj.39339.490301.AD. 335
- Corrigan JM, Burstin H: Measuring quality of performance: Where is it headed, and who is making the decisions?. J Fam Pract. 2007, 56 (10 Suppl A): 4A-7A.
- Browne K, Roseman D, Shaller D, et al: Measuring patient experience as a strategy for improving primary care. Health Aff (Millwood). 2010, 29 (5): 921-5. 10.1377/hlthaff.2010.0238.View Article
- Jenkinson C, Coulter A, Bruster S, et al: Patients' experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care. 2002, 11 (4): 335-9. 10.1136/qhc.11.4.335.PubMed CentralView Article
- Salisbury C, Wallace M, Montgomery AA: Patients' experience and satisfaction in primary care: secondary analysis using multilevel modelling. BMJ. 2010, 341: c5004-10.1136/bmj.c5004.PubMed CentralView Article
- Lagu T, Hannon NS, Rothberg MB, et al: Patients' Evaluations of Health Care Providers in the Era of Social Networking: An Analysis of Physician-Rating Websites. J Gen Intern Med. 2010, 25 (9): 942-6. 10.1007/s11606-010-1383-0.PubMed CentralView Article
- Fotaki M, Roland M, Boyd A, et al: What benefits will choice bring to patients? Literature review and assessment of implications. J Health Serv Res Policy. 2008, 13 (3): 178-84. 10.1258/jhsrp.2008.007163.View Article
- Bevan G, Helderman JK, Wilsford D: Changing choices in health care: implications for equity, efficiency and cost. Health Econ Policy Law. 2010, 5 (3): 251-67. 10.1017/S1744133110000022.View Article
- Emmert M, Maryschok M, Eisenreich S, et al: [Websites to assess quality of care--appropriate to identify good physicians?]. Gesundheitswesen. 2009, 71 (4): e18-27. 10.1055/s-0028-1103288.View Article
- Hussey PS, Anderson GF, Osborn R, et al: How does the quality of care compare in five countries?. Health Aff (Millwood). 2004, 23 (3): 89-99. 10.1377/hlthaff.23.3.89.View Article
- McGlynn EA, Asch SM, Adams J, et al: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348 (26): 2635-45. 10.1056/NEJMsa022615.View Article
- Qaseem A, Snow V, Gosfield A, et al: Pay for performance through the lens of medical professionalism. Ann Intern Med. 2010, 152 (6): 366-9.View Article
- Stubbe JH, Gelsema T, Delnoij DM: The Consumer Quality Index Hip Knee Questionnaire measuring patients' experiences with quality of care after a total hip or knee arthroplasty. BMC Health Serv Res. 2007, 7: 60-10.1186/1472-6963-7-60.PubMed CentralView Article
- Wensing M, van Lieshout J, Jung HP, et al: The Patients Assessment Chronic Illness Care (PACIC) questionnaire in The Netherlands: a validation study in rural general practice. BMC Health Serv Res. 2008, 8: 182-10.1186/1472-6963-8-182.PubMed CentralView Article
- Mayring P, Huber G, Grtler L, eds: Mixed Methodology in Psychological Research. 2008, Sense Publishers: Rotterdam
- Ramsay J, Campbell JL, Schroter S, et al: The General Practice Assessment Survey (GPAS): tests of data quality and measurement properties. Fam Pract. 2000, 17 (5): 372-9. 10.1093/fampra/17.5.372.View Article
- Defossez G, Mathoulin-Pelissier S, Ingrand I, et al: Satisfaction with care among patients with non-metastatic breast cancer: development and first steps of validation of the REPERES-60 questionnaire. BMC Cancer. 2007, 7: 129-10.1186/1471-2407-7-129.PubMed CentralView Article
- McKinley RK, Manku-Scott T, Hastings AM, et al: Reliability and validity of a new measure of patient satisfaction with out of hours primary medical care in the United Kingdom: development of a patient questionnaire. BMJ. 1997, 314 (7075): 193-8.PubMed CentralView Article
- Steine S, Finset A, Laerum E: A new, brief questionnaire (PEQ) developed in primary health care for measuring patients' experience of interaction, emotion and consultation outcome. Fam Pract. 2001, 18 (4): 410-8. 10.1093/fampra/18.4.410.View Article
- Kaplan RM, Saccuzzo DP: Psychological Testing. Principles, Applications, and Issues. 2009, Belmont: Wadsworth
- Fan VS, Burman M, McDonell MB, et al: Continuity of care and other determinants of patient satisfaction with primary care. J Gen Intern Med. 2005, 20 (3): 226-33. 10.1111/j.1525-1497.2005.40135.x.PubMed CentralView Article
- Duffy FD, Gordon GH, Whelan G, et al: Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Acad Med. 2004, 79 (6): 495-507. 10.1097/00001888-200406000-00002.View Article
- Healthcarescoop T: [http://www.theHealthcarescoop.com]
- IOM: Crossing the quality chasm: a new health system for the 21st century, ed. Io Medicine. 2001, Washington (DC): National Academies Press
- Nutbeam D: The evolving concept of health literacy. Soc Sci Med. 2008, 67 (12): 2072-8. 10.1016/j.socscimed.2008.09.050.View Article
- Nutbeam D: Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promotion International. 2000, 15 (3): 259-267. 10.1093/heapro/15.3.259.View Article
- Eysenbach G, Powell J, Englesakis M, et al: Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions. BMJ. 2004, 328 (7449): 1166-10.1136/bmj.328.7449.1166.PubMed CentralView Article
- Damman OC, van den Hengel YK, van Loon AJ, et al: An international comparison of web-based reporting about health care quality: content analysis. J Med Internet Res. 2010, 12 (2): e8-10.2196/jmir.1191.PubMed CentralView Article
- Strech D: [Physician rating sites from an ethical viewpoint. A Roadmap]. Z Evid Fortbild Qual Gesundhwes. 2010, 104 (8-9): 674-681. 10.1016/j.zefq.2010.09.019.View Article
- Webster G: Final Report on the Patient Satisfaction Questionnaire Project. Edited by: ABoIMCoEoC Competence. 1989, Philadelphia
- Baker R: Development of a questionnaire to assess patients' satisfaction with consultations in general practice. Br J Gen Pract. 1990, 40 (341): 487-90.PubMed Central
- Poulton BC: Use of the consultation satisfaction questionnaire to examine patients' satisfaction with general practitioners and community nurses: reliability, replicability and discriminant validity. Br J Gen Pract. 1996, 46 (402): 26-31.PubMed Central
- Klingenberg A, Bahrs O, Szecsenyi J: [How do patients evaluate general practice? German results from the European Project on Patient Evaluation of General Practice Care (EUROPEP)]. Z Arztl Fortbild Qualitatssich. 1999, 93 (6): 437-45.
- Greco M, Powell R, Sweeney K: The Improving Practice Questionnaire (IPQ): a practical tool for general practices seeking patient views. Educ Prim Care. 2003, 14 (4): 440-448.
- Brinkmann A, Steffen P, Pfaff H: [Patient surveys as an element of quality management in outpatient care: development and assessment of a questionnaire]. Gesundheitswesen. 2007, 69 (11): 585-92. 10.1055/s-2007-990307.View Article
- Pfaff H, Freise DC, Mager G, et al: Patient surveys as an element of quality management in outpatient care: development and assessment of a questionnaire. 2003, Sankt Augustin: Asgard-Verlag
- Meakin R, Weinman J: The 'Medical Interview Satisfaction Scale' (MISS-21) adapted for British general practice. Fam Pract. 2002, 19 (3): 257-63. 10.1093/fampra/19.3.257.View Article
- Garratt AM, Bjaertnes OA, Krogstad U, et al: The OutPatient Experiences Questionnaire (OPEQ): data quality, reliability, and validity in patients attending 52 Norwegian hospitals. Qual Saf Health Care. 2005, 14 (6): 433-7. 10.1136/qshc.2005.014423.PubMed CentralView Article
- Van der Feltz-Cornelis CM, Van Oppen P, Van Marwijk HW, et al: A patient-doctor relationship questionnaire (PDRQ-9) in primary care: development and psychometric evaluation. Gen Hosp Psychiatry. 2004, 26 (2): 115-20. 10.1016/j.genhosppsych.2003.08.010.View Article
- Grogan S, Conner M, Norman P, et al: Validation of a questionnaire measuring patient satisfaction with general practitioner services. Qual Health Care. 2000, 9 (4): 210-5. 10.1136/qhc.9.4.210.PubMed CentralView Article
- Nubling M, Muhlbacher A, Niebling W: [Patient survey in general practice: development, validation and application of a survey instrument]. Z Arztl Fortbild Qualitatssich. 2004, 98 (4): 301-8.
- Gericke CA, Schiffhorst G, Busse R, et al: [A validated questionnaire for measuring patient satisfaction in general and specialist ambulatory medical care: the Qualiskope-A]. Gesundheitswesen. 2004, 66 (11): 723-31. 10.1055/s-2004-813750.View Article
- Bitzer EM, Dierks ML, Dörning H, et al: Zufriedenheit in der Arztpraxis aus Patientenperspektive - Psychometrische Prüfung eines standardisierten Erhebungsinstrumentes. Zeitschrift für Gesundheitswissenschaften. 2007, 7 (3): 196-209.
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/10/332/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<url>http://creativecommons.org/licenses/by/2.0</url>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.