- Research article
- Open Access
- Open Peer Review
REFERQUAL: a pilot study of a new service quality assessment instrument in the GP exercise referral scheme setting
BMC Health Services Researchvolume 6, Article number: 61 (2006)
The development of an instrument accurately assessing service quality in the GP Exercise Referral Scheme (ERS) industry could potentially inform scheme organisers of the factors that affect adherence rates leading to the implementation of strategic interventions aimed at reducing client drop-out.
A modified version of the SERVQUAL instrument was designed for use in the ERS setting and subsequently piloted amongst 27 ERS clients.
Test re-test correlations were calculated via Pearson's 'r' or Spearman's 'rho', depending on whether the variables were Normally Distributed, to show a significant (mean r = 0.957, SD = 0.02, p < 0.05; mean rho = 0.934, SD = 0.03, p < 0.05) relationship between all items within the questionnaire. In addition, satisfactory internal consistency was demonstrated via Cronbach's 'α'. Furthermore, clients responded favourably towards the usability, wording and applicability of the instrument's items.
REFERQUAL is considered to represent promise as a suitable tool for future evaluation of service quality within the ERS community. Future research should further assess the validity and reliability of this instrument through the use of a confirmatory factor analysis to scrutinise the proposed dimensional structure.
Exercise Referral Schemes (ERSs) in the UK involve a recommendation from a clinician to a patient concerning the up-take of physical activity, based upon certain pre-determined criteria. The patient then attends a local exercise centre, where an advanced exercise instructor will create an appropriate individualised programme, usually designed to last around 12–15 weeks. The publication of 'Exercise Referral Systems: A National Quality Assurance Framework'  was, in part, aimed at improving standards among existing ERSs and aiding the development of new systems. However, many schemes both pre- and post-publication of this document suffered from and struggled with issues relating to poor levels of adherence [2, 3]. Client perceptions of excellent service quality are highlighted as being crucial to the process of attracting new members and retaining existing members in the leisure management industry , a theme not uncommon within other service industries [5–7]. The development of a tool to accurately assess service quality within the Exercise Referral setting, could, therefore, have a considerable impact on informing ERS co-ordinators of the factors that affect adherence and subsequent enable the establishment of strategic interventions to reduce the likelihood of client drop-out. The concept of service quality has been the subject of considerably lively academic debate for two decades, since being brought to the fore by Parasuraman, Zeithaml and Berry (PZB) [8, 9] in SERVQUAL [see Additional File 1], a multiple-item scale aimed at the measurement of service quality, structured around five dimensions:
Tangibles: Physical facilities, equipment, and appearance of personnel.
Reliability: Ability to perform the promised service dependably and accurately.
Responsiveness: Willingness to help customers and provide prompt service.
Assurance: Knowledge and courtesy of employees and their ability to inspire trust and confidence.
Empathy: Caring, individualised attention the firm provides its customers.
The SERVQUAL instrument comprises twenty two questions tailored to assess customers' expectation of service quality, followed by a further twenty two items designed to identify the clients' perception of the same issues. Each item is scored on a seven point Likert scale, thus allowing the equation Q = P - E to be calculated .
Since PZB's  publication, SERVQUAL has been the target of substantial scrutiny whilst also dominating service quality assessment [10–15], being applied to numerous settings, including health [16–20] but not to ERSs. A considerable part of the academic debate over the past two decades has discussed the conceptual accuracy of SERVQUAL's perception minus expectation equation and whether performance-only measures would be more appropriate [21, 22]. This discussion is beyond the scope of this paper; however, it should be noted that in settings such as ERSs, where the perceptions of the absent client are as valuable (and possibly even more valuable) as the life-long adherent, performance-only measures are of no use [21, 22]. Furthermore, Sureshchandar et al.  go as far as to say there is "universal conformity that the twenty two items [of SERVQUAL] are reasonably good predictors of service quality in its entirety" (p. 2).
Applying SERVQUAL in its pure form (i.e. without modification) to any service is widely criticised in the literature [14, 24]. However, tailoring the instrument to a specific setting by adding additional items or modifying existing questions to supplement knowledge and understanding, SERVQUAL can give a unique insight into the service quality . PZB [8, 9] confirm adaptations to SERVQUAL are necessary if an accurate measure of service quality is to be established across a diverse range of industries.
Creation of REFERQUAL
Tailoring and augmenting the SERVQUAL instrument has been identified as a necessary requirement when being applied to a new service industry . A client adherence classification structure was created in conjunction with several ERS organisers. The structure is self-reporting, exhaustive and mutually exclusive. The five statements are constructed to classify clients relating to the extent of adherence to the referral process:
A Referred clients who did not attend at all.
B Clients who started, but dropped out before completion.
C Clients attending, but had not completed the referral at the time of receiving the questionnaire.
D Clients who completed the referral but did not continue exercising.
E Clients who are still exercising having completed referral.
In addition to the self-reporting adherence criteria, clients were also requested to complete certain demographic information addressing factors raised by the literature such as gender, age, occupation [26–29] and marital status [30, 31]. Two further questions emanating from the literature were included at the beginning of the instrument as demographic issues due to the nature of the items containing no element of expectation and so could not be included as a perception minus expectation item:
'Do you consider yourself to be physically active whilst carrying out the duties demanded by your occupation?'
'Was reducing weight one of the reasons you were referred to the scheme?'
An adaptation of the Blair et al.  7-day Physical Activity Recall (7PAR) questionnaire was also included in REFERQUAL to assess the exercise level of participants outside any occupational demand. The inclusion of the adapted 7PAR was aimed at offering the potential for differentiation between those participants who were sufficiently physically active and those who were not irrespective of adherence grouping. Furthermore, respondents were invited to report the location of the physical activity (i.e. leisure centre or elsewhere), offering the researcher greater insight into whether the service quality of the operational aspects of the scheme or the overall management of the referral site may have most critically affected adherence.
Considering the lengthy and relatively complex nature of SERVQUAL, the 7PAR was simplified from five to three levels of physical activity (light, moderate and vigorous), although duration of exercise was retained. Definitions and examples of light, moderate and vigorous physical activity were given.
Many client-related factors established as being significant determinants to adherence in the literature, formed the foundation from which tailoring of the SERVQUAL instrument was undertaken. Part of the tailoring of the SERVQUAL instrument involved the creation of two new dimensions as a number of topics raised in the literature did not conceptually 'fit' into the existing dimensional framework. The first of these relates to the relationship between the client and the GP – an association distinct to the ERS industry from any previously examined. The second relates to personal perceptions of exercise. The existing 22 items of SERVQUAL were supplemented with the questions [see Additional File 2].
Modification of SERVQUAL to eliminate negatively-worded questions included in the original instrument has received unanimous support, this was also taken into account when tailoring the items to the ERS Setting .
The pre-pilot REFERQUAL appended with a feedback sheet was distributed to six attendees at five ERSs and also to the corresponding scheme organisers. The five ERSs had been selected to participate in the main post-pilot study. Feedback was received with respect to the instrument's ease of use, wording and any other topic the respondents felt relevant. Some questions were subsequently removed the instrument whilst others were modified following feedback to enhance applicability and clarity or to eliminate duplicity.
Subsequently, one exercise class comprising 30 clients was invited to participate in the pilot study. The Exercise Professional leading the group distributed REFERQUAL by hand and responses were completed whilst the exercisers were at the centre. The Exercise Professional subsequently repeated the operation two weeks later with the same group. Twenty-seven of the 30 in the group completed both questionnaires, a response rate of 90%, the other three participants were absent for one of the two sessions. Again, respondents were also invited to comment on the instruments ease of use, wording and any general concerns relating to REFERQUAL.
In order to determine test re-test reliability, correlation will be calculated via Pearson's 'r' or Spearman's 'rho' depending on the distribution of the individual items. One sample Kolmogorov-Smirnov (K-S) tests will be used to determine whether items are drawn from normally distributed data. K-S testing is a suitable measure of distribution regardless of sample size , is consistent against all alternatives  and frequently outperforms other measures . Internal Consistency will be measured via Cronbach's α for the overall instrument and the individual dimensions. Cronbach's α is widely regarded a reliable and versatile coefficient, particularly applicable on Likert scale items .
K-S tests were conducted on each of the 35 perception/expectation scores elicited from each completion (p < 0.05). In cases where both item scores were drawn from normally distributed data, correlation was calculated via Pearson's "r" (see Table 1).
Instances featuring one or both items being drawn from non-normally distributed data, correlation calculations were made via Spearman's rho (see Table 2). Significant, positive correlations are demonstrated for all of the items calculated using Pearson's "r" (mean = 0.957, SD = 0.02) and Spearman's rho (mean = 0.934, SD = 0.03). Furthermore, Cronbach's Alpha was calculated for all 7 dimensions (35 items) at 0.903 and subsequently for each individual dimension to demonstrate internal consistency (see Table 3). Internal consistency was calculated of the second of the two completions of REFERQUAL.
The reliability and GP dimensions score particularly highly at 0.802 and 0.857 respectively. Most others scored satisfactorily; however, the tangibles and responsiveness dimensions were a little low at 0.554 and 0.619 respectively.
The overwhelmingly supportive statistical analysis above combined with the feedback received from the respondents and scheme organisers confirmed REFERQUAL's applicability and suitability for further evaluation of service quality in the ERS setting. The high correlation scores generated across all 35 items are especially encouraging that REFERQUAL is a valid and reliable research instrument in this setting explaining, at worst, 74% of the variance.
The dimensional reliability scores are also heartening, especially considering the overall instrument internal consistency (α = 0.903). The two dimensions scoring slightly lower than the others (responsiveness and tangibles) are not considered to give cause for concern. Responsiveness is widely acknowledged to be a fundamental contributor to the understanding and assessment of service quality [7, 37] and REFERQUAL's modifications contain no new items that may have been responsible for this abnormality. Further investigations utilising this tool with much larger samples than this small-scale pilot should result in a more satisfactory report for this dimension.
However, the applicability of tangibles as a dimension within service quality assessment has received criticism, relating to client's perception of the concept as a proxy for evaluating service outcomes . However, tangibles is generally retained in factor analysis  and the relative importance of the this dimension is thoroughly discussed within the literature, resulting in general affirmative agreement . However, tangibles are the least critical of the proposed service quality dimensions . Cleanliness, modern equipment and aesthetic appeal will rarely countervail poor quality products, unhelpful information and impolite staff. Future studies should incorporate factor analysis to establish whether tangibles remain distinct from the other 6 or whether this dimension requires further revision or augmentation.
Calculations within this investigation have been made via the perception/expectation score generated by the SERVQUAL format. Future studies should analyse both perception and expectation items separately to confirm the validity and reliability of this model particularly bearing in mind the fundamental advantage of the perception/expectation 'gap' model over the performance-only instruments – that the perceptions of those clients not visiting the referral sites at all could be gathered. This investigation solely sought the views of active exercises at various stages of completion of referral. Future studies should seek to investigation across the entire A-E spectrum outlined above.
Preliminary results indicate REFERQUAL to represent a promising model of service quality assessment within the ERS setting. Correlational findings are extremely supportive and are underlined by satisfactory reliability scores. However, future studies incorporating the views of the entire adherence spectrum featuring far greater sample sizes and subsequent factor analysis on the dimensional structure of REFERQUAL will offer far greater insight into the appropriability of this new model.
National Health Service: Exercise Referral Systems: A National Quality Assurance Framework. 2001, National Health Service
Bird SR, Smith A, James K: Exercise Benefits and Prescription. 1998, Cheltenham, Stanley Thomas
Riddoch C, Puig-Ribera A, Cooper A: Effectiveness of Physical Activity Promotion Schemes in Primary Care: A Review. 1998, London, Health Education Authority
Kim C, Kim SY: 'Segmentation of Sport Centre Members in Seoul Based on Attitudes Toward Service Quality'. Journal of Sport Management. 1998, 12: 273-287.
Park S, Kim Y: 'Conceptualising and Measuring the Attitudinal Loyalty Construct in Recreational Sport Contexts'. Journal of Sport Management. 2000, 14: 197-207.
Triado XM, Aparico P, Rimbau E: 'Identification of Factors of Customer Satisfaction in Municipal Sport Centres in Barcelona: Some Suggestions for Satisfaction Improvement'. Cyber-Journal of Sport Marketing. 1999, 3: 1-11.
Youssef FN, Nel D, Bovaird T: 'Health Care Quality in NHS Hospitals'. International Journal of Health Care Quality Assurance. 1996, 9 (1): 15-28. 10.1108/09526869610109125.
Parasuraman A, Zeithaml V, Berry L: 'A Conceptual Model of Service Quality and its Implications for Future Research'. Journal of Marketing. 1985, Autumn: 41-50. 10.2307/1251430.
Parasuraman A, Zeithaml VA, Berry LL: 'SERVQUAL: A Multiple-Item Scale for Measuring Consumer Perceptions of Service Quality'. Journal of Retailing. 1988, 64 (1): 12-40.
Brady MK, Cronin JJ, Brand RR: 'Performance-only Measurement of Service Quality: A Replication and Extension'. Journal of Business Research. 2002, 55: 17-31. 10.1016/S0148-2963(00)00171-5.
Hemmasi M, Strong KC: 'Measuring Service Quality for Strategic Planning and Analysis in Service Firms'. Journal of Applied Business Research. 1994, 10 (4): 24-38.
Ko YJ, Pastore DL: 'Current Issues and Conceptualisations of Service Quality in the Recreation Sport Industry'. Sport Marketing Quarterly. 2004, 13: 158-166.
Newman K: 'Interrogating SERVQUAL: A Critical Assessment of Service Quality Measurement in a High Street Retail Bank'. International Journal of Bank Marketing. 2001, 19 (3): 126-139. 10.1108/02652320110388559.
Robinson S: Measuring Service Quality: Current Thinking and Future Requirements. Marketing Intelligence and Planning. 1999, 17 (1): 21-32. 10.1108/02634509910253777.
van Iwaarden J, van der Wiele T, Ball L, Millen R: 'Applying Research to Web Sites: An Exploratory Study. International Journal of Quality and Reliability Management. 2003, 20 (8): 919-935. 10.1108/02656710310493634.
Asubonteng P, McCleary KJ, Swan JE: SERVQUAL Revisited; A Critical Review of Service Quality. Journal of Services Marketing. 1996, 10 (6): 62-81. 10.1108/08876049610148602.
Babakus E, Mangold GW: 'Adapting the SERVQUAL Scale to Hospital Services: An Empirical Investigation'. Health Services Research. 1992, 26 (6): 767-786.
Clow KE, Fischer AK, O'Bryan D: 'Patient Expectations of Dental Services'. Journal of Health Care Marketing. 1995, 15 (3): 22-31.
Desombre T, Eccles G: 'Improving Service Quality in Trust Hospitals: Lessons From The Hotel Sector'. International Journal of Health Care Quality Assurance. 1998, 11 (1): 21-27. 10.1108/09526869810199610.
O'Connor SJ, Shewchuk RM, Carney LW: 'The Great Gap: Physicians' Perceptions of Patient Service Quality Expectations Fall Short of Reality'. Journal of Health Care Marketing. 1994, 14 (2): 32-9.
Cronin JJ, Taylor SA: 'Measuring Service Quality: A Re-examination and Extension'. Journal of Marketing. 1992, 56: 55-68. 10.2307/1252296.
Teas RK: 'Expectations, Performance Evaluation and Consumers' Perceptions of Quality'. Journal of Marketing. 1993, 57: 18-34. 10.2307/1252216.
Sureshchandar GS, Rajendran C, Kamalanabhan TJ: 'Customer Perceptions of Service Quality: A Critique'. Total Quality Management. 2001, 12 (1): 111-127. 10.1080/09544120020010138.
Cronin JJ, Taylor SA: 'SERVPERF versus SERVQUAL: Reconciling Performance-Based and Perceptions-Minus-Expectations Measurement of Service Quality'. Journal of Marketing. 1994, 58: 125-131. 10.2307/1252256.
Parasuraman A, Zeithaml V, Berry L: 'Reassessment of Expectations as a Comparison Standard in Measuring Service Quality: Implications for Future Research'. Journal of Marketing. 1994, 58: 111-124. 10.2307/1252255.
Burton NW, Turrell G: 'Occupation, Hours Worked, and Leisure-Time Physical Activity'. Preventive Medicine. 2000, 31: 673-681. 10.1006/pmed.2000.0763.
Salmon J, Owen N, Bauman A, Schmitz MKH, Booth M: 'Leisure-Time, Occupational and Household Physical Activity Among Professional, Skilled, and Less-Skilled Workers and Homemakers'. Preventive Medicine. 2000, 30: 191-199. 10.1006/pmed.1999.0619.
Seefeldt V, Malina RM, Clark MA: 'Factors Affecting Levels of Physical Activity in Adults'. Sports Medicine. 2002, 32 (3): 143-168. 10.2165/00007256-200232030-00001.
Taylor AH: 'Evaluating GP Exercise Referral Schemes: Findings from a Randomised Controlled Study'. 1996, Chelsea School Research Centre, Topic Report 6, University of Brighton
Burke V, Giangiulio N, Gillam HF, Beilin LJ, Houghton S, Milligan RAK: 'Health Promotion in Couples Adapting to a Shared Lifestyle'. Health Education Research. 1999, 14 (2): 269-288. 10.1093/her/14.2.269.
Hardcastle S, Taylor AH: 'Looking For More than Weight Loss and Fitness Gain: Psychosocial Dimensions Among Older Women in a Primary Care Exercise-Referral Program'. Journal of Ageing and Physical Activity. 2001, 9: 313-328.
Blair SN, Haskell WL, Ho P: 'Assessment of Habitual Physical Activity by a Seven-day Community Survey and Controlled Experiments'. American Journal of Epidemiology. 1985, 122: 794-804.
Gwanyama PW: 'K-S Test for Goodness of Fit and Waiting Times for Fatal Plane Accidents'. International Journal of Mathematical Education in Science and Technology. 2005, 35 (4): 333-343.
Feltz CJ, Goldin GA: 'Partition-Based Goodness-of-Fit Tests on the Line and the Circle'. Australian and New Zealand Journal of Statistics. 2001, 43 (2): 207-220. 10.1111/1467-842X.00166.
Tan Y-P, Nagamani J, Lu H: 'Modified Kolmogorov-Smirnow Metric for Shot Boundary Detection'. Electronics Letters. 2003, 39 (18).
Thomas JR, Nelson JK: Research Methods in Physical Activity. 2001, Leeds, Human Kinetics, Fourth
Brady MK, Cronin JJ: 'Some New Thoughts on Conceptualizing Perceived Service Quality: A Hierarchial Approach'. Journal of Marketing. 2001, 65: 34-49. 10.1509/jmkg.18.104.22.16834.
Mels G, Boshoff C, Nel D: 'The Dimensions of Service Quality: The Original European Perspective Revisited'. Service Industries Journal. 1997, 17 (1): 173-189.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/6/61/prepub
The authors are keen to thank all the Exercise Professionals, Scheme Organisers and participants that have contributed to the refining of the original REFERQUAL instrument.
Declaration of competing interests
The author(s) declare that they have no competing interests.
DC: Responsible for the overall study design and the acquisition and analysis of the data. DC was also responsible for the drafting of the document.
IA: Made substantial contributions to the conceptual design of the study and responsible for considerable drafting input.
AI: Made substantial contributions to the conceptual design of the study and responsible for considerable drafting input. Also responsible for statistical guidance and advice.
PB: Made substantial contributions to the conceptual design of the study and responsible for considerable drafting input.