In a large national in-center HD population, non-responders to the ICH CAHPS survey differed substantially from responders. Specifically, non-responders were more likely to be men, non-white, younger, single, dual Medicare/Medicaid eligible, less educated, non-English speaking, inactive on the transplant list, and had longer ESRD vintage, lower BMI and lower serum albumin, worse functional status, and more hospitalizations, missed treatments, and shortened treatments. These results demonstrate underrepresentation of important groups of in-center HD patients, broadly spanning individuals with fewer socioeconomic advantages and greater illness burden. It is possible that these results could introduce biases into facility-level ICH CAHPS survey results, particularly given low overall response rates, resulting in missed opportunities to assess and improve patient experience among the most vulnerable hemodialysis patients.
CAHPS surveys are widely used in US medical settings to evaluate patient experience, with other CAHPS surveys targeting hospitals, nursing homes and other settings. The ICH CAHPS is unique as it evaluates facilities with a relatively low number of patients per facility and with longstanding patient-facility relationships rather than discreet episodes. There is limited published literature on characteristics of non-responders to other CAHPS surveys. Most importantly, previous assessments use only limited patient-reported characteristics unlike our study where we use extensive characteristics gathered using reliable data sources rather than patient self-report. Even so, similar to our findings, analysis of Medicare Managed Care (MMC) CAHPS survey from 1997 and 1999 found significantly higher non-response rates in participants who were male and non-white [15]. Likewise, analysis of the Hospital CAHPS (HCAHPS) pilot survey data from 2002 to 2003 also found male sex, younger age, and non-white race to be significantly associated with non-response [16]. Finally, in a large sample of Medicare CAHPS participants from 2007 there were significantly higher non-response rates in participants who were men, non-white, and dual eligible for Medicare and Medicaid [17].
These results have important implications for assessment of dialysis patient experience. The DCI patient population is similar to the broader US dialysis population with the exception that there is a slightly higher proportion of black patients receiving dialysis care at DCI [18]. To our knowledge, this is the only non-CMS dataset linking individual patient-level clinical data to ICH CAHPS responses, and this is the first study assessing the differences in characteristics, including laboratory variables and treatment characteristics, between responders and non-responders. Previous published work on the ICH CAHPS survey is restricted to reports on the development and testing of this survey, where there was a response rate of 46% and there was no published evaluation of non-responders [8, 19]. Within one of these study cohorts, response rates were noted to be higher among those assigned to mail followed by telephone versus telephone only survey administration [20]. Although supervision of ICH CAHPS administration was transferred from AHRQ to CMS in 2014, the current survey remains similar to the one administered in 2012, with the major exception that limited assistance is now allowed, consistent with the ‘expanded usable’ criteria used in secondary analyses in this manuscript.
Payers increasingly are moving towards value based purchasing models, with performance metrics critical to quantify value. Before the addition of ICH CAHPS as a performance metric, the ESRD QIP was composed of only clinical and laboratory measures, most of which were not specifically patient-centered outcomes [21]. Patient experience measures have been widely implemented in other areas of healthcare, and use of the ICH CAHPS survey represents an important milestone for in-center HD; however, attempts to address patient-centered care using a patient-reported outcome measures with low response rates may have limitations. Paradoxically, we found that non-responders tended to be patients who are disproportionately represented in the US ESRD population as compared to the general population (specifically younger, black, male, and diabetics) [18]. These differences in characteristics associated with non-response raise the possibility of non-response bias; however further research is needed in evaluating whether or not these characteristics are also associated with experience scores and will thereby affect facility performance ratings and performance-based payments as well as misrepresent key areas needed for intervention to improve patient experience [22].
The specific reasons for non-response remain unknown. Neither the former AHRQ nor the current CMS administration process collects reasons for non-response unless it is due to incorrect contact information. Comorbid conditions common among dialysis patients include physical, cognitive, and visual impairments that may limit the ability of HD patients to respond to a survey themselves. Accordingly, and particularly in view of the survey’s length (currently 62 questions), the initial decision by AHRQ to not allow any assistance may have had important implications. Using a less restrictive method of classifying survey completion, more consistent with current CMS guidance, we were able to include approximately 5% more responses across most demographic characteristics; notably, inclusion of these surveys did not change the predictors of non-response.
Our results may have substantial implications for dialysis facilities if characteristics associated with non-response are also associated with experience scores. Starting in calendar year 2016 (and reflected in 2018 payments to facilities), survey results are a clinical performance measure within the ESRD QIP and experience scores can impact facility payments from CMS [9].
An important strength of this study is that it documents new information about the real-world administration of the ICH CAHPS survey. Additionally, this study provides information that can no longer be gathered since survey vendors are now barred from providing patient-level data to dialysis facilities. Other strengths include having a large number of survey responses from a national dialysis provider linked to extensive facility gathered patient-level demographic, clinical, and functional data. Limitations include not knowing the precise date of survey completion during the survey administration period, which required the use of proximate covariate data. As with most surveys, we do not have information on reasons for non-response.
ICH CAHPS survey response rates remain low overall (only 33% despite allowing limited assistance with survey completion [10]). Future studies should provide ongoing evaluation into the presence of and reasons for non-response to this survey to inform strategies for engaging populations that have a greater likelihood of non-response, specifically patients with greater illness burden and fewer socioeconomic advantages, in order to improve the generalizability and utility of surveys of patient experience.