Data collection and samples
Data were collected as part of a larger study (‘Strengthening patient competence: Breast cancer patients’ information and training needs’ (“PIAT”)). The study was conducted in a sample of German breast cancer center hospitals. We included hospitals that were certified according to the criteria of the German Cancer Society and the German Society for Senology [34] as of May 31, 2012. We excluded hospitals that took part in a mandatory patient survey conducted in one federal state (North Rhine-Westphalia) [35] to avoid surveying patients twice. A total of 247 breast cancer center hospitals met these inclusion criteria. From these hospitals, we randomly selected 98 to participate in the study. Fifty-six (57%)a of the 98 hospitals consented to voluntarily participate in the study. The main reason of refusal was that the hospital already participated in another patient survey.
We used self-administered questionnaires to collect data from hospital directors (or their proxies) – the key informant survey – and patients – the patient survey. The study protocol was approved by the Ethics Committee of the Medical Faculty of the University of Cologne.
Hospital key informant survey
The key informant survey was conducted between June 26 and August 31, 2013. We mailed the questionnaire to a representative (the director of the breast center or a designated proxy) of each of the 56 consenting hospitals and asked them to fill out and return the questionnaire. The questionnaire contained the items we designed to assess HLHO (to be described in detail in Section HLHO items) as well as questions regarding other structural and process characteristics of the hospital (e.g., size, teaching status, ownership). The survey was designed according to the Dillman’s method – that is, we made three mail contact attempts [36], plus a final telephone reminder. Fifty-one of the consenting 56 hospitals returned the questionnaire (91%). These 51 hospitals make up the sample for analysis at the hospital level.
Patient survey
The patient survey was conducted in the 56 consenting hospitals between February 1 and August 31, 2013. Patients were included if they: (1) had undergone inpatient surgery for newly diagnosed breast cancer between February 1 and August 31, 2013; (2) had at least one malignancy; and (3) had at least one postoperative histological evaluation. Shortly after the surgery and at the end of their hospital stay, eligible patients were asked to give written consent to participate in the survey. Once they had consented, the questionnaire was handed to them to be filled out shortly before discharge, i.e. at the end of their hospital stay. In addition to the survey responses given by patients, participating hospitals provided information on each surveyed patient’s disease and treatment characteristics (e.g., cancer stage, type of surgery). Patients were also followed with additional surveys at 10 and 40 weeks after discharge. For the purpose of this study, we used only data obtained from the first wave of patient survey – the survey right before the hospital discharge.
Of the 1,846 patients meeting the eligibility criteria, 1,543 consented to the study (83.6%). Of these, 1,359 returned the questionnaire before the discharge (88.1%). Five responses were deemed unusable because they missed the hospital identifier and could not be matched to the hospital data. Of the remaining responses, 1,224 were treated in one of the 51 hospitals responding to the key informant survey and these patients made up the patient sample for the study. Our analysis showed that patients in the study sample did not differ from patients treated in the five hospitals that did not return the key informant survey with regard to cancer stage, age and education.
Measures
HLHO items
We employed a mixture of methods to develop items for assessing HLHO. First, we drafted a pool of provisional items based on the Brach et al. paper [32] and earlier research e.g. [21-29], a thorough review of the literature on health literacy and context as mentioned in the introduction, and a focus group in October 2012 with six representatives from different breast care center hospitals that discussed the role of hospitals in providing patients with adequate information and addressing poor individual literacy. Second, we held a workshop with employees of breast care center hospitals in January 2013 to discuss and select items that best reflected hospitals’ implementation of the 10 HLHO attributes defined in Brach et al. [32]. The workshop participants (N = 15) included quality managers (1), doctors (2), registered/specialist nurses (9), center coordinators (2), and self-help representatives (1). A consensus was reached that the final set of items should be parsimonious, easily understandable to respondents, and pertinent to the practices in German breast cancer center hospitals. The discussion at the workshop resulted in a draft of 10 items, with one item measuring each of the 10 HLHO attributes. Third, the 10 items were then reviewed by researchers from different disciplines (nursing, sociology, psychology, health economics), many of whom trained in developing survey questions to improve the wording and to ensure face validity.
Each of the final set of items (Table 1) was answered on a seven-point scale, ranging from ‘not at all’ to ‘to a very large extent’. This scale format was chosen to reflect the continuous character of each of the 10 attributes and to avoid the agree/disagree format [37]. Note that the order of items in Table 1 differs from the order of HLHO attributes appeared in Brach et al. [32] (see list above). The third attribute (‘workforce’) is represented by the tenth item in Table 1. Participants in the January 2013 workshop were perplexed by the workforce question and later discussion suggested that the issue should be addressed later in relation to other HLHO attributes. Moreover, a short introduction was added, which included a brief (perhaps oversimplified) definition of health literacy to familiarize respondents with the concept and the role of hospitals in promoting patient health literacy.
Other hospital characteristics
In order to identify patterns of HLHO implementation and to adjust for relevant hospital differences, the ‘conventional set’ [38] of hospital structure characteristics was assessed in the hospital questionnaire, including teaching status (non-teaching vs. teaching), ownership status (public, charitable, for-profit), and patient volume (annual number of surgeries performed on newly diagnosed breast cancer patients).
Patient characteristics
To assess the predictive validity, we used information from the patient survey to construct a variable regarding the adequacy of information provided in the breast care hospital. The variable was based on eight items asking patients their perceived adequacy of information that they received from the hospital regarding: (1) breast cancer self-help groups; (2) psychological support programs; (3) rehabilitation possibilities; (4) “patient guideline”, a brochure by the German Cancer Society and the German Cancer Aid; (5) obtaining a second opinion from another doctor; (6) dealing with side effects of treatment; (7) possible critical incidents that may occur at home; and (8) activities that should be avoided during treatment.
For each item, there were six possible answers (‘I received too little information’ , ‘the information was exactly right for me’ , ‘I received too much information’ , ‘I was over-challenged with the information’ , ‘I wasn’t offered information’ , and ‘I didn’t want any information’). ‘The information was exactly right for me’ was coded 1; ‘I didn’t want any information’ was coded missing; and all other answers were coded 0. Exploratory factor analysis of the eight items (principal component extraction method with varimax rotation) suggested two latent factors, whose initial eigenvalues were 3.4 and 1.2, respectively. Because the second factor had a small eigenvalue (barely larger than 1), we decided to use the average of responses to all eight items to present the perceived adequacy of information provided by the hospital. The composite variable was constructed for patients with a least five valid (i.e. non-missing) answers. The Cronbach’s alpha of the variable was 0.81, suggesting satisfactory internal reliability.
Patient attributes may be related to different information demands. Our analysis accounted for patient age (categorized into younger than 40 years, 41 to 50, 51 to 60, 61 to 70, over 71), type of surgery (mastectomy vs. breast conserving treatment), UICC cancer stage (stages 0-I vs. stages II-IV) [39], and health literacy. Health literacy was assessed using the three ‘best performing’ items provided by Chew et al. [12,13], which have been widely used in surveys [40,41]. We used the mean value of the items for patients with at least two valid answers to represent health literacy (Cronbach’s α = .75).
Validity and reliability assessment
We imputed the missing values on the HLHO items using the expectation-maximization (EM) algorithm in the software NORM [42,43]. The EM algorithm estimates missing data using an iterative maximum-likelihood estimation procedure [44].
We employed classical measurement theory to validate the 10-item instrument. In classical measurement theory, the two key psychometric properties of an instrument are its reliability, defined as the extent to which the instrument produces consistent results, and validity, the degree to which the instrument measures what it purports to measure. In assessing the psychometric properties of the instrument, we assumed the 10 items contributed to a total measure of the concept, Health Literate Healthcare Organization. On the basis of this assumption, the following four steps were taken to assess the reliability and validity of the instrument. First, we performed item analysis to examine the extent to which each item was correlated with the score of the total instrument. Each item’s relationship with the total score was assessed using corrected item-total correlation. In addition, we calculated the Cronbach’s alpha to examine the internal consistency of the items, or the degree to which hospital key informants answered consistently on the 10 items.
Second, we performed exploratory factor analysis to confirm the existence of a dominant latent factor and confirmatory factor analysis to confirm the unidimensionality of the ten items. To assess the factor structure of the item set, a principal components analysis with varimax rotation was performed. The global fit of a one-factor model in confirmatory factor analysis was assessed using the following measures and criteria: a non-significant chi square value (p > 0.05), a root mean square error of approximation (RMSEA) value of <0.08, and comparative fit index (CFI) and Tucker-Lewis Index (TLI) values of ≥0.90 [45].
Third, we performed several bivariate tests to examine criterion validity and to identify patterns of HLHO implementation among German breast cancer center hospitals. We calculated Spearman’s rho to examine the correlation between HLHO-10 and hospital volume. After testing of normality (Shapiro-Wilk test) and homogeneity of variance (Levene statistic), we conducted t-test and ANOVA to examine differences in HLHO-10 by teaching and ownership status.
Finally, to assess predictive validity, we tested in a hierarchical linear model of whether hospitals’ HLHO-10 score was positively correlated with the perceived adequacy of information provided to patients [46]. Both the HLHO-10 score and the adequacy of information variable were transformed into z-scores to facilitate interpretation [47]. In performing the hierarchical linear analysis, we first fit the two-level model without predictors (null model) to calculate an intraclass correlation coefficient (ICC). The ICC of the null model represents the proportion of variance in the dependent variable attributable to the hospital level. Following this, both patient characteristics and HLHO-10 (grand-mean centered) were added to the model to test the association between HLHO-10 and the extent to which patients consider the information provided as adequate. To account for hospital-level differences, hospital volume, teaching status, and ownership status were added to the model in a final step. Variables representing missing values on categorical variables were included in the models but omitted in the results presented below. Observations with missing values on continuous variables (i.e., health literacy, adequacy of information provided) were excluded from the analysis. IBM SPSS Statistics 22.0 was used for descriptive analysis, IBM SPSS AMOS 22.0 for the structural equation modeling, and HLM 7 for multilevel analysis.