This study relied on data from 3 major sources: the archived Nursing Home Compare (NHC) data of 2017; the 2016 Maryland nursing home experience-of-care survey conducted by the Maryland Health Care Commission (MHCC); and on-line consumer ratings we collected from 4 popular social networking sites.
The NHC data are maintained and updated by the CMS and contain key nursing home characteristics such as facility name and address, nurse staffing, deficiency citations, consumer complaints filed against the facility, and 5-star ratings. The 5-star quality ratings were designed to simplify information for consumers by aggregating quality measures into a rating system of one to five stars, with more stars indicating better quality. The ratings are derived from 3 domains of quality: nurse staffing to resident ratios (for registered nurses [RNs] and all nursing staff including RNs, licensed practical nurses, and certified nursing assistants), deficiency citations (assigned during annual and complaint inspections), and clinical outcomes of residents based on Minimum Data Set assessments; an overall rating further aggregate the three domains .
The MHCC publishes on-line the nursing home experience-of-care rating scores annually based on mailed surveys of designated responsible parties (i.e. family members or legal guardians/representatives) of all long-term residents in Maryland . This study followed the methodologies (survey design, samples, and survey methods) described in previous reports [15, 17, 29]. Briefly, the 2016 survey was conducted between March and June of 2016, and responses reflected family member evaluations of care provided from late 2015 to early 2016. All Maryland nursing homes serving long-term residents (n = 222 facilities) participated in this year’s survey and surveys were mailed to all responsible parties (n = 16,631) of their residents. Follow-up mails and phone calls were made to non-respondents. A total of 8356 completed surveys were received finally, resulting in an overall response rate of 53%. The survey asked 17 questions to assess 5 domains of resident care including (1) staff and administration, (2) care provided to residents, (3) food and meals, (4) autonomy and resident rights, and (5) physical aspects of the facility. The rating of each domain is the average of scores of all questions within the domain, and has a range between 1 (worst experience with care) and 4 (best experience with care). The survey also asked two additional questions about (1) overall experience with care in the nursing home on a rating from 1 (worst possible care) to 10 (best possible care) and (2) whether the respondent would recommend the nursing home to those who need nursing home care (yes/no).
Using the list of Maryland nursing homes published by the MHCC, we first conducted Google Maps search to obtain each nursing home’s Google Customer Reviews 5-star rating scores from past or existing patients/families, and to identify the web page of each facility as well. We then searched within each facility’s page for a link to the Facebook page and to the Yelp page of the nursing home. For nursing home websites that did not include a link to their Facebook or Yelp page, we searched Facebook and Yelp respectively for the facility’s official page, and confirmed this information using facility name and address. Finally, we identified all Maryland nursing homes from caring.com, an online reviews website specifically designed for customer search for and rating on professional senior care providers such as assisted living facilities, nursing homes, and hospices.
All the 4 crowdsourcing sites (Facebook, Yelp, Google Customer Reviews, and caring.com) allow customers to rate their experiences with healthcare providers using 1 (worst experience) to 5 (best experience) stars and post optional review texts [16, 25, 26, 28]. We collected all star-ratings posted on the 4 sites during the period of July 2015 to July 2017; no potentially identifiable information (e.g. reviewer ID or user name) was collected from these sites. We chose this period because it matches roughly the periods of data collection in the Maryland nursing home care experience survey (2016) and in the 2017 NHC quality measures (e.g. 5-star ratings largely derived from 2015 to 17 data).
We analyzed facility-level average score of 5-star ratings from the 4 social media or consumer review websites (hereafter referred to as social media ratings), experience-of-care ratings, NHC 5-star overall ratings, and individual NHC quality measures including annual number of deficiency citations, case mix adjusted hours per resident day for RNs and for all nurses, and number of complaints filed by consumers or caregivers against the facility during 2015–17 that resulted in a deficiency citation. The goal of these analyses was to determine how well the average score of social media ratings is correlated with or predictive of the NHC measures and the survey-based care experience ratings for Maryland nursing homes. We performed descriptive analyses and ran Pearson correlation analyses on all measures and scores from alternative sources.
We further fit separate multivariable linear regression models to test the association of the average social media rating score (independent variable) with each NHC measure or the overall or domain-specific experience-of-care rating (dependent variable). All regression models controlled for nursing home and county covariates including number of certified beds, total number of residents, profit status (for-profit or not), chain affiliation (yes/no), a case mix index calculated based on the Resource Utilization Groups classification system, percentages of Medicare residents in the nursing home, percentage of Medicaid residents, percentage of white residents, and a measure of market competition for nursing home care calculated from the county-level Herfindahl–Hirschmann index. In each model, the average social media rating, which ranged from 1 to 5 continuously, was categorized as < 2 stars, from 2 (inclusive) to 4 stars, and ≥ 4 stars, with the first group serving as the comparison group. We present adjusted NHC measures or experience-of-care ratings by social media rating group based on model predictions.