You are here

  • Home
  • Archive
  • Volume 12, Issue 1
  • Correlations between care users’ and the healthcare inspectorate’s ratings of the quality of care in long-term care homes

Article Text

Article menu

Download PDFPDF

Original research
Correlations between care users’ and the healthcare inspectorate’s ratings of the quality of care in long-term care homes
  1. Flora Palimetaki1,
  2. Karlijn Woutersen1,
  3. Anne Margriet Pot1,2,3
  1. 1Health and Youth Care Inspectorate, Utrecht, Netherlands
  2. 2Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Rotterdam, Netherlands
  3. 3Optentia, North-West University, Vanderbijlpark, South Africa
  1. Correspondence to Mrs Flora Palimetaki; f.palimetaki{at}igj.nl

Abstract

Objectives Increasing emphasis is being placed on person-centredness as a quality requirement for long-term care (LTC). Although healthcare inspectorates value the importance of care users’ experiences, they struggle to address these experiences in regulatory practice. The aim of this study is to explore the correlations between care users’ and the healthcare inspectorate’s ratings of the quality of LTC in The Netherlands.

Design The correlations between care users’ ratings from a public Dutch online patient rating site and the Dutch Health and Youth Care Inspectorate’s ratings of the quality of care were examined using Spearman rank correlations. The inspectorate’s ratings cover three themes: ‘attention to person-centred care’, ‘working towards sufficient and competent care staff’ and ‘focusing on quality and safety’.

Setting Ratings of the quality of care were obtained for 200 LTC homes in The Netherlands between January 2017 and March 2019. These LTC homes had 6 to 350 residents (M=89; SD=57) and belonged to organisations with 1–40 LTC homes in total (M=6; SD=6).

Participants Publicly available anonymous ratings of the perceived quality of care by care users were extracted from the Dutch patient rating website ‘www.zorgkaartnederland.nl’. Care users’ ratings were available for the 2 years prior to an assessment by the inspectorate for 200 LTC homes.

Results We found a weak, significant correlation between the mean care users’ ratings and the inspectorate’s aggregated scores for the theme ‘person-centred care’ (r=0.26, N=200, padj<0.01); no other correlations were significant.

Conclusions This study showed only a weak correlation between care users’ ratings and ratings of the Dutch Inspectorate of the quality of ‘person-centred care’ in LTC homes. Therefore, it may be fruitful to intensify or innovate approaches to involve care users’ experiences in regulation to do them justice.

  • Quality measurement
  • Patient satisfaction
  • Patient-centred care
  • Nursing homes

Data availability statement

Data are available on reasonable request. Data are avalable on reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2022-001897

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Healthcare inspectorates value the importance of care users’ experiences of the quality of care provided in long-term care (LTC) homes, however, they struggle to address these experiences in regulatory practice.

    • Earlier studies showed weak to low-to-medium positive correlations between care users’ and inspectorates’ ratings of the quality of care provided by LTC homes.

    WHAT THIS STUDY ADDS

    • Care users’ ratings of LTC homes are weakly related to the inspectorates’ ratings of person-centred care, but not to the ratings of other aspects of the quality of care.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • We recommend to investigate approaches to intensify or innovate the involvement of care users’ experiences in regulatory practice to do them justice.

    Introduction

    Increased emphasis is being placed on person-centredness as a quality requirement for long-term care (LTC), such as the care provided by nursing homes and other LTC homes.1 2 The importance of person-centred care, that is, care that fulfils the needs of the person in accordance with their preferences, has been widely discussed. The WHO states that LTC should be person-centred care that is consistent with peoples’ basic rights, fundamental freedoms and human dignity.1 3 Users of LTC must be invited and empowered to participate in decisions about their care, and their wishes should be protected and their dignity and autonomy promoted.3 4

    Person-centred care is also an important issue for healthcare inspectorates when they assess the quality of care in LTC homes.5 Healthcare inspectorates are entrusted with the task of supervising healthcare by making sure that care providers comply with the relevant legal and field standards through site visits, reports of serious incidents and analysis of relevant information. As an acknowledgement of the importance of person-centred care, regulators have attempted to incorporate care users’ experiences into the inspection process by collecting information from care users, including people living in a nursing home setting.6 Braithwaite and Makkai showed that—in general—an inspection process is both reliable and practical, regardless of the severity of the needs of the users in care homes.7 The involvement of care users and family members in the inspection process may contribute to justice and empowerment among care users and their families and improve the quality and legitimacy of regulation.8

    Although care users’ experiences are valued by regulators during assessments of the quality of LTC provided in care homes, research shows that it is not easy to collect data on care users’ experiences. For example, the Dutch Health and Youth Care Inspectorate introduced ‘Mystery Guests’ as an equivalent of the ‘mystery shopper’ in the consumer sector to assess the quality of LTC. However, a pilot study revealed that inspectors did not use the information gathered by the mystery guests, as the mystery guests evaluated the quality of care and reported their findings in a manner that did not align with the practices of the inspectorate.9 In a subsequent study, experts by experience were selected and trained to assess the quality of care provided in LTC homes. Experts by experience are lay people whose experiences lie between the world of care users and the inspectors of LTC homes,10 who either have personal experience with LTC homes or experience as an informal carer of users of an LTC. However, the results of the study showed that the added value of the experiences reported by these experts was limited, as the factual information produced during the inspectorate’s assessments was repeatedly valued as being more legitimate. In other words, as a cultural effect, the inspectors valued their professional knowledge more highly than the practical wisdom of the experts by experience. As a result, the experts by experience were unable to contribute their own experiences during the assessment by the inspectorate, except when the experts by experience established contact and spoke with care users.10 The Care Quality Commission (CQC), the regulator of health and social care in England, also included experts by experience (either service users or lay people) in their inspection teams. A review of policy documents and interviews with CQC staff and patient and public representatives showed that the CQC tried to include the users’ experiences within their inspections and ratings. However, the process by which the users’ experiences were incorporated was not transparent and the CQC stopped engaging with the experts by experience at that time.11

    Nevertheless, regulators in several countries, including The Netherlands and the UK, continue to improve their approaches to include care user information and involvement in their inspection processes and assessments6; this raises the question of whether the quality of nursing home care as assessed by regulators reflects the quality of care perceived by the care users.

    Care users’ ratings collected from publicly available online review websites have previously been used to explore the usability and relationship of care users’ feedback with inspection outcomes.12–15 These websites were originally intended to help future care users and their representatives compare care providers and make informed decisions when selecting a specific care organisation. Research in the USA has shown that Yelp care users’ five-star ratings of nursing home care were significantly higher than the Nursing Home Compare inspection ratings.16 However, another study from the USA showed that the aggregate scores of users’ five-star ratings collected from four social media sites (Facebook, Yelp, Google Consumer Reviews and caring.com) correlated positively to Nursing Home Compare inspection ratings.17

    Since care users’ experiences feature prominently on the agenda of regulators in many countries, this study aimed to explore whether care users’ and the inspectorate’s ratings of the quality of nursing home care in The Netherlands are related, and if so, to what extent. Due to the focus of the Dutch Inspectorate to involve care users and family members in regulatory practice,6 we hypothesise that both the inspectorate’s and care users’ ratings will show at least some overlap, particularly with respect to person-centred care.

    Methods

    Selection of LTC homes

    This study investigated the quality of care provided by a wide range of LTC homes in The Netherlands, here called LTC homes, for which public care users’ ratings of the quality of care were available and ratings of the Dutch Health and Youth Care Inspectorate (the inspectorate) (N=200; see below for more details on the selection process).

    In The Netherlands, there are different types of LTC homes, ranging from small-scale living arrangements, such as care farms or group living homes, to traditional large-scale nursing homes.18 The Care Needs Assessment Centre decides whether people are entitled to care in an LTC home as stipulated in the LTC-act (Wet Langdurige Zorg: WLZ). One of the requirements is that the person needs constant care or supervision.19

    Patient and public involvement

    Patient or public were not directly involved in this study. However, we did use publicly available care users' ratings of the quality of care provided in LTC homes and investigated the correlations with the inspectorate’s ratings as described below.

    Care users’ ratings

    Care users’ ratings of the quality of care provided in LTC homes were extracted from a public Dutch online rating site managed by The Netherlands Patient Federation.20 Care users or their representatives—relatives or other close ones—can rate the quality of care on a voluntary basis by scoring six items on a scale from 1 to 10, where 10 is the best score (see table 1 for a description of the items).

    View this table:
    Table 1

    Care users’ ratings of long-term care homes (N=200) on the six items of the Dutch online patient rating website ‘ZorgkaartNederland’ (The original Dutch text can be requested from the first author)

    For the main analyses, we calculated the mean care users’ ratings for the six items as well as the total care users’ ratings for each LTC home. To ensure the relevance of the care users’ ratings, we only used data on care users’ ratings published within the 2 years prior to the rating of each LTC home by the inspectorate. The ratings are published anonymously on the ZorgkaartNederland website.20 ZorgkaartNederland does not collect any data on the characteristics of the people who provide the ratings.20 21

    In total, care users’ ratings were available for 2152 LTC homes.

    Ratings of the inspectorate

    Inspectors visit LTC homes and use a framework to rate adherence to the standards for quality of nursing home care.5 22 The framework consists of eight items covering three themes: ‘attention to person-centred care’, ‘working towards sufficient and competent care staff’ and ‘focusing on quality and safety’ (see table 2). The items are rated by at least two inspectors on a four-point scale: ‘does not meet the standard’, ‘largely does not meet the standard’, ‘largely meets the standard’ or ‘meets the standard’. Inspectors base their ratings on at least three different sources per item, including conversations with care staff, care users or family members; observations of care; and findings from care user-files and other documents. For the analyses in this study, we calculated the aggregated rating per theme for each LTC home. We first dichotomised the scores for the items as ‘does not (or does not largely) meet the standard’ (0) and ‘(largely) meets the standard’1 and then summed the scores per theme and for the three themes for each LTC home.

    View this table:
    Table 2

    Pattern and structure matrix obtained by principal component analysis with oblimin rotation of the eight items used to assess quality of care by the inspectorate (N=200)

    For 301 LTC homes, a regular rating conducted by the inspectorate between January 2017 and March 2019 was available. To guarantee the independence of the observations, we only included the first LTC home rated for care organisations of which multiple LTC homes were visited. In addition, we did not include follow-up ratings by the inspectorate for LTC homes that did not meet the standard during an earlier assessment, or ratings based on assessments focused on a specific theme.

    In total, 228 of the 301 LTC homes with ratings of the inspectorate were eligible for this study. Thirty-seven (12.2%) of the 301 inspectorate’s ratings of LTC homes were excluded because some items were not scored; for example, due to a lack of time during the assessment or a lack of information required for formal assessment of an item. Of the remaining 264 LTC homes, 36 LTC homes belonged to care organisations for which another LTC home was already included in this study. To guarantee the independence of the observations, we only included the first LTC home assessed if multiple homes run by the same care provider were visited.

    Since no care user ratings were available for 28 of these 228 LTC homes, we used data of 200 LTC homes for which care user ratings were available in the 2 years prior to an assessment by the inspectorate. The number of available care users’ ratings per item ranged from 1 to 225 per LTC home over a period of 2 years (M=31; SD=33). This range of ratings can be explained by the different number of residents living in LTC homes. The included LTC homes had 6–350 residents (M=89; SD=57) and belonged to organisations with 1–40 LTC homes in total (M=6; SD=6).

    Analysis

    First, we conducted Mann-Whitney U tests to compare the ratings of the LTC homes that were assessed by the inspectorate and LTC homes that were not assessed in order to investigate the generalisability of our results to the group of LTC homes that was not assessed. Since the LTC homes that were not assessed lacked a rating date, we included all care users’ ratings from 2 years prior to the date of the first rating by the inspectorate included in this study up to the date of the last rating. For these analyses, we only included the first LTC home that received a rating of care users in each care organisation.

    Next, principal component analysis (PCA)23 was used to check the distribution of the scores for the items assessed under the three themes that the inspectorate aims to rate. In PCA, items that share the most common explained variance cluster together. Oblimin rotation24 was applied as we expected that the different themes would correlate with each other.

    To investigate our hypothesis, whether care users’ and the inspectorate’s ratings of quality of care of LTC homes show at least some overlap, we calculated the Spearman rank correlations between the overall mean care users’ ratings and the inspectorate’s scores for each theme. We included all LTC homes, also those with only a few ratings, to ensure the inclusion of small LTC homes with few residents. However, it allows extreme ratings that would otherwise have averaged out. Therefore, we reanalysed these correlations for a subgroup of the LTC homes, only including LTC homes with at least 30 care users’ ratings to investigate to what extend the results of the full sample are influenced by extreme values.25 To correct for multiple comparisons, we corrected the p values for each analysis using the false discovery rate (FDR) procedure to reduce the type I error. P values were considered significant if their corrected value did not exceed 0.05.26

    We performed post-hoc analyses to obtain more insight into the items that explain the significant correlation, that is, the extent to which care users’ ratings on the six separate items correlated with the theme ‘attention to person-centred care’. As before, we calculated the Spearman’s rank correlations to investigate the extent to which care users’ ratings on the six separate items correlated with the individual themes. Conversely, we also examined the correlations between the scores for the separate items in the significantly correlated themes and the total mean care users’ ratings.

    All analyses were carried out using SPSS IBM V.24.27 The FDR correction was performed using R V.3.6.1.28

    Results

    Comparison of care users’ ratings for included and excluded LTC homes

    The mean total score for the care users’ ratings of the quality of care was 7.9 (N=200, SD=1.1; table 1). Mann-Whitney U tests revealed a weak, but significant, difference between the care users’ ratings for the LTC homes assessed by the inspectorate (M=7.9, N=200) and the LTC homes not rated by the inspectorate (M=8.3, N=219, U=16 310.5, p<0.001).

    PCA of the ratings of the inspectorate

    With respect to the ratings by the inspectorate, we first conducted PCA23 to investigate the consistency of the inspectorate’s framework across the three themes and the eight underlying items. The PCA showed that the three-dimensional solution was consistent with the themes as conceptualised in the original framework, except for item 6 ‘The care organisation ensures that sufficient expert caregivers are available, geared to the older people’s present and current care needs’ (see table 2). This item had the highest loading on the theme ‘focusing on quality and safety’, but was originally conceptualised as part of the theme ‘working towards sufficient and competent care staff’. The three principal components explained 40.7%, 16.0% and 11.3% of the variance, respectively. Based on the face validity of these results, we used this new structure instead of the original; thus, item 6 was included in the theme ‘focusing on quality and safety’ (see table 2).

    The aggregated scores on the three themes of the framework used by the inspectorate indicated that the mean aggregated ratings for ‘attention to person-centred care’ are higher than the mean aggregated ratings for the two other themes (table 3).

    View this table:
    Table 3

    Aggregated ratings of the inspectorate for the themes of the quality of care in LTC homes (N=200)

    Correlations between care users’ and the inspectorate’s ratings

    We observed a weak, significant positive correlation between the mean care users’ ratings and the aggregated ratings by the inspectorate for the theme ‘attention to person-centred care’ (r=0.26, N=200, padj<0.01). Thus, higher ratings of care users were related with higher ratings of the inspectorate of the theme ‘Attention to person-centred care’. No other correlations were significant (padj>0.05; table 4). Similar results were obtained when only LTC homes with more than 30 ratings were included (N=75; 32.8%). However, after FDR correction, the correlation between care users’ ratings and the inspectorate’s ratings of the theme ‘attention to person-centred care’ was no longer significant.

    View this table:
    Table 4

    Spearman’s rank correlations between care users’ and the inspectorate’s ratings on the quality of care provided in LTC homes with ≥1 user ratings (N=200)

    Since there was a significant correlation between the total care users’ ratings and the scores on the inspectorate’s theme ‘attention to person-centred care’, we next carried out a post-hoc analysis (see table 4). We found weak, significant correlations between the care users’ ratings for each of the six individual items and the total scores on inspectorate’s theme ‘attention to person-centred care’ (r=0.21–0.30, padj<0.01). We also analysed the correlation between the mean total care users’ ratings and the scores for the three individual items of the inspectorate’s theme ‘attention to person-centred care’. Weak, positive correlations were observed between the care user’s ratings and each individual item (r=0.16–0.24, padj<0.5).

    Discussion

    Key findings

    This study explored the relationships between care users’ ratings of the quality of care provided in LTC homes in The Netherlands and ratings of the Dutch Health and Youth Care Inspectorate. Ratings of both care users and the inspectorate were available for 200 LTC homes.

    We observed a weak, positive correlation between care users’ and the inspectorate’s ratings of the quality of care, but only for the inspectorate’s ratings on the theme ‘attention to person-centred care’. This finding is consistent with our hypothesis that there would be some overlap, particularly with respect to person-centred care. We did not find any relationship between care users’ and the inspectorate’s ratings for the two other themes assessed during inspection visits: ‘working towards sufficient and competent care staff’ and ‘focusing on quality and safety’. We observed a non-significant relationship of a similar size after excluding LTC homes with less than 30 care users’ ratings. This finding shows that although the power was too low to find a significant correlation of this size for this subsample, a similar relationship seems present. The correlation we found for the full sample was thus not solely caused by extreme values for LTC homes with only a few ratings.

    Post-hoc analyses revealed weak to low-to-medium correlations between the individual items of care in the user ratings and the inspectorate’s scores for the theme ‘attention to person-centred care’. Care users’ rating of the items ‘Alignment of care to own life’ showed the highest correlation with the inspectorate’s scores on the theme ‘attention to person-centred care’. Conversely, weak but significant correlations were observed between the total care users’ ratings and the three individual items of the theme ‘attention to person-centred care’.

    Reflection on findings

    Our findings are in line with earlier studies conducted in the USA and Great Britain, which reported discrepancies and weak to low-to-medium correlations between care users’ ratings of the care provided by nursing homes on social media or online review sites and regulators’ ratings16 17 29

    The weak correlations might be partly explained by the duration of the measurements, since the care users’ ratings covered a period of 2 years prior to the ratings of the inspectorate. We cannot rule out the impact of this time period on the quality of care. However, analysis of a shorter time period, for example, 1 year was not considered suitable, as the number of available care users’ ratings would have been substantially lower. Nevertheless, it seems unlikely that the duration of measurement alone is responsible for the weak relationship between the care users’ and inspectorate’s ratings.

    There is no evidence that gaming, whereby ratings are artificially manipulated,17 may play a significant role in the outcomes of this study. The patient rating website for the quality of care in The Netherlands is an independent website run by a not-for profit organisation, the Patient Federation, that verifies the quality of the ratings that are posted. Only ratings that comply with the code of conduct are posted online. About 90% of the submitted ratings are approved immediately.21 30 To prevent posting of improper (ie, junk or spam) reviews, the editorial office examines all reviews before publishing. In addition, the email addresses and IP addresses of the reviewers are also checked to prevent improper use and duplicates.21 This increases the quality of the ratings that have been posted.13

    The small size of the correlation found in this study between care users’ ratings of the quality of care in LTC homes and the inspectorates’ ratings of person-centred care at least raises the question whether the inspectorate takes sufficient account of the care users’ experiences during their assessment of the quality of LTC. This finding suggests there is room for the inspectorate to intensify or innovate approaches to address care users’ experiences in their assessments of the quality of care to do them justice.

    Strengths and limitations of the study

    One methodological strength of this study is the sample size of 200 LTC homes. All LTC homes were from different providers, and the sample represents about 40% of all care providers with one or more LTC homes in The Netherlands known to the inspectorate.31 However, the LTC homes that were assessed by the inspectorate were not selected at random; therefore, the sample is not necessarily representative. The selection process for inspection also includes LTC providers where risks were foreseen,6 15 which may have contributed to the slightly lower care users’ ratings for the LTC homes that were assessed by the inspectorate compared with the LTC homes that were not assessed.

    The fact that the ratings of LTC homes were obtained from the independent, non-commercial ZorgkaartNederland website is also a strength of this study. The threshold to post a review is low; in other words, anyone can post a review. Although this could potentially lead to misuse of the platform, the ZorgkaartNederland editorial office verifies the legitimacy of each review before publishing the reviews online, which increases the validity of the ratings.13 21 However, it is a limitation that the care users’ ratings are published anonymously on the ZorgkaartNederland website. ZorgkaartNederland does not ask for or publish any characteristics from people who place a rating. Therefore, analyses of the relationships between the ratings of care users (or their representatives) and any of their personal characteristics were not possible.

    Another strength of this study is that we could relate the inspectorate’s ratings for different aspects (themes) of the quality of care, to care users’ ratings and examine the differences in the strengths of these relationships. In contrast, earlier studies only described general ratings of the quality of care by inspectorates.17

    A last point to be addressed is that all three authors of this study are paid staff of the Dutch Health and Youth Care Inspectorate. However, in this respect, it is important to mention that all the data used in this study are publicly available, the methods used to analyse the data are transparent and the interpretation of the findings highlight the need for reflection on the own assessment process of the Dutch Health and Youth Care Inspectorate.

    Recommendations for further research

    Although the Dutch Inspectorate values and attempts to incorporate the experiences of care users and their family members in their assessments, this study shows that their ratings of LTC homes are only weakly related to the experiences of care users based on ratings from the Dutch patient rating site ZorgkaartNederland. Integration of the experiences of care users in the inspection process is challenging, as previous research has shown.6 9–11

    Although a variety of approaches are currently used to promote patient and family involvement in healthcare regulation, the extent to which these approaches are incorporated into regulatory procedures is unclear. It might be fruitful to intensify some of these approaches; for example, by systematically collecting data on care users’ experiences during inspection visits and investigate how they can be embodied in regulatory practice.

    In addition, new regulatory approaches may be needed to do justice to care users’ experiences. For example, regulators may need to move from an outcome-based approach to a process-based and reflexive regulatory approach, in which regulators and service providers organise themselves around care users. This could allow regulators to assess the presence and quality of the processes that allow organisations to provide person-centred care.32 33 Further research is needed to develop and substantiate such regulatory innovations with evidence.

    Conclusions

    This study observed a weak correlation between care users’ ratings of the quality of care they receive in LTC homes in The Netherlands and the inspectorate’s ratings of the provision of person-centred care in these homes. No correlations were found between care users’ ratings and inspectorate’s ratings of other aspects of the quality of care in LTC homes. Further research is needed to investigate how healthcare regulators can do justice to care users’ experiences during their assessment of the quality of LTC, particularly person-centred care.

    Data availability statement

    Data are available on reasonable request. Data are avalable on reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    No ethical approval required. Only publicly available data were used. Note: The ESHPM ethical review policy states: Any research conducted by ESHPM-affiliate researchers involving one of the following aspects must be reviewed by the Research Ethics Review Committee (RERC): Human beings; Potential misuse of research results; (Special categories of) personal data; Potential conflict of interest; Non-EU countries; External stakeholders, among which funding organisations; Environment, health and safety issues, including potential harm to researchers (Ethical review | Erasmus School of Health Policy & Management Employees | Erasmus University Rotterdam (eur.nl)). Based on these criteria, our study does not require ethical review.

    Acknowledgments

    We thank the members of the research consultation group at the Dutch Health and Youth Care Inspectorate in The Netherlands for early discussions on this study.

    References

    1. ↵
      1. Pot AM,
      2. Briggs AM,
      3. Beard JR
      . The sustainable development agenda needs to include long-term care. J Am Med Dir Assoc 2018;19:725–7. doi:10.1016/j.jamda.2018.04.009
      OpenUrlPubMed
    2. ↵
      1. Beard JR,
      2. Officer A,
      3. de Carvalho IA, et al
      . The world report on ageing and health: a policy framework for healthy ageing. Lancet 2016;387:2145–54. doi:10.1016/S0140-6736(15)00516-4
      OpenUrlCrossRefPubMed
    3. ↵
      1. WHO
      . Framework for countries to achieve an integrated continuum of long-term care. Geneva: WHO, 2021: 30–11. Available: www.who.int/publications/i/item/9789240038844
    4. ↵
      1. OECD
      . Health at a glance 2021: OECD indicators. Paris: OECD Publishing, 2021. doi:10.1787/ae3016b9-en
    5. ↵
      ICJ. toetsingskader voor zorgaanbieders waar mensen wonen die langdurige zorg nodig hebben [Utrecht]. 2018. Available: www.igj.nl/zorgsectoren/verpleeghuiszorg/publicaties/toetsingskaders/2017/07/01/toetsingskader-voor-zorgaanbieders-waar-mensen-wonen-die-langdurige-zorg-nodig-hebben
    6. ↵
      1. Wiig S,
      2. Rutz S,
      3. Boyd A, et al
      . What methods are used to promote patient and family involvement in healthcare regulation? a multiple case study across four countries. BMC Health Serv Res 2020;20:616. doi:10.1186/s12913-020-05471-4
    7. ↵
      1. Braithwaite J,
      2. Makkai T
      . Can resident-centred inspection of nursing homes work with very sick residents? Health Policy 1993;24:19–33. doi:10.1016/0168-8510(93)90085-4
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Fricker M
      . Epistemic injustice: power and the ethics of knowing. Oxford: Oxford University Press, 2007. doi:10.1093/acprof:oso/9780198237907.001.0001
    9. ↵
      1. Adams SA,
      2. Paul KT,
      3. Ketelaars C, et al
      . The use of mystery guests by the dutch health inspectorate: results of a pilot study in long-term intramural elderly care. Health Policy 2015;119:821–30. doi:10.1016/j.healthpol.2015.02.008
      OpenUrlPubMed
    10. ↵
      1. de Graaff MB,
      2. Stoopendaal A,
      3. Leistikow I
      . Transforming clients into experts-by-experience: a pilot in client participation in dutch long-term elderly care homes inspectorate supervision. Health Policy 2019;123:275–80. doi:10.1016/j.healthpol.2018.11.006
      OpenUrl
    11. ↵
      1. Richardson E,
      2. Walshe K,
      3. Boyd A, et al
      . User involvement in regulation: a qualitative study of service user involvement in care quality commission inspections of health and social care providers in england. Health Expect 2019;22:245–53. doi:10.1111/hex.12849
      OpenUrl
    12. ↵
      1. Greaves F,
      2. Pape UJ,
      3. King D, et al
      . Associations between internet-based patient ratings and conventional surveys of patient experience in the english NHS: an observational study. BMJ Qual Saf 2012;21:600–5. doi:10.1136/bmjqs-2012-000906
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Kleefstra SM,
      2. Zandbelt LC,
      3. Borghans I, et al
      . Investigating the potential contribution of patient rating sites to hospital supervision: exploratory results from an interview study in the netherlands. J Med Internet Res 2016;18:e201. doi:10.2196/jmir.5552
    14. ↵
      1. Griffiths A,
      2. Leaver MP
      . Wisdom of patients: predicting the quality of care using aggregated patient feedback. BMJ Qual Saf 2018;27:110–8. doi:10.1136/bmjqs-2017-006847
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Kool RB,
      2. Kleefstra SM,
      3. Borghans I, et al
      . Influence of intensified supervision by health care inspectorates on online patient ratings of hospitals: a multilevel study of more than 43,000 online ratings. J Med Internet Res 2016;18:e198. doi:10.2196/jmir.5884
    16. ↵
      1. Johari K,
      2. Kellogg C,
      3. Vazquez K, et al
      . Ratings game: an analysis of nursing home compare and yelp ratings. BMJ Qual Saf 2018;27:619–24. doi:10.1136/bmjqs-2017-007301
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Li Y,
      2. Cai X,
      3. Wang M
      . Social media ratings of nursing homes associated with experience of care and “ nursing home compare” quality measures. BMC Health Serv Res 2019;19:260. doi:10.1186/s12913-019-4100-7
    18. ↵
      1. de Boer B,
      2. Beerens HC,
      3. Katterbach MA, et al
      . The physical environment of nursing homes for people with dementia: traditional nursing homes, small-scale living facilities, and green care farms. Healthcare (Basel) 2018;6:137. doi:10.3390/healthcare6040137
    19. ↵
      Government of the Netherlands. nursing homes and residential care: goverment of the Netherlands. 2022. Available: www.government.nl/topics/nursing-homes-and-residential-care
    20. ↵
      1. ZorgkaartNederland
      . ZorgkaartNederland. 2022. Available: www.zorgkaartnederland.nl/
    21. ↵
      1. ZorgkaartNederland
      . Hoe werkt zorgkaartnederland. 2021. Available: www.zorgkaartnederland.nl/content/hoe-werkt-zorgkaartnederland
    22. ↵
      1. Zorginstituut-Nederland
      . Kwaliteitskader verpleeghuiszorg samen leren en verbeteren. 2021. Available: www.zorginzicht.nl/binaries/content/assets/zorginzicht/kwaliteitsinstrumenten/kwaliteitskader-verpleeghuiszorg---2021.pdf
    23. ↵
      1. Todorov H,
      2. Fournier D,
      3. Gerber S, et al
      . Principal components analysis: theory and application to gene expression data analysis. Genomics and Computational Biology 2018. Available: https://nbn-resolving.org/urn:nbn:de:101:1-2018100619033322703228
    24. ↵
      1. Lorenzo-Seva U
      . The weighted oblimin rotation. Psychometrika 2000;65:301–18. doi:10.1007/bf02296148
      OpenUrl
    25. ↵
      1. Grimm LG
      . Statistical applications for the behavioral sciences. New York: J. Wiley, 1993.
    26. ↵
      1. Glickman ME,
      2. Rao SR,
      3. Schultz MR
      . False discovery rate control is a recommended alternative to bonferroni-type adjustments in health studies. J Clin Epidemiol 2014;67:850–7. doi:10.1016/j.jclinepi.2014.03.012
      OpenUrlCrossRefPubMed
    27. ↵
      1. IBM Corporate
      . IBM SPSS statistics for windows. 24.0. Armonk, NY: IBM Corp, 2016.
    28. ↵
      1. Team RC
      . R: A language and environment for statistical computing. In: Computing Rf, ed. version R 3.6.1 ed. Vienna, Austria, 2019. Available: https://www.R-project.org/
    29. ↵
      1. Williams A,
      2. Straker JK,
      3. Applebaum R
      . The nursing home five StAR rating: how does it compare to resident and family views of care? Gerontologist 2016;56:234–42. doi:10.1093/geront/gnu043 Available: https://doi.org/10.1093/geront/gnu043
      OpenUrlCrossRefPubMed
    30. ↵
      1. ZorgkaartNederland
      . Gedragscode ZorgkaartNederland 2020, Available: www.zorgkaartnederland.nl/content/gedragscode
    31. ↵
      ICJ. verpleeghuiszorg beeld: inspectie gezondheidszorg en jeugd; ministerie van volksgezondheidwelzijn en sport. 2021. Available: www.igj.nl/zorgsectoren/verpleeghuiszorg/verpleeghuiszorg-in-beeld
    32. ↵
      1. de Kam D,
      2. van Bochove M,
      3. Bal R
      . Disruptive life event or reflexive instrument? on the regulation of hospital mergers from a quality of care perspective. J Health Organ Manag 2020;ahead-of-print(ahead-of-print):489–503. doi:10.1108/JHOM-03-2020-0067
    33. ↵
      1. Gilad S
      . It runs in the family: meta-regulation and its siblings. Regulation & Governance 2010;4:485–506. doi:10.1111/j.1748-5991.2010.01090.x
      OpenUrlCrossRef

    Footnotes

    • Contributors FP, KW and AMP designed the study. FP and KW carried out the analyses under supervision of AMP, FP prepared the first draft of the manuscript, KW and AMP reviewed and revised the manuscript. All authors have read and approved the final version of the manuscript. AMP is responsible for the overall content as guarantor.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests All three authors are paid members of staff of the Dutch Health and Youth Care Inspectorate.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.