Emilia-Romagna Regional Blood System accreditation as an example of improvement through application of specific requirements: a retrospective analysis

Participants

We initially presented this project to the Regional Blood System Quality Workgroup made up of all the Quality Managers of Hospitals and Associations.

The Regional Blood Centre then invited the Emilia-Romagna Hospitals and Associations to voluntarily and anonymously participate in this study. Participants authorised the use of the data collected for publication in aggregate and anonymous form.

Subject of the study

We reviewed the first accreditation inspection (2013–2014) and the two following maintenance inspections (2015–2016 and 2017–2018). We evaluated the state of Branches (active, newly opened and closed) and NCs observed by the regional inspectors for each of these assessments. The checklist of specific requirements is defined in the Agreement between State and Regions of 16 December 2010,41 which, in our region, was implemented in the Regional Committee Resolution number 819 of 13 June 2011.42 During the inspections, general requirements, common to every health organisation, were also evaluated, but they were not the subject of this study. As for NCs, we specify that in the Emilia-Romagna accreditation model, the level of compliance to each requirement in the checklist can be expressed with a scale of four values: ‘YES’ (compliant), ‘yes’ (partially compliant), ‘no’ (non-compliant—work in progress), ‘NO’ (non-compliant). Requirements can concern an entire organisation or its Branches and can have a value of ‘NA’ if not applicable to that organisation or Branch.54 For this study, we considered only requirements with ‘no’ or ‘NO’ values in the checklist, which, therefore, represent NCs.

Data entry

Participants had to access a web application that we developed specifically. For each of the three inspections evaluated in this study, it was necessary to enter: (1) the number of active branches; (2) the number of branches closed or newly opened since the previous inspection; (3) the new values of NCs observed in the previous inspection (‘YES’, ‘yes’, ‘no’, ‘NO’ or ‘NA’); (4) new NCs (‘no’ or ‘NO’). These values had to be drawn from the inspection reports received by the Regional Health and Social Agency, the technical body that assesses compliance with the accreditation requirements on behalf of the region.

To maintain the anonymity of the participants, they were marked only with their type of facility (hospital or association) and a unique alphanumeric code generated at the first access to the web application. Branches were marked with a progressive number generated by the web application and an optional alphanumeric code, chosen by the participants to distinguish them more efficiently when entering data. Participants could receive technical support from the administrators only through a messaging system in which they were identified by their unique alphanumeric code.

Data were collected between 3 May 2019 and 31 January 2020. Participants could choose to enter data in multiple sessions, each time accessing the web application with their unique alphanumeric code.

Data analysis

All data entered were exported from the web application database and analysed with spreadsheets. The NCs were grouped by the category of their requirements: structural, technological and organisational. The analysis was made both overall and separately for Hospitals and Associations to highlight possible peculiarities of each type of organisation.

Statistical analysis was performed using R software (V.4.1.0); we used Wilcoxon signed rank test with continuity correction to compare the number of NCs in each branch in the first (2015–2016) and second (2017–2018) maintenance inspections versus the number of NCs in each branch in the first accreditation inspection (2013–2014).

Patient and public involvement

It was not appropriate to involve patients or the public in the design, or conduct, or reporting or dissemination plans of our research, since it was based on the analysis of institutional inspections reports.

Participants

Seventeen organisations took part in the study, 11 (65%) out of 11 hospitals in the region (100%) and 6 (35%) out of 9 associations in the region (67%).

Participating structures had a total of 174 branches, 42 belonging to hospitals (24%) and 132 belonging to associations (67%).

Organisation branches

The number of branches changed over the years because of new openings and closures due to reorganisations or NCs that were too difficult to correct. Table 1 shows this data in detail.

The number of branches for participating hospitals remained essentially unchanged from 2013 to 2018; the number of branches for participating associations went from 130 in 2013 to 108 in 2018 (−17%).

Non-conformities

Table 2 contains the NCs grouped by category (structural, technological or organisational), type of structure (hospital or association) and year of inspection. There were many more NCs for associations, but it must be considered that they also had many more branches than hospitals; therefore, to better compare the two types of structures, in table 3, we report the mean number of NCs per branch.

Overall, the inspectors observed 2381 NCs, 21% of which for hospitals and 79% of which for associations. However, the number of NCs per branch was comparable between hospitals and associations since 76% of branches belonged to associations.

Following the accreditation inspections and the consequent improvement actions, the NCs were reduced by 88% overall, more for associations (−91%) than for hospitals (−76%).

Over 50% of NCs were organisational, both for hospitals and for associations. Hospitals were affected by structural NCs more than associations (24% vs 9%), especially in the first 2-year period.

The structural NCs were overall reduced by 97% (89% already by the second inspection). While associations resolved all of their structural NCs, hospitals still had some of these problems as of 2018.

The technological NCs were overall reduced by 91% (82% already by the second inspection). We highlight a significant increase in the number of technological NCs for hospitals in the 2017–2018 period (26 NCs), mostly related to devices for the transportation of blood and blood components.

The organisational NCs were overall reduced by 85% (77% already by the second inspection). Their resolution rate was the lowest of the three categories of observed NCs.

Table 4 shows the most frequent NCs that most affected the Regional Blood System, listed separately by hospitals and associations.

The NCs relating to management software were the most critical since they remained substantially unchanged over time. NCs concerning the transportation of blood and blood components and its validation were considerably reduced, although there was still room for improvement. NCs concerning scales for blood collection were eliminated thanks to public tenders carried out by groups of local health authorities (the so-called wide areas).

Table 5 shows the NCs remaining after the last inspection in 2018, grouped by category (total and mean number per branch).

Statistical analysis

We used a paired difference test, so we had to discard 28 of the 174 branches that were not open on all of the three inspections (3 newly opened and 25 closed). We, therefore, made calculations on the remaining 146 branches.

If compared with the first accreditation inspection (2013–2014), the number of NCs in branches was significantly lower both in 2015–2016 (p<2.2e-16) and in 2017–2018 (p<2.2e-16). We repeated the same analysis stratifying according to the type of facility (hospitals and associations), the category of NCs (structural, technological and organisational) and both criteria simultaneously. In all these cases, we got p values ≤0.001. This data show that there was a statistically significant improvement already by the second inspection, not only for the whole regional Blood System but also both for hospitals and associations and in every category of NCs.

We finally tried to perform the same analysis on organisations instead of branches, but the sample became thereby too small to give significant p values for all of the above comparisons. In particular, comparisons concerning the six associations gave p values ranging from 0.05 and 0.07.

Current evidence about the impact of accreditation

In 2020, Araujo et al conducted the most recent systematic review about the impact of accreditation on hospital outcomes. They stated that previous literature reviews gave controversial results: not all of them found evidence to support a link between hospital accreditation and measurable changes in healthcare quality indicators. This could be explained in part by the complexity of accreditation programmes and in part by methodological differences between the selected studies. Therefore, they chose a more systematic approach, analysing only studies that quantitatively examined differences in health quality indicators before versus after hospital accreditation or among accredited versus non-accredited hospitals. In particular, they watched for changes in seven healthcare quality dimensions and concluded that accreditation may have a positive impact on efficiency, safety, effectiveness, timeliness and patient-centeredness. In turn, only one study analysed the impact on access, and no study had investigated the impact on equity dimension yet.12

Compliance with requirements and actual improvement

We too followed a quantitative approach in a sense, since we analysed the number of NCs over time. However, we believe that the accreditation process does not only consist in complying with a list of requirements but also has led to an actual improvement of the Blood System quality. Here are some examples: (1) Electronic scales allow for complete traceability of blood units volume, collection time and operators involved. Thanks to this, organisations can quickly identify non-conforming units and verify that operators maintain their competence, (2) continuous monitoring of transportation temperature guarantees the safety and efficacy of blood components, (3) the adjustment of premises for blood collection has made it possible to define paths ensuring the privacy and security of donors, operators and products.

Compliance maintenance over time

Compliance is maintained on a daily basis, even outside the inspections, through continuous monitoring of the activity with: (1) process and outcome indicators; (2) internal audit following relevant NCs, changes, developments, and in any case, at least every year. There are, in fact, some organisational requirements that explicitly dictate: (1) periodic management reviews to highlight quality problems and the need for corrective and preventive actions and (2) internal audits to verify the compliance with current legislation, standards and procedures.

Differences between hospitals and associations

Hospitals had already undergone accreditation since the early 2000s but limited to general requirements. Thanks to this, they had already developed a quality management system and were more ready than associations to meet specific organisational requirements. In fact, in the first 2-year period, hospitals had four organisational NCs per branch compared with seven for associations. At the end of the analysed period, however, both types of structures had 1.1 organisational NCs for branch: this testifies for the commitment of the associations and a solid organisational synergy between the two kinds of structures, which is one of the objectives of the Blood System accreditation.

In the first 2-year period, on the other hand, associations were less affected by structural NCs than hospitals (1.2 per branch vs 2.3 per branch) and resolved them more effectively than hospitals, partly through reorganisations that led to the closure of some branches. In fact, public premises are more complex to modify because they also host healthcare activities other than blood collection. In addition, each structural change requires a bureaucratic procedure and a public tender for transparency reasons. On the contrary, blood donor associations are private subjects with a more agile organisation than public health service: they can more readily approve structural changes through a governing council, subject to financial availability.

For other aspects, the possibility of carrying over public tenders is one of the strengths of the hospitals and also the associations can take advantage of them. For example, regional or wide area tenders contributed effectively and uniformly to resolving some technological issues, especially the ones concerning the scales for blood collection.

Future prospects

We believe that more public tenders for goods and services could prove helpful for closing the remaining organisational and technological NCs, such as those concerning the management software and the transportation of blood and blood components. However, this approach alone will not be enough: also the discussion and collaboration between head physicians and quality managers of hospitals and associations will be essential to close the remaining organisational NCs, contributing to the regional standardisation of processes and, therefore, to the maturity of the whole system.

Strengths and limitations

To our knowledge, this is the first study in Italy to evaluate the Blood System’s change over time and to analyse so many blood organisations altogether.

Three associations out of nine did not participate in the study; therefore, the collected data are significant but not complete. In addition, during the last maintenance inspection (2017–2018), the region did not visit all branches on-site to check the status of NCs observed in the previous inspection (2015–2016). In these cases, the Regional Health and Social Agency settled for the self-assessment checklists submitted by hospitals and associations in the preinspection phase. Compliance values for these requirements are, therefore, possibly biased.