Article Text
PDF
Abstract
The ability to measure the extent to which an organisation is highly reliable, or the extent to which reliability may change over time, has not kept up with the development of theory. The paper examines aspects of workplace culture, employee motivation and leadership behaviours that support continuous learning and improvement in an effort to measure the transition to high reliability.
To evaluate the effectiveness of its high reliability initiative, one children’s hospital sought to build measures that would provide an assessment of progressive movement towards a ‘culture of safety’, and track the success over time. This paper reports on the development of two scales (trust in team members and trust in leadership) that are intended to measure two cultural conditions fostered by the five high reliability principles and a composite measure on local learning activities. The two scales are strongly associated with local learning activities in employees’ work areas and with employees’ willingness to participate in extra role activities. We suggest that they are foundational to creating a psychologically safe environment and thus to becoming a high reliability organisation.
- patient safety
- safety culture
- hospital medicine
- paediatrics
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
The ability of organisations to measure progress towards becoming highly reliable has been limited and generally focused on organisation-wide changes.
WHAT THIS STUDY ADDS
This study offers new measures of workplace culture from the perspective of employees, and examines their relationship with behaviours that enable progress towards high reliability.
HOW THIS MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
By using these new measures, it is hoped that organisations will be able to identify specific areas where cultural change could enable and support their progress towards becoming a highly reliable organisation.
Introduction
The ability to measure the extent to which an organisation is highly reliable, or the extent to which it may become more or less highly reliable over time as a result of interventions or safety boundary drift, has not kept up with the development of theory.1 High reliability organisations (HRO) foster five principles which lead to highly reliable behaviours—deference to expertise, preoccupation with failure, reluctance to simplify, sensitivity to operations and practice resilience.2 However, the lack of quantitative measurement of these five high reliability principles has been noted.3 The current paper examines how to measure two aspects of workplace culture (trust in team members and trust in leadership) and their association with employee motivation to step outside their formal role (extra role motivation) and a set of behaviours that local leaders engage in to support continuous learning and improvement (termed here ‘local learning activities’). We posit that these two aspects of workplace culture are key components of a psychologically safe environment, which is a foundation to being able to foster and enable the five high reliability principles.
What is a ‘culture of safety’?
The development of the notion of HROs was launched in the mid-1980s, with the publication of a case study of three organisations (nuclear power plants, air traffic control systems and naval aircraft carriers) that showed remarkable safety and resilience.4 In these organisations, the cost of error would have been catastrophic for the public. The three organisations are part of the national critical infrastructure and are hence highly regulated (which may contribute to their excellent safety records). Their error records were and remain impressive. Later work, most notably by Sutcliffe and Weick,2 built a theoretical framework around this foundational case study which did not presuppose the highly regulated context (thus did not presuppose having a powerful external driver), but that nonetheless was and is quite plausible. In it, the authors sought to detail how HROs are different from organisations that are not highly reliable.
The theoretical work has been adapted to a wide range of industries, including healthcare, seeking to achieve a similarly reliably low error rate.5–7 However, Chassin and Loeb6 argue that while HRO principles are relevant to hospitals, a framework for high reliability that is drawn specifically for the hospital environment, which they sketch out, is necessary for hospitals to achieve high reliability. The complexity of patient needs, the many avenues through which errors and safety risks can reach patients, the traditional role of expertise in healthcare and other factors all weigh against a simple translation of the high reliability work from Sutcliffe and Weick to hospitals. The Chassin and Loeb framework is based on the engagement of safety leaders from a wide range of hospitals. It sets out a maturity model for three dimensions: leadership, safety culture and robust process improvement. Each dimension has a number of domains, each of which contain descriptions of HRO maturity. The safety culture dimension includes the following domains: trust, accountability, identifying unsafe conditions, strengthening systems and assessment.
Even with such a useful framework, the operationalisation of these HRO principles and ways of doing business have proved thorny. Organisational assessments that measure ‘HRO-ness’ typically do so at the organisation level.3 8 9 With some exceptions, questionnaires are completed by staff who are asked to report on organisational policies, monitoring and accountabilities—not their own behaviour or how their behaviour is mediated by those system-wide efforts. These tools are useful for organisational self-reflection and planning at the system level, but do not appear to offer a route to quantifying progress towards becoming an HRO. That is, they do not measure the cultural changes that underpin high reliability culture within the employees’ ‘felt’ work environment.
High reliability requires systems to be in place that reproduce the conditions in which error-free work can be conducted without fail—such systems are structured around policies and standard procedures, but are brought to life by nurturing a specific type of culture.10 As Chassin and Loeb point out, this safety culture is dominated by a feeling of trust and accountability,6 or as others have pointed out, the work culture must be dominated by psychological safety, where ‘speaking up’ is the cultural norm and mistakes are viewed as learning opportunities.11 Indeed, where staff members do not feel psychologically safe, staff members are less likely to report errors that they later admit they witnessed.12 This leads to lower reporting of near misses and small errors but results in more significant risks to patients. Thus, a psychologically safe culture supports adherence to policies and procedures, on the one hand, and energises the kind of mindfulness among staff that enable them to identify when the policy or procedure is not helping, on the other hand. Such a culture creates the psychologically safe space where it is understood that there are moments when independent action is needed (eg, the HRO principle sensitivity to operations), and the social norms by which individuals with the deepest knowledge and skills are supported in independent action in those moments (eg, deference to expertise and preoccupation with failure). The highly reliable organisation must embrace two sides of the coin—policies/procedures and a culture of safety. Thus, to grasp progress towards the fuller adoption of the HRO principles, it is not sufficient to assess policies and procedures; leaders also need tools to measure culture and cultural change.
Another complicating factor is that the drivers of this culture are quite interdependent, and that the causal relationships are not clear. Outside of the HRO literature, many research teams have attempted to measure and understand the relationship between safety and certain aspects of organisation culture. Liu et al13 attempt to link employee trust in the organisation’s safety infrastructure and the employee’s engagement in safety behaviour. The research team found no statistically significant direct connection between the two, but were able to identify the mediating role of psychological safety and sense of belonging. What one might consider a partially contrary finding, Kath et al14 found that trust mediated the relationship between employee safety motivation and outcomes. Avram et al15 report that job satisfaction mediates organisational trust and employee’s perception of the organisation’s policies and practices. Turning from trust to psychological safety, researchers have examined the relationship between safety and leader inclusiveness,16 between employee ‘voice’ and extra role behaviours17 and between employee ‘voice’ and safety-specific leadership.18 What is not clear is how these concepts weave together, which reflect actions employees can take or leaders can take, what is the impact of those actions on the build-up of a culture that gives HRO policies and practices sticking power?
This paper attempts to fill in gaps and coordinate across concepts to provide a robust foundation for measuring organisational culture change that enables HRO achievement. The paper examines two aspects of workplace culture (trust in team members and trust in leadership) and their association with two behaviours. The first behaviour is within the power of any employee to embrace, that is, extra role motivation (the willingness of staff members to step out of their formal role to support the organisation), and the second behaviour is within the power of leaders, setting up local learning activities (a set of behaviours that local leaders engage in to support continuous learning and improvement).
Building a culture of safety
This paper reports measures developed in support of one children’s hospital to become a highly reliable organisation. An internal leadership group was developed to create a framework and action plan to implement high reliability principles across the organisation. After completing a key driver diagram process and developing a logic model that tied planned activities to desired outcomes,19 the HRO planning team concluded that the prime measure of their success would be that we fostered a ‘culture of safety’ and that foundational to the culture of safety was establishing a psychologically safe work environment where trust in leaders and trust in peers would be the norm.
Turning to the literature on published practices, the needs assessment and an internally developed key driver diagram, the planning team committed to a multimodal implementation process. The intervention includes (1) safety science and error prevention training, (2) safety coaching, (3) proactive safety rounds, (4) quality improvement and analytics training, (5) local learning initiatives, and (6) a just culture algorithm based on appreciative inquiry to enhance the root cause analysis process.
These efforts would be scaffolded on top of robust targeted interventions and related reporting systems that had been put into place for safety risks related to patient care, such as hand hygiene, central line bloodstream infections, falls and self-harm. Multidisciplinary intervention teams would continue to implement the Joint Commission’s Robust Process Improvement framework to identify specific risks and to reduce variability in practice in parallel with the multimodal implementation plan for high reliability.
The HRO effort would complement the organisation-wide policies and procedures and guide leaders in the development of specific safety-enhancing behaviours among staff. The ultimate goal was to create an environment where staff members would feel motivated to challenge procedures and processes when they felt it was warranted, and where these challenges would support local learning as part of continuous quality improvement.
While it is commonplace to think of culture as a single, hard-to-define force, the internal planning team was able to identify specific cultural changes we wanted to encourage using the key driver diagram we developed. The schema in figure 1 describes how the elements that drive the culture of safety are understood by the internal planning group. The multimodal HRO effort seeks to trigger organisation-wide cultural changes, specifically expanding trust in leadership and trust in team members. Two behaviours (one at the supervisor level and another at the staff level) were seen to feed into to support psychological safety. As this ‘virtuous cycle’ develops, it is theorised, the culture of safety will take hold and expand.
Building the culture of safety and enabling high reliability behaviours.
Methods
The data presented in this paper were gathered through the biennial employee engagement survey in one large independent children’s hospital in the Midwest of the USA. The purpose of the biennial survey is to gauge employee engagement and to identify departments and units in the hospital that may be in need of leadership or other supports.
The survey was prepared and conducted by a survey vendor; the hospital developed and added items related to evaluating its HRO efforts. These latter items are the ones being analysed in this paper.
Patient and public involvement
Patients, patient families and the public did not participate in the development of the study.
Item development
The decision was made to use the biennial survey to gather baseline information on the hospital’s current safety culture. The survey is administered to all employees. Because physicians are not directly employed by the hospital, they are excluded from the survey.
The items from the vendor’s survey were assessed to determine whether they reflected one of the domains related to the high reliability principles—deference to expertise, preoccupation with failure, reluctance to simplify, sensitivity to operations and practising resilience. After review of the vendor’s items by internal content experts, the decision was made to review sets of items available in the public domain. In particular, the ORO 2.0 survey,20 the Resilience Analysis Grid21 and the Safety Attitude Questionnaire22 were reviewed by the team.
Either because of length, item focus or redundancy with the vendor’s survey, each of the surveys identified in the public domain was rejected. Instead, the team member serving as the project evaluator developed an initial set of questions that three internal content experts reviewed, revised and expanded on. The internal team settled on 18 candidate items that we believed could measure progress towards a culture of safety.
Once the data were gathered, they were factor analysed and tested for reliability using Cronbach’s alpha. Items that did not contribute to the factor loadings (0.5 or above) or that undermined the reliability scores were examined and reviewed by the project team. The team dropped five items from the 18, resulting in 13 final items. Two items were reverse coded and we assessed that they were not completed accurately by many respondents (many individual respondents appeared to contradict themselves); three items were highly redundant.
The 13 items were further reduced to one composite measure and two brief scales that assess the culture of safety (13 items total): (1) the presence of local learning activities (the composite measure), (2) trust in team members, and (3) trust in leadership. The presence of local learning activities measures the extent to which supervisors actively support a learning environment. The other two (trust in team members and trust in leaderships) measure cultural norms that are understood to be foundational to developing a culture of safety.
A final single item was used to determine extra role motivation for staff. This item ‘I am motivated to go above and beyond what is expected of me in my job’ was determined to be a key individual-level factor in supporting a culture of safety.
All items have 5-point response categories (strongly agree=5 to strongly disagree=1) that are averaged for the total score to make them comparable; all items entered the principal component analysis at 0.5 or higher. The details of these scales are offered in table 1.
Culture of safety scales
Survey response
The employee engagement survey was conducted online and fielded to all non-physician staff over a 2-week period in September 2020. The response rate was 77%.
Table 2 offers an overview of the population surveyed. To simplify the analysis of employee roles, individuals were coded as participating in direct patient care (DPC) at least 50% of the time, and whether or not they were a nurse. Together, these two fields were used to create a single variable that divided the respondents into ‘DPC, nurse’ (28.0%), ‘DPC, not nurse’ (21.2%), ‘Not DPC, nurse’ (7.1%) and ‘Not DPC, not nurse’ (43.8%). About three-quarters of the respondents worked during the day; and about two-fifths were ‘exempt’, which means they earn a salary and do not receive overtime when they work longer hours. Almost 40% of the respondents worked at the hospital 2 years or less (12.4% less than 1 year; 27.5% 1–2 years) and 17.1% worked at the hospital for 16 years or more. The respondents are heavily female (82.6%); 55.2% indicate that they are White, 13.3% that they are Black or African American, 11.2% that they are Latinx, 11.7% report a different racial or ethnic category (this is a combination of Asian, Hawaiian/Pacific Islander and several other categories) and 8.6% report that they preferred not to report their race or ethnicity.
Demographics by outcomes and scales (mean scores)
Analysis
SPSS v. 28 was used to conduct the analysis. Means were compared among key groups (eg, nurses, non-nurses, non-clinical staff; by tenure at the organisation, by gender, by race/ethnicity and by work shift) to understand how these factors were associated with the measures of culture under study. Correlations among the measures of culture and regression analyses were conducted to understand the relatedness of the aspects of culture and to identify covariates.
Findings
Turning to the comparison of means across groups (table 2), in general, the following groups score higher in the three culture measures than other groups: not DPC—nurses (nurses who are not in DPC), workers who work the day shift (except for trust in leadership), exempt (salaried) employees, new employees and males. The one group that is consistently lower across the domains are individuals who preferred not to name their race or ethnicity (n=399, or 8.6% of the total population).
There is a strong association between trust in team members and trust in leadership (table 3). There is also a statistically significant correlation between extra role motivation and trust in team members, though it is less notable than its correlations with trust in leadership and the presence of local learning activities. Indeed, the presence of local learning activities has a strong association with the other measures that track aspects of a culture of safety.
Culture of safety measure correlations
We created two regression equations—the first is to understand what might drive extra role motivation and the second is to understand more fully the connections among the local learning activities and other measures of culture. Results are summarised in table 4. To isolate the impact of the effects of culture from role characteristics (role, shift, exempt status) and personal characteristics (race/ethnicity, sex), groups of variables were entered stepwise. Because of the high level of correlation among the independent variables, tests for collinearity were conducted and it was determined that each variable was contributing enough of its own explanatory power to remain in the equations. In both models, the cultural measures dominated the models, with very little difference in R2 resulting from the addition of the role and personal characteristics.
Regressions (standardised beta coefficients)
The presence of local learning activities explains a large portion of the variability in extra role motivation, although not as much as does trust in team members. Trust in leadership also has a strong hand to play in explaining the variability of extra role motivation. It is useful to note that none of the role characteristics appear to play a hand in the variability of extra role motivation. Being Black/African American or being female has a small association with extra role motivation, holding all else constant. Being someone who ‘preferred not to indicate’ their race or ethnicity had no association with extra role motivation, controlling for all of the variables in the model.
The strength of the regression of local learning activities provided findings that surprised the authors. That trust in team members and the trust in leadership explain a great deal of its variability underlies, perhaps its capacity to support the development of the culture of safety. With a single point in time, we are limited in the ability to explore causation or even theorise a path model, but the strong association is suggestive of the need for additional work.
The other findings in the regression have a much smaller impact on the models (though they remain statistically significant). Being African American or being female is associated with being more motivated ‘to go above and beyond’; being Latinx is modestly and positively associated with reporting the presence of local learning activities. These findings bear reflection in the context of the hospital’s equity, diversity and inclusion efforts. Being exempt and working the day shift are negatively, though modestly, associated with the local learning environment. The finding on the exempt employees likely reflects the fact that a disproportionate number of exempt employees are non-clinical, and participate much less directly in the patient safety efforts. The finding on the day shift employees is contrary to expectations, but may reflect the additional administrative workload that many daytime clinical staff experience. This is an area for further exploration.
Conclusions
We began with two questions, both of which can only be somewhat answered by this paper. (1) How is the ‘culture of safety’ (specifically a psychologically safe workplace) measured? (2) What measures can be used to evaluate whether it is changing over time? Only an answer to the first of these two questions is attempted in this paper. However, the model we have explored presupposes that an organisation’s culture can change in a purposeful way, and that specific types of activities that foster psychological safety will advance it towards a true culture of safety, and thus will enable a highly reliable organisation.
Relying on the research literature in the safety sciences and on our own internal needs assessment and experts, the paper sought to knit together key domains in organisational culture that appeared to enable the organisation to become an HRO. We followed the Chassin and Loeb framework generally, but chose to focus the culture work on building a psychologically safe work environment, which we believed would enable our other efforts towards high reliability.
The paper reports on three newly defined scales, two of which are likely directed at least in part by organisation-wide efforts and one that is in the local leaders’ sphere of influence and control. The first two were intended to capture cultural changes related to psychological safety observed among our staff through the processes of safety improvement efforts (trust in leaders and trust in team members), as well as in reference to the five HRO principles. The third measures the presence of local learning activities within teams across the organisation, something that the HRO initiative is piloting.
As the literature informed our hospital’s planning team and as we report in the introduction, there is a great deal of interdependence among the cultural norms that enable high reliability and little direction in terms of measuring the high reliability principles. We sought to measure cultural dimensions that would enable the high reliability principles. In the case of this paper, the strong correlations among cultural measures suggest that where one is present the others tend also to be present. Even so, it is helpful that each scale measures a different aspect of the underlying culture of high reliability, and perhaps suggests a different set of actionable interventions. For example, in units with a relatively low trust in leadership, developing safety coaches in the unit provides safety intervention training for everyone, including the leaders, and may overtime build more trust between leaders and staff. However, where units show less in trust in peers, instituting more local learning activities would set up a learning environment that would support listening and respectful exchange and thus it may be a helpful tool to improve collaboration and trust among peers.
It is also worth noting that although the correlations are high, from the employee’s perspective, individuals in different roles and with different traits report varying degrees of these experiences (the correlation between trust in team members and trust in leadership is not perfect, and extra role motivation is more strongly associated with trust in leadership than trust in team members). Also, there is room for improvement (few scores were higher than 4.0). For example, in one unit with low trust in leadership, low trust in coworkers and lower than acceptable safety metrics, the hospital leadership intervened with a multimodal approach. Leadership was changed, safety coaches were recruited, trained and supported, and staff were encouraged to participate in quality improvement practices. In the short run, the result was higher error reporting, a wide expansion of local learning activities and process/quality improvement efforts. Because of the expansion of psychological safety, more risks to patients were identified and then new processes could be put into place to improve patient safety.
Finally, in complex and complicated work settings, such as a hospital, many people give instructions to a single staff person in the course of the day. For example, nurses receive instructions from their nursing unit leader, but may also receive instructions from physician specialists or therapists providing service on the unit. Thus, when nurses consider who is in leadership, they may consider a wide range of individuals outside of the official hierarchical structure. Also, because most units in our hospital are team led, it is not always productive to isolate the actions of a specific leader. Nonetheless, it is helpful to provide leaders a concrete action (such as building local learning activities) to foster the broader sense of trust in team members and trust in leadership, and to support the motivation of staff to step outside their role when they believe it is necessary. At a more fundamental level, it is also helpful, as the example in the previous paragraph shows, to hold leaders accountable and assure leaders are in place who will actively foster the psychologically safe space that is needed for identification of risks and attention to them.
Significant caveats are warranted. This study was conducted at a single hospital in a single industry. While it has forwarded the beginning of a framework for the measurement of the culture of safety in an organisation, replication studies are needed. Further, as mentioned earlier, the fact that the data for this paper were collected at a single point in time makes it impossible to speculate about causation or to even propose a causal order for the mechanisms of cultural change. Finally, the fact that physicians could not be included as survey participants limits the ability of the study to describe how all members of the care team perceive or participate in the culture of safety.
In applying these findings to our organisational journey to becoming a highly reliable organisation, we learnt that there are local differences among the overall scores on these scales and that there is room to improve. These have informed the pilot efforts we are making in several units. For example, the HRO leadership training, staff training and interventions have been shaped by the findings. We know that some units and some staff groups need more support around trust in team members, trust in leaders and engagement in local learning activities. As these interventions mature, we plan to measure the extent to which cultural change has occurred and whether that cultural change is associated with improved patient safety.
Taking one step back and looking at the findings from the larger HRO literature, it seems that one path to address the lack of measurement in HRO initiatives is to apply the theories from the literature rigorously in organisations, as has been attempted here. We encourage other organisations to use these items to measure and operationalise their staff’s safety culture as it relates to HRO principles. Operationalising the underlying enabling culture will strengthen both the ability to develop and understand HROs and will strengthen the HRO theoretical literature.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
References
Footnotes
Contributors JC contributed to the development of the survey, analysis and writing. DW acted as guarantor of the study. MG, DK, DH, MAB and DW contributed to guidance, feedback and edits to the survey; development of the conceptual framework for the paper; and review of the paper drafts and final submission.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.