Article Text
PDF
Abstract
Background Frailty is a robust predictor of poor outcomes among patients with chronic obstructive pulmonary disease yet is not measured in routine practice. We determined barriers and facilitators to measuring frailty in a hospital setting, designed and implemented a frailty-focused education intervention, and measured accuracy of frailty screening before and after education.
Methods We conducted a pilot cross-sectional mixed-methods study on an inpatient respiratory ward over 6 months. We recruited registered nurses (RNs) with experience using the Clinical Frailty Scale (CFS). RNs evaluated 10 clinical vignettes and assigned a frailty score using the CFS. A structured frailty-focused education intervention was delivered to small groups. RNs reassigned frailty scores to vignettes 1 week after education. Outcomes included barriers and facilitators to assessing frailty in hospital, and percent agreement of CFS scores between RNs and a gold standard (determined by geriatricians) before and after education.
Results Among 26 RNs, the median (IQR) duration of experience using the CFS was 1.5 (1–4) months. Barriers to assessing frailty included the lack of clinical directives to measure frailty and large acute workloads. Having collateral history from family members was the strongest perceived facilitator for frailty assessment. The median (IQR) percent agreement with the gold-standard frailty score across all cases was 55.8% (47.2%–60.6%) prior to the educational intervention, and 57.2% (44.1%–70.2%) afterwards. The largest increase in agreement occurred in the ‘mildly frail’ category, 65.4%–81% agreement.
Conclusions Barriers to assessing frailty in the hospital setting are external to the measurement tool itself. Accuracy of frailty assessment among acute care RNs was low, and frailty-focused rater training may improve accuracy. Subsequent work should focus on health system approaches to empower health providers to assess frailty, and on testing the effectiveness of frailty-focused education in large real-world settings.
- Health professions education
- Hospital medicine
- Chronic disease management
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is already known on this topic
Frailty is a robust predictor of poor clinical outcomes among patients with chronic obstructive pulmonary disease yet is not measured in routine clinical practice.
What this study adds
This study demonstrates that barriers to assessing frailty, such as large acute workloads and lack of widespread clinical directives in hospital, are external to the measurement tool itself, and the accuracy of frailty assessments is low. A structured frailty-focused educational intervention has the potential to improve the accuracy of frailty assessments among hospitalised patients but requires testing in larger real-world samples.
How this study might affect research, practice or policy
Introducing routine patient frailty assessments to improve patient-centred care models for hospitalised patients first requires system-wide approaches to empower frontline staff with resources including clinical directives and targeted frailty-focused education to ensure accurate measurements.
Introduction
Chronic obstructive pulmonary disease (COPD) is a progressive, incurable, chronic lung disease caused by tobacco smoke exposure.1 2 Approximately 25%–50% of individuals with COPD live with some degree of frailty, defined as a multidimensional state of increased vulnerability to health stressors that is caused by accumulation of health deficits across multiple domains.3–6 Frailty among individuals with COPD is associated with frequent hospitalisation, longer hospital stays, increased costs, increased mortality and poor quality of life.4 5 7–10
Multiple validated instruments can reliably identify an individual’s degree of frailty.3 The Clinical Frailty Scale (CFS) is a practical, validated bedside tool for clinicians, which relies on the clinical history from patients, and is scored from 1 (very fit) to 9 (terminally ill).6 Compared with the 80-item Frailty Index, the CFS explains 80% of the variation in frailty index scores while taking less than a minute to administer.6 11 Increasing degrees of frailty on the CFS correlate with increasing risk of death and future need for long-term care among adults over 65 years of age.6 Progressive degrees of frailty are defined by increasing limitations in performing activities of daily living (ADLs) and Instrumental ADLs (online supplemental appendix B).6 Given the prevalence and prognostic value of frailty among patients with chronic lung disease, it is important for healthcare providers to accurately recognise and assess the degree of frailty, to facilitate appropriate treatments, medical decisions and care plans to meet individual patient needs.12 13
Supplemental material
Previous studies have evaluated the accuracy of frailty assessments among junior medical residents as well as multidisciplinary clinical staff who are experienced in the practice of geriatric medicine.14 However, there is no evidence regarding the reliability of frailty assessments when performed by respiratory clinicians with little prior experience performing frailty assessment, especially using instruments that rely on subjective clinical data. Experts advocate that frailty assessments be incorporated into clinical pathways and programmes to provide risk stratification, yet there is currently no standardised educational process among respiratory healthcare staff to facilitate this change in practice.12 15
The aims of this study were to identify perceived barriers and facilitators to assessing frailty for respiratory patients in the acute care setting, and to design and implement a frailty-focused education intervention centred on common rater biases to educate acute care respiratory registered nurses (RNs). We sought to determine the degree of agreement between RNs and a gold-standard frailty measurement before and after education. If implementation of this educational intervention is successful, we hypothesise that frailty-specific rater training will improve the accuracy of frailty assessments between RNs and a gold-standard frailty assessment performed by geriatricians in the future.
Some of the preliminary results of this study have been previously reported in the form of an abstract.16
Methods
Study design
We conducted a pilot cross-sectional mixed-methods study on an inpatient respiratory hospital ward over 6 months in 2017. Figure 1 describes the study process.
Participant recruitment and study procedures. *Intervention delivered to small groups (six to eight participants) in 1-hour sessions. RN, registered nurse.
The study consisted of: (1) a focus group among experienced RNs to identify barriers and facilitators that influence the current practices of RNs when rating frailty in hospital; (2) development and implementation of a new education intervention designed to address rater biases; and (3) measurements of agreement between RN and gold-standard frailty ratings before and after the educational intervention.
Setting
The study was conducted at The Ottawa Hospital (TOH) on the acute inpatient respiratory ward. TOH is a quaternary care academic hospital with approximately 1200 inpatient beds, serving a local population of >1 million individuals.17
Participants
We recruited RNs working on the acute inpatient respiratory ward at TOH who had experience using a new hospital-based COPD care model that incorporated the CFS instrument to assess frailty within a clinical pathway. RN participants were selected by purposive sampling based on guidance from the nursing clinical leads.
Focus group: barriers and facilitators to rating clinical frailty
We first conducted a focus group with six of the most experienced (>5 years in practice) RNs on the inpatient respiratory ward to identify nursing perceptions of the CFS instrument, perceived barriers and facilitators to using the CFS in a busy clinical setting, and to determine whether a rater training course would be perceived as useful (online supplemental appendix A). Verbal consent was obtained prior to participation in the focus group. A recording of the focus group discussion was independently reviewed by two study authors (AL, SM), to identify the emerging themes. The final themes were determined by consensus.
Educational intervention design
We developed a novel, structured, rater training, frailty-focused education intervention to review and overcome known rater biases for frontline healthcare workers.18–20 The course content was adapted from previous rater training concepts and included four key components: (1) introduction to the frailty concept, (2) frame of reference training, (3) rater error training and (4) performance dimension training (table 1).18–24 Frame of reference training was presented using a clinical example of an older patient with comorbidities who was hospitalised with acute exacerbation of COPD. Following independent frailty ratings, participants openly discussed specific patient factors that determined their rating. Rater error training was presented with an explanation of four common rating biases (leniency effect, halo effect, central tendency effect and primacy effect). Four clinical vignettes were presented along with a corresponding frailty rating and reasoning given by a fictitious rater to highlight each rating bias. In a facilitated group discussion, participants were asked to identify the rater bias present, and tips to avoid common rating biases were discussed. Performance dimension training involved a review of the CFS instrument degrees of frailty (online supplemental appendix B). Participants were separated into small groups (two to three members) and asked to rate the degree of frailty in two clinical vignettes of older patients admitted for acute exacerbation of COPD. After discussion, one spokesperson from each group presented a summary of the case as well as their suggested frailty rating. Questions and disagreement were encouraged and facilitated, and each rating was challenged by the facilitator to probe the participants’ underlying reasoning for the rating assigned. The goal was to reach a group consensus for each of the four vignettes discussed.
Components of the rater training education intervention to assist with recognition and assessment of frailty
Education intervention delivery
The education intervention was delivered to groups of six to eight RN participants over four separate 1-hour small group sessions, facilitated by the study team members with training and experience in medical education and rater training (AL, SM).
Assessing clinical frailty ratings
Based on the authors’ clinical experience with patients admitted to the respiratory medicine ward, the CFS category ‘very fit’ (score=1) was not included in the study, as hospitalised patients with COPD at our centre generally did not meet criteria for ‘very fit’.25 For the purposes of this study, frailty categories studied ranged from ‘well’ (rating of 1) and final category was ‘terminally ill’ (rating of 8) (online supplemental appendix B).
Pre-intervention
One week prior to the rater training course, RN participants completed a structured questionnaire of baseline demographic questions, number of months of experience on the acute respiratory ward and the number of times the participant had used the CFS in clinical practice. Questions regarding barriers and facilitators to using the CFS were based on the responses provided by the initial focus group of experienced RNs (n=6), in addition to factors the study team deemed relevant. Participants were asked to rank eight potential barriers to using the CFS: from 1 (most important barrier) to 8 (least important barrier) (online supplemental appendix C). Participants were asked to select all items from a standardised list that they felt would facilitate the use of the CFS.
The second portion of the structured questionnaire presented 10 fictional clinical vignettes describing hospitalised patients with acute exacerbation of COPD (online supplemental appendix D). The clinical vignettes were developed, edited and reviewed by study authors (physicians and nurses) with experience in assessing and treating hospitalised patients with COPD (AL, ND, CB, SM). The information presented in each vignette simulated the degree of information received upon a standard admission history and chart review by an RN. All cases were reviewed by two of the study authors (AL, SM) to standardise the presentation of the cases. There was one case for each CFS category, with two cases each for the ‘vulnerable’ (rating of 3) and ‘moderately frail’ (rating of 5) categories. RN participants were asked to provide a frailty rating using the CFS for the patient described in each clinical vignette. Ratings were based on the description of the baseline functional status of the patient (at 2 weeks prior to hospitalisation). Participants were required to complete the questionnaire and frailty rating exercise to participate in the rater training educational course.
Post-intervention
One week following the rater training course, all participants were asked to complete a second questionnaire. RNs were asked to assign frailty ratings to the same 10 clinical vignettes, presented in random order to reduce the potential for recall bias.
Gold-standard frailty assessments
The clinical vignettes used in the questionnaires were distributed to a geriatrician (SH) and advance practice geriatrics nurse (TP) who each assigned a frailty assessment to each vignette. This geriatrics team had extensive experience in using the CFS instrument in routine clinical settings. Any disagreements in the ratings between the geriatrics team and authors of the vignettes were resolved by consensus.
Outcomes
The primary outcomes were the identification of barriers and facilitators to assessing frailty in the acute care setting, and the degree of agreement between RNs and the gold-standard frailty rating prior to the education intervention. Secondary outcomes included the per cent agreement between RNs and the gold-standard frailty after the educational intervention.
Statistical analysis
Demographic data of the RN participants were described using means (±SD) and proportions. Barriers to using the CFS were ordered from most important (lowest score) to least important (highest score), by calculating the sum of each participant’s ranking, where the lowest sum indicated the most important perceived barrier. Facilitators were described by the proportion of RNs who indicated their importance.
The agreement between RNs and the gold-standard frailty rating was assessed for each case using the percentage of agreement (number of RNs who agreed with the gold standard/total number of RNs), before and after the education intervention. McNemar’s test was used to compare the degrees of agreement before and after the education intervention.
Results
Initial focus group with experienced RNs to explore frailty assessment in acute care
Two main themes emerged from the focus group of six experienced acute care RNs; (1) the RNs found that the CFS instrument was straightforward to use, and (2) the barriers to using the frailty scale are external to the frailty scale itself, with time to assess and perform the rating being the most significant perceived barrier. Other barriers identified were included in the standardised questionnaire for RN study participants (online supplemental appendix C). The CFS was recognised as being important for patient care; however, RNs found their opportunity to perform frailty assessments was infrequent.
RN participant demographics
Among 26 participating RNs, a range of nursing experience was observed with a median (IQR) of 15 (15.5–105) months. The median (IQR) duration of prior use of the CFS instrument was minimal at 1.5 (1–4) months. A total of five RNs were lost to follow-up at the end of the study.
Barriers and facilitators to using the CFS in a clinical setting
Table 2 describes the perceived barriers and facilitators to using the CFS instrument at the time of hospitalisation. The most important barriers to using the CFS on an acute care respiratory ward were having a lack of clinical directive, lack of collateral patient history to assign the frailty score and high inpatient workload at the time that frailty rating was required. Understanding the CFS and timely completion of the CFS were not highly ranked barriers to use. The most frequently identified facilitators to using the CFS included the presence of family members at the bedside to provide additional history at the time of patient admission (n=21, 78%), increasing the flexibility in the time window to complete the CFS (ie, to complete within first 24 hours of admission) (n=19, 70%) and more education on how to use the CFS (n=14, 52%).
Barriers and facilitators to using the CFS in hospital (most to least important) among 26 acute care respiratory RNs
RNs’ agreement with gold-standard frailty assessment
The median (IQR) overall per cent agreement across all cases was 55.8% (47.2%–60.6%) prior to the education intervention, and 57.2% (44.1%–70.2%) afterwards.
Following the education intervention, there was an increased degree of agreement with gold-standard frailty ratings for all frailty categories (table 3), although none were statistically significant by the McNemar’s test p value cut-off of <0.05.
Per cent agreement between RN and gold-standard frailty ratings before and after the frailty education intervention
The largest increase in agreement occurred within the ‘mildly frail’ category, 65.4%–81% agreement after the educational intervention.
Discussion
In this pilot study, the most important barriers to assessing patient frailty in the acute care respiratory setting were external to the CFS instrument itself and included a perceived lack of time due to large workloads, lack of collateral clinical history from caregivers and a lack of clinical directives to routinely perform frailty assessments on hospitalised patients. Further, half (51%) of our sample of RNs felt that frailty-focused education would facilitate the application of the CFS instrument in the hospital setting. At baseline, there was low agreement (56%) between RNs and a gold-standard frailty rating and following a structured rater training education intervention, there was a non-significant increase in agreement in all categories of frailty.
Advocates for person-centred care emphasise the benefits of using frailty assessments as a risk stratification tool to treat, prognosticate and support patients with chronic disease, calling frailty ‘a new vital sign’.12 13 15 Our study demonstrates that real barriers do exist to measuring frailty in the acute care setting and these must be addressed at a health system level before frailty assessments can be integrated into standard care processes for patients with chronic lung disease. Our findings among respiratory RNs are echoed in qualitative interviews with emergency, anaesthesia and surgery providers in Singapore, which identified the need for frailty-focused educational programmes for both patients and providers, an integrated approach to frailty screening, and hospital-wide adoption of a common frailty screening tool.26 Data from European health policy providers identified that widespread implementation of frailty screening and management requires a ‘culture shift’ and redeployment of health resources to facilitate an integrated and multidisciplinary care approach.27 Empowering other members of the healthcare team (allied health professionals) and patients and caregivers to participate in the frailty assessment in early stages of the hospitalisation may address important barriers to performing frailty assessments in acute care settings. Further, targeted education on potential interventions that can be applied for progressive degrees of frailty may increase the drive to perform frailty assessments in acute care.
While systemic change in health systems is needed to incorporate patient frailty assessment, prevention and supportive intervention, accurate identification of frailty is also essential. In our small pilot study, only half of participating RNs agreed with the gold-standard frailty assessment, and modest non-significant improvement occurred after further education. This is in contrast to data from the critical care literature which suggest that agreements in frailty assessment using the CFS instrument were ‘good’ with a kappa score of 0.64 (0.4–0.87, p<0.0001) between medical students and critical care attendings.28 A larger critical care study also found adequate reliability of frailty measurements between research coordinators, occupational therapists and geriatric medicine trainees.29 Robust data supporting the reliability of frailty assessments among lung health practitioners are lacking, and our pilot study identified a clear desire among RNs for frailty-focused training. The increase in cognitive load of a busy medical inpatient unit may increase the risk of rating error and bias.30 31 In the context of frailty assessment in hospitalised patients, there is a risk of both halo and primacy effects, where the rater may be biased by the patient’s clinical or functional state in the moment of their acute illness. Targeting such common and context-specific rating errors may decrease rater bias and improve accuracy of assessments. Given the small sample sizes and observational nature of this study, the observed increases in agreement of frailty scores between participants and the gold standard cannot be solely attributed to the educational intervention and must be further studied with larger samples and with real-world patients.
To our knowledge, this pilot study is the first to describe barriers to frailty assessment in the acute care respiratory setting and to design and implement a structured rater training educational intervention targeted at improving frailty assessments among frontline nursing staff in an acute care hospital. Strengths of this study include the components of the educational intervention, which were developed and rooted in standardised rater training literature and the qualitative feedback elicited from frontline healthcare staff in an acute care clinical setting. Limitations of our study include the small sample size and the use of 10 standardised clinical vignettes as opposed to an authentic clinical setting with real-world patients. Other limitations include the potential for recall bias among RNs, given the same clinical vignettes were used before and after the intervention (in scrambled order). However, we attempted to minimise the effect of recall bias by allowing a 1-week interval delay before RNs rescored the vignettes in sporadic order.
Conclusions and lessons learnt
This study demonstrates that barriers to assessing patient frailty, such as large acute workloads and lack of widespread clinical directives in hospital, are external to the measurement tool itself. Accuracy of frailty assessment among acute care RNs was low. Frailty-focused rater training was successfully implemented but did not statistically improve accuracy in this small pilot study. Our results support (1) the need for high-level health system planning and hospital-wide approaches to address barriers to incorporating patient frailty assessments in a meaningful way, and (2) expanding the rater training educational intervention to an iterative training process with larger sample sizes of real-world patients and scalable training formats to test its effectiveness in the acute care respiratory setting.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by The Ottawa Health Sciences Research Ethics Board (OHSN-REB) ID: 20170676-01. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We gratefully acknowledge Jan Leahy, Sonia Joanise and Ley-Ann Mondor for their nursing leadership, assistance with study coordination and collaboration. We are highly appreciative of the nursing staff on the respiratory ward at The Ottawa Hospital who kindly agreed to participate in this study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors AL, ND, CB, SH, TP and SM conceived the initial study design and collected data. AL, SM and ND conducted the analysis. All authors (AL, ND, CB, SH, TP, MC, DK, DM, JL, AF, SM) contributed to the analysis or interpretation of the data/analysis and critical review and revision of the manuscript for intellectual content. SM accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests The authors declare no financial or personal conflicts of interest related to this work.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.