You are here

  • Home
  • Archive
  • Volume 13, Issue 1
  • Enhancing pain care with the American Pain Society Patient Outcome Questionnaire for use in the emergency department (APS-POQ-RED): validating a patient-reported outcome measure

Article Text

Article menu
Original research
Enhancing pain care with the American Pain Society Patient Outcome Questionnaire for use in the emergency department (APS-POQ-RED): validating a patient-reported outcome measure
  1. James A Hughes1,
  2. Sarah Hazelwood2,
  3. Anna-Lisa Lyrstedt3,
  4. Lee Jones4,5,
  5. Nathan J Brown3,6,
  6. Rajeev Jarugula2,
  7. Clint Douglas7,8,
  8. Kevin Chu3,6
  1. 1School of Nursing, Queensland University of Technology, Kelvin Grove, Queensland, Australia
  2. 2Emergency Department, The Prince Charles Hospital, Chermside, Queensland, Australia
  3. 3Emergency and Trauma Centre, Royal Brisbane and Women's Hospital, Herston, Queensland, Australia
  4. 4Queensland University of Technology, Brisbane, Queensland, Australia
  5. 5QIMR Berghofer Medical Research Institute, Herston, Queensland, Australia
  6. 6Faculty of Medicine, University of Queensland Faculty of Medicine and Biomedical Sciences, Herston, Queensland, Australia
  7. 7Faculty of Health, Queensland University of Technology, Brisbane, Queensland, Australia
  8. 8Metro North Hospital and Health Service, Herston, Queensland, Australia
  1. Correspondence to James A Hughes; j1.hughes{at}qut.edu.au

Abstract

Background In general, the quality of pain care in emergency departments (ED) is poor, despite up to 80% of all ED patients presenting with pain. This may be due to the lack of well-validated patient-reported outcome measures (PROMs) of pain care in the ED setting. The American Pain Society-Patient Outcome Questionnaire-Revised Edition (APS-POQ-R), with slight modification for ED patients, is a potentially useful PROM for the adult ED, however it is yet to be completely validated.

Methods Adult patients, who had presented with moderate to severe acute pain, were recruited at two large inner-city EDs in Australia. A modified version of the APS-POQ-R was administered at the completion of their ED care. Responses were randomly split into three groups and underwent multiple rounds of exploratory and confirmatory factor analysis with testing for construct, convergent, divergent validity and internal consistency.

Results A total of 646 ED patients (55.6% female), with a median age of 48.3 years, and moderate to severe pain on arrival, completed the ED-modified APS-POQ-R. Psychometric evaluation resulted in a reduced nine-question tool, which measures three constructs (pain relief and satisfaction (α=0.891), affective distress (α=0.823) and pain interference (α=0.908)) and demonstrated construct, convergent, divergent validity, and internal consistency.

Conclusions This new tool, which we refer to as the American Pain Society-Patient Outcome Questionnaire-Revised for the ED (APS-POQ-RED), should form the basis for reporting patient-reported outcomes of ED pain care in future quality improvement and research.

  • Emergency department
  • Pain
  • Quality measurement

Data availability statement

Data are available upon reasonable request. Data available on reasonable request and approapiate approvals have been received.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2023-002295

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THE TOPIC

    • Pain is the most common symptom on presentation to the emergency department and is widely reported as poorly treated.

    • A range of patient-reported outcomes are used as outcome measures in quality improvement and research in the emergency department. However, many of these have poor, or no, validity in the setting.

    WHAT THIS STUDY ADDS

    • This study presents a psycometrically validated instrument for measuring the patient-reported outcomes of acute pain care in the adult emergency department.

    HOW THIS RESEARCH MIGHT AFFECT RESEARCH, PRACTICE AND/OR POLICY

    • This shortened, emergency department-specific patient-reported outcome measure (APS-POQ-RED) seeks to provide a standardised, validated measure of patient-reported outcomes of acute pain care in the ED for quality and research purposes.

    Introduction

    Pain is the most common symptom in people who present to emergency departments (ED).1 2 However, due to a lack of well-validated tools to measure patient-reported outcomes (PROs) of pain care in the ED setting, pain care outcomes from the patient’s perspective are not well understood.3 This makes it difficult to assess and improve the quality of pain care in the ED. Unidimensional measures of pain intensity are only partially valuable tools because they are limited in scope and fail to account for the multidimensional experience of pain.4–6 Time-based metrics, such as ‘time to be seen’ and ‘time to first analgesic medication’, assume that ‘faster is better’ and provide only a superficial view of the health services’ response to pain rather than the outcomes experienced by the patient.7–9 Furthermore, there are only weak associations between the PROs (such as patient experience) and time-based metrics,9 meaning the patient may receive poor outcomes despite timely care. Previously it has been reported that ED patients have poor recall of the pain care they receive however have a better recall of the outcomes of this care.10 The symptom of pain and how a patient responds to pain treatment are specific to the individual and best reported by the individual experiencing the symptom.3 Thus, measurement tools must reflect pain’s subjectivity and individual responses to care11 from the only person who can accurately gauge the outcome, the patient experiencing the symptom.

    In its simple form, a PRO is ‘a measurement of any aspect of a patient’s health status that comes directly from the patient’.12 When considering a symptom such as pain, PROs offer a measurement of the patient’s perspective of the outcomes of care.13 The incorporation of the patient’s perspective of care can have a significant impact on the delivery of effective care and influence change in the manner in which care is delivered.14 While PROs of pain care have been reported in quality and research activities in the ED, the description of their development and validation within this environment is limited. The majority of patient-reported outcomes measures (PROMs) used in the ED are taken directly from other settings without modification or validation,3 which may limit the patient’s understanding of the instrument and the accuracy of any derived measurements.15

    The most promising and easily modifiable PROM for use in ED is the revised edition of the American Pain Society’s Patient Outcome Questionaire (APS-POQ-R).3 16 It encompasses five broad but essential domains of pain: intensity, patient satisfaction, side effects of medications, emotional and physical functioning. Our initial exploratory factor analysis of a modified APS-POQ-R for use in the ED demonstrated potential construct and discriminate validity based on patient urgency in a single centre.16 However, the lack of testing of the proposed structure in an additional sample leaves the construct validity and internal consistency incomplete.15 To overcome these limitations of the previous work16 and provide significant evidence of the validity and reliability15 of a modified APS-POQ-R, we conducted a multisite, multistage study of the APS-POQ-R, modified for use in the adult ED using elements of classical test theory.

    Methods

    Aims and objectives

    This study aims to take the previously described adaptation of the APS-POQ-R, which has demonstrated incomplete validity and assesses the construct validity and internal consistency in adults presenting to the ED with moderate to severe pain. The specific objectives are to:

    1. Test the construct validity of a previously described structure16 of the APS-POQ-R modified for use in adults in the ED.

    2. Propose a new construct structure for the modified version of the APS-POQ-R

    3. Test the new construct structure proposed in objective two.

    Setting

    The study was conducted at the EDs of two principal referral hospitals in Brisbane, Australia, with data prospectively collected between September 2021 and January 2022. Both of these ED’s are located in inner-city hospitals with a universal healthcare system. The Royal Brisbane and Women’s Hospital is the principal referral centre for the state, and sees all adult presentations and specialities with a census of approximately 85 000 per year. The Prince Charles Hospital is located in a large tertiary hospital and sees all adult and paediatric presentations with a census of approximately 100 000 per year. 17

    Instrument

    The original APS-POQ-R18 is an interviewer-administered PROM that was developed for postoperative and cancer-related pain and has been used or adapted to other forms of pain,19 translated into several languages20–22 and demonstrates cross-cultural validity.23 In studies in the adult ED, the APS-POQ-R has been used as a PROM, either in part4 5 24 25 or in translated (Danish) entirety.26 However, except for the Danish translation, the validity of the APS-POQ-R in the ED setting has not been fully established. The APS-POQ-R contains two sections, 18 questions that originally mapped to five constructs and a second section that describes the use of non-pharmacological analgesia. Consistent with previously reported validity, this work focuses on the first section.18

    In a previous study,16 we conducted an exploratory factor analysis (EFA) of a version of the APS-POQ-R slightly modified to better suit patients with moderate to severe acute pain in the ED setting, retaining 18 questions from the original APS-POQ-R.18 Those modifications consisted of changes to the wording of questions related to the reporting timeframe, that is, the phrase ‘…(in) the first 24 h in the hospital or after your operation’ was changed to ‘…in the emergency department’. There was also an unintentional modification due to a transcription error, in which the question about how much the pain caused the patient to feel anxious was omitted. In the current study, we repeated this Exploratory Factor Analysis (EFA) by including the question about anxiety. Furthermore, an additional modification was made in this work. The two questions relating to the effects of pain on sleep (in the ED) were excluded from the analysis (for objectives two and three) because of their lack of relevance to acute pain in the ED setting.

    Participants

    Patients were elidgible for inclusion if they presented to ED with acute pain for less than 6 weeks and had an initial documented or self-reported pain score of 4/10 or greater. In addition, patients were ineligible if there were triaged into the most urgent category (Australasian Triage Scale, category one), intoxicated with alcohol or other drugs, aged below 18 years, unable to give consent, did not speak English or had cognitive impairment.

    Data collection

    For each participant, data were collected by a research nurse directly into an electronic data capture form (RedCap, Vanderbilt University) using a tablet computer following study enrolment. In addition, information that the patient did not provide was collected from the ED Information System. Data included the participant’s age, sex, mode of arrival to the ED, Charleston comorbidity score and time from ED arrival to the first analgesic medication.

    Sample and sample size

    We used a convenience sample of patients who met the inclusion criteria. During the shifts of the research nurses (0700–1600, Monday to Friday), consecutive ED patients were approached by the research nurse and invited to participate. Patients were only enrolled after their ED care had been completed, either before discharge or while awaiting admission to the hospital. No patients were enrolled once they had left the ED.

    The minimum sample size was calculated based on 18 questions over three stages of data analysis. While there is some conjecture around sample size in factor analysis, a sample size of 10 responses per question or an overall sample size of 200 (for less than 40 questions) is deemed adequate.15 27 Therefore, we aimed to collect 200 participants per objective or 600 participants across the two sites. Funding for research assistants was available for 18 weeks at each site, and while we aimed for a minimum of 200 patients at each site, data collection would continue until the end of the 18 weeks.

    Statistical analysis

    The methods and statistical analysis used in this work are based on the previous revision of the APS-POQ-R,18 the factor analysis described in the first revision of the APS-POQ-R for the ED16 and the process and measures described by Frost et al.15 Data were analysed using SPSS (IBM Corp. Released 2021. IBM SPSS Statistics for Windows, Version 28.0. Armonk, NY: IBM Corp). Descriptive statistics were used to summarise the population characteristics. Continuous data were summarised using means and SD. Frequencies and percentages summarised categorical data. The population was split into three even groups using random number allocation (random.org), and each group was used to test one of the objectives. Analysis of variance and the χ2 tests were used to determine between-group differences in characteristics (table 1). Finally, the responses to the 18 questions in the APS-POQ-RED were summarised as means and SD for all patients and each group (table 2).

    View this table:
    Table 1

    Description of the population and differences between groups

    View this table:
    Table 2

    Summary of responses from the first nine questions of the modified APS-POQ-R

    The previously described fit of the APS-POQ-R modified for use in the ED was tested with confirmatory factor analysis (CFA) in IBM Amos V.28. The fit of this structure was assessed using the Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI) and Tucker-Lewis Fit Index (TLI).28 The model was considered a good fit if the CFI and TLI were greater than 0.929 and the RMSEA was less than 0.08.30

    EFA (principal axis factoring) was then conducted using IBM SPSS V.28, with a Promax rotation and Kaiser Normalisation, to identify a new structure within the current data. Models with eigenvalues greater than 1 were investigated for the best statistical and clinical fit. Questions were considered loading to a factor if the coefficient was at least 0.4. In addition, the discriminant validity (correlation between factors) and internal consistency (Cronbach alpha) for each factor were reported within the final proposed structure.

    The structure of the APS-POQ-R proposed in the EFA was then tested using CFA in IBM Amos V.28. The fit of this structure was assessed using the RMSEA, CFI and TLI28 with the previously defined thresholds. Construct validity of the final APS-POQ-R tool was assessed using Composite Reliability (CR) for internal validity (good CR≥0.7), convergent and divergent validity was assessed using Average Variance Extracted (AVE) and Maximum Shared Variance (MSV). Convergent and discriminant validity indicate how individual items correlate with their latent factors. Convergent validity measures the level of variance explained by a construct versus the level due to measurement error and can be judged by AVE≥0.5, whereas discriminant validity compares the amount of the variance explained by each construct to the shared variance of other constructs to ensure that each construct measures different aspects. Therefore, MSV should be less than AVE and the square root of AVE greater than inter-construct correlations.31–33

    Ethical considerations

    This study received approval from the Human Research Ethics Committee of the Royal Brisbane and Women’s Hospital. All patients gave written informed consent after explaining the study by one of the research assistants and were free to withdraw from the study until publication.

    Patient and public involvement

    Patients were not involved in the planning or conduct of this this study. However, patients recruited to the study were given the option of receiving a copy of the outcomes at the time of consent.

    Results

    A total of 653 patients were recruited for the study across the two sites. One patient withdrew consent after data collection, one was under 18 years at the time of consent and five who never experienced more than 3/10 pain in the ED were excluded from data analysis. This left a total sample of 646 patients, randomly split into three groups of 215, 215 and 216. Table 1 shows no statistically significant between-group differences in demographic or clinical characteristics. Table 2 summarises the answers to each question of the APS-POQ-R for the three groups and the whole sample.

    Confirmatory factor analysis

    The previously described structure was tested using CFA on group one as described in the methods. The structure of the model and individual factor loadings can be found in the online supplemental material (online supplemental figure 1). This model did not prove to be a good fit when tested. The CFI reported for this model ranged from 0.605 to 0.715, well outside the 0.900 threshold considered a good fit. The RMSEA for this model was 0.153 (90% CI 0.143 to 0.164, p<0.001), also well above the 0.08 threshold for a good model fit. Standardised factor loadings ranged from 0.22 to 1.01, with three of the five subscales having factors that loaded less than the 0.7 minimum. Therefore, this data had a poor fit for the previously proposed structure. Analysis progressed to identify a new structure using EFA.

    Supplemental material

    Exploratory factor analysis

    An EFA conducted on group two data resulted in a Kaiser-Meyer-Olkin measure of sampling adequacy of 0.792, and a statistically significant Bartlett’s test of sphericity (χ2 (120) = 1539.0, p<0.001). A solution with up to five factors (subscales) had an Eigenvalue of greater than 1. Considering the structure, questions and analysis, a three-factor solution, consisting of the pain relief and satisfaction subscale, the affective distress subscale and the pain interference subscale, explained 53.6% of the variation in the data, was selected as the solution to undergo further CFA. The four and five-factor solutions split questions into constructs that did not demonstrate clinical applicability or a clear concept. Table 3 shows the internal consistency of each of the three factors, and box 1 outlines the eliminated questions. Online supplemental table 1 shows the pattern matrix for the three-factor solution.

    View this table:
    Table 3

    Subscale item to total correlations and Cronbach alpha for a three-factor solution

    Box 1

    Questions removed from the instrument

    2. On this scale, please indicate your worst pain in the emergency department?

    5.Pain can affect our mood and emotions. On this scale, please select the one number that best shows how much the pain caused you to feel:

    A. Depressed.

    6. Have you had any of the following side effects? Please select ‘0’ if no; if yes, please circle the number that best shows the severity of each.

    1. Nausea.

    2. Drowsiness.

    3. Itching.

    4. Dizziness.

    8. Were you allowed to participate in decisions about your pain treatment as much as you wanted to?

    Confirmatory factor analysis

    A CFA was conducted on group three data. The overall structure and coefficients of the analysis are shown in online supplemental figure 2. All of the questions loaded to the factors at >0.7. The fit indices indicated that the model was a good fit: TLI=0.959, and CFI=0.973, both meeting the minimum threshold of 0.900. The RMSEA was 0.079 (90% CI 0.052 to 0.106, p=0.039) indicating a reasonable fit.

    Online supplemental table 2 demonstrates the validity and reliability of the model presented. In online supplemental table 2A, the CR of the three constructs presented is greater than 0.7 and therefore demonstrates the reliability of these constructs.31 The average variance explained for each of the three constructs is greater than 0.5 therefore, the three constructs demonstrate convergent validity.33 Discriminant validity is demonstrated in online supplemental table 2A, where the MSV is less than the average variance explained and in table six, where the square root of the AVE is greater than the interconstruct correlations.32

    Discussion

    In this three-stage prospective validation of the APS-POQ-R for acute pain presenting to the adult ED, we have identified nine questions that map to three latent constructs (subscales). In patients presenting to the ED with moderate to severe acute pain. These nine questions measure the PROs of pain relief and satisfaction, affective distress and pain interference. This demonstrates the utility of the modified APS-POQ-R in measuring PROs of pain care in the adult ED. Therefore, our revised edition of the APS-POQ-R for acute pain in the ED will now be known as the APS-POQ-RED.

    The structure described in the previous adaptation of the APS-POQ-R16 could not be replicated within this study, despite one of the study locations being the same and replicating the inclusion and exclusion criteria. This highlights the limitations of reporting only EFA as a measure of construct validity without the concurrent use of CFA in a separate sample.15 The generalisability of an EFA model to a CFA model is significantly influenced by several factors, such as the assumption of error within the data set and the distributional assumption of the data collected.34 The limitations of this earlier work have been overcome in the study presented by ensuring that both EFA and CFA are completed on separate samples collected simultaneously within the same population.

    The pain severity and satisfaction subscale is the first of the three latent constructs measured within the revised tool, comprising the least pain severity the patient reported in the ED, how often they were in severe pain, the amount of relief and their overall satisfaction. This construct correlates well with the most commonly measured PROs in the ED literature (pain intensity and patient satisfaction)16 as well as the two of the constructs measured in the initial development of the APS-POQ-R (pain severity and sleep interference, perceptions of care).18 The relief of pain, including the endpoint of pain relief, has been previously shown to be highly correlated with patient satisfaction. Taylor et al have shown that targeting adequate analgesia (decrease of at least two points until the pain is rated as less than four on an 11-point scale) is associated with high levels of patient satisfaction.4 25 Time to first analgesic medication has been shown to influence patient satisfaction as a surrogate measure of the time in severe pain.9 This construct is related to treatment. In future studies of patient-reported pain care outcomes using the APS-POQ-RED in the ED, this construct should be correlated with the treatment given, including time to the first analgesic medication.

    The affective distress subscale comprising the degree to which the patient felt frightened, anxious or helpless because of their pain was the second subscale identified in the revised tool. Pain rarely exists without co-occurring symptoms such as anxiety.35 36 Alterations in the affective functioning of the patient can be a result of the pain, the lack of knowledge of the cause or the treatment and receiving care in an environment that may be unfamiliar and imposing. The relationship between pain and affective distress is well acknowledged within the paediatric ED pain literature, with numerous interventions designed to reduce both.37–42 However, only 19% of PROs reported in the adult literature include emotional functioning as an outcome of pain care.3 A recent review of pain and anxiety measures in the adult ED found that no measure of the co-occurring symptoms of anxiety and pain had been reported in practice outside of quality improvement or research spheres.36 The acknowledgement of affective (emotional) distress as an outcome of effective pain care in the adult ED is the first step in acknowledging co-occurring symptoms (or symptom clusters) within the adult ED.

    The third subscale identified within the tool is the pain interference subscale. This subscale identified the impact of pain on the patient’s physical functioning through its two questions. Pain is known to impact the patient’s ability to function; however, this relationship is not linear as seemingly minor pain can impair a patient’s function.6 Reduction in function can often be the precursor to patients seeking care in an ED. A lack of functional improvement secondary to disease progression is a leading cause of representation.43 44

    The instrument presented in this work has several uses within the ED. The first use is to measure the outcomes of interventional and quality improvement studies in a consistent manner that is comparable across studies. With an increasing focus on the pharmacological treatment of pain in the ED, the use of objective PROMS in interventional studies, especially opiate alternatives and sparing projects, allows the patient voice to complement other outcomes of the intervention. This instrument goes beyond simple intensity measurement, acknowledging that pain is a multifaceted experience. This instrument gives voice to factors such as side effects and the amount of relief felt by the patient in a simple way that is capturable in the unique ED setting. The use of this instrument as an outcome measure in interventional studies in the ED ensures that the impact of the intervention is captured from the patient perspective. The second way this instrument may be useful is in identifying the relationship between process/health service outcomes of pain care in the ED and those reported by the patient. Previous work in this area has demonstrated a poor relationship between process measures and patient-centred outcomes, however, the outcome measures used were limited and not validated for acute pain in the ED.9 The reduction in the number of questions within the instrument has distinct importance in the ED where patients may be time limited and patient flow is important. However, there is also an inherent limitation in the reduction of the questions, meaning some outcomes are not longer measured. Correlations with the measures described above will add further strength to the reduction in questions. However, this instruments usage in the ED may still be limited to interventional studies and Quality Improvement activities, as asking nine questions in routine care may be onerous on overstretched clinicians. Aspects of this instrument, such as the pain relief and satisfaction subscale, may be more useful in everyday care, as four questions would be more achievable in routine care.

    Conclusion

    In this work, we have demonstrated the structural validity and internal consistency of a modified version of the APS-POQ-R for use in moderate to severe acute pain within the adult ED. The modified version, which we call the APS-POQ-RED, uses nine questions to measure three subscales (pain relief and satisfaction, affective distress and pain interference subscales). The modification of the instrument to focus on care received in the ED, as well as the removal of irrelevant questions, have made this a specific, validated tool for use in quality improvement and research activities within the ED.

    Data availability statement

    Data are available upon reasonable request. Data available on reasonable request and approapiate approvals have been received.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study involves human participants. This study received approval from the Human Research Ethics Committee of the Royal Brisbane and Women’s Hospital (LNR/2019/QRBW/55143). Participants gave informed consent to participate in the study before taking part.

    Acknowledgments

    The authors would like to thank the staff of both the Emergency and Trauma Centre at the Royal Brisbane and Women’s Hospital and the Emergency Department of The Prince Charles Hospital for their support in conducting this study, especially during the middle of a Pandemic.

    References

    1. ↵
      1. Hughes JA,
      2. Douglas C,
      3. Jones L, et al
      . Identifying patients presenting in pain to the adult emergency department: a binary classification task and description of prevalence. Int Emerg Nurs 2023;68:101272. doi:10.1016/j.ienj.2023.101272
      OpenUrl
    2. ↵
      1. Cordell WH,
      2. Keene KK,
      3. Giles BK, et al
      . The high prevalence of pain in emergency medical care. Am J Emerg Med 2002;20:165–9. doi:10.1053/ajem.2002.32643
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Wong A,
      2. Potter J,
      3. Brown NJ, et al
      . Patient-reported outcomes of pain care research in the adult emergency department: a scoping review. Australas Emerg Care 2021;24:127–34. doi:10.1016/j.auec.2020.10.003
      OpenUrl
    4. ↵
      1. Jao K,
      2. McD Taylor D,
      3. Taylor SE, et al
      . Simple clinical targets associated with a high level of patient satisfaction with their pain management. Emerg Med Australas 2011;23:195–201. doi:10.1111/j.1742-6723.2011.01397.x
      OpenUrlCrossRefPubMed
    5. ↵
      1. Taylor D,
      2. Lee M,
      3. Johnson OG, et al
      . Patient satisfaction with their pain management: the effect of provision of pain management advice. Emerg Med Australas 2016;28:8–9.
      OpenUrl
    6. ↵
      1. Gordon DB
      . Acute pain assessment tools: let us move beyond simple pain ratings. Curr Opin Anaesthesiol 2015;28:565–9. doi:10.1097/ACO.0000000000000225
      OpenUrlCrossRefPubMed
    7. ↵
      1. Hatherley C,
      2. Jennings N,
      3. Cross R
      . Time to analgesia and pain score documentation best practice standards for the emergency department–a literature review. Australas Emerg Nurs J 2016;19:26–36. doi:10.1016/j.aenj.2015.11.001
      OpenUrlPubMed
    8. ↵
      1. Hughes JA,
      2. Alexander KE,
      3. Spencer L, et al
      . Factors associated with time to first analgesic medication in the emergency department. J Clin Nurs 2021;30:1973–89. doi:10.1111/jocn.15750
      OpenUrl
    9. ↵
      1. Hughes JA,
      2. Alexander KE,
      3. Spencer L, et al
      . Factors associated with the experience of patients presenting in pain to the emergency department. J Clin Nurs 2022;31:1273–84. doi:10.1111/jocn.15983
      OpenUrl
    10. ↵
      1. Taylor DM,
      2. Valentine S,
      3. Majer J, et al
      . Discordance between patient‐reported and actual emergency department pain management. Emerg Med Australas 2021;33:517–23. doi:10.1111/1742-6723.13690
      OpenUrl
    11. ↵
      1. Flynn SB,
      2. Gordee A,
      3. Kuchibhatla M, et al
      . Moving toward patient‐centered care in the emergency department: patient‐reported expectations, definitions of success, and importance of improvement in pain‐related outcomes. Acad Emerg Med 2021;28:1286–98. doi:10.1111/acem.14328
      OpenUrlCrossRef
    12. ↵
      1. Acquadro C,
      2. Berzon R,
      3. Dubois D, et al
      . Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the patient‐reported outcomes (PRO) harmonization group meeting at the food and drug administration, February 16, 2001. Value Health 2003;6:522–31. doi:10.1046/j.1524-4733.2003.65309.x
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Meadows KA
      . Patient-reported outcome measures: an overview. Br J Community Nurs 2011;16:146–51. doi:10.12968/bjcn.2011.16.3.146
      OpenUrlCrossRefPubMed
    14. ↵
      1. Black N
      . Patient reported outcome measures could help transform healthcare. BMJ 2013;346. doi:10.1136/bmj.f167
    15. ↵
      1. Frost MH,
      2. Reeve BB,
      3. Liepa AM, et al
      . What is sufficient evidence for the reliability and validity of patient‐reported outcome measures Value Health 2007;10 Suppl 2:S94–105. doi:10.1111/j.1524-4733.2007.00272.x
      OpenUrlPubMed
    16. ↵
      1. Hughes JA,
      2. Jones L,
      3. Potter J, et al
      . An initial psychometric evaluation of the APS-POQ-R in acute pain presenting to the emergency department. Australas Emerg Care 2021;24:287–95. doi:10.1016/j.auec.2020.12.001
      OpenUrl
    17. ↵
      1. Queensland Government
      . Emergency departments - performance data pre October 2020. Queensland Health. Brisbane; 2020.
    18. ↵
      1. Gordon DB,
      2. Polomano RC,
      3. Pellino TA, et al
      . Revised American pain society patient outcome questionnaire (APS-POQ-R) for quality improvement of pain management in hospitalized adults: preliminary psychometric evaluation. J Pain 2010;11:1172–86. doi:10.1016/j.jpain.2010.02.012
      OpenUrlCrossRefPubMed
    19. ↵
      1. Chaw SH,
      2. Lo YL,
      3. Lee JY, et al
      . Evaluate construct validity of the revised American pain society patient outcome questionnaire in gynecological postoperative patients using confirmatory factor analysis. BMC Anesthesiol 2021;21:20. doi:10.1186/s12871-020-01229-x
    20. ↵
      1. Fang H,
      2. Liang J,
      3. Hong Z, et al
      . Psychometric evaluation of the Chinese version of the revised American pain society patient outcome questionnaire concerning pain management in Chinese orthopedic patients. PLoS One 2017;12:e0178268. doi:10.1371/journal.pone.0178268
    21. ↵
      1. Schultz H,
      2. Skræp U,
      3. Schultz Larsen T, et al
      . Psychometric evaluation of the Danish version of a modified revised American pain society patient outcome questionnaire (APS-POQ-RD) for patients hospitalized with acute abdominal pain. Scand J Pain 2019;19:117–30. doi:10.1515/sjpain-2018-0106
      OpenUrl
    22. ↵
      1. Erden S,
      2. Karadağ M,
      3. Güler Demir S, et al
      . Cross-cultural adaptation, validity, and reliability of the Turkish version of revised American pain society patient outcome questionnaire for surgical patients. Agri 2018;30:39–50. doi:10.5505/agri.2018.21548
      OpenUrl
    23. ↵
      1. Botti M,
      2. Khaw D,
      3. Jørgensen EB, et al
      . Cross-cultural examination of the structure of the revised American pain society patient outcome questionnaire (APS-POQ-R). J Pain 2015;16:727–40. doi:10.1016/j.jpain.2015.03.016
      OpenUrl
    24. ↵
      1. Sepahvand M,
      2. Gholami M,
      3. Hosseinabadi R, et al
      . The use of a nurse-initiated pain protocol in the emergency department for patients with musculoskeletal injury: a pre-post intervention study. Pain Manag Nurs 2019;20:639–48. doi:10.1016/j.pmn.2019.02.012
      OpenUrlCrossRef
    25. ↵
      1. Taylor DM,
      2. Fatovich DM,
      3. Finucci DP, et al
      . Best‐practice pain management in the emergency department: a cluster‐randomised, controlled, intervention trial. Emerg Med Australas 2015;27:549–57. doi:10.1111/1742-6723.12498
      OpenUrlCrossRef
    26. ↵
      1. Schultz H,
      2. Abrahamsen L,
      3. Rekvad LE, et al
      . Patient-controlled oral analgesia at acute abdominal pain: a before-and-after intervention study of pain management during hospital stay. Appl Nurs Res 2019;46:43–9. doi:10.1016/j.apnr.2019.02.006
      OpenUrl
    27. ↵
      1. Kyriazos TA
      . Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. PSYCH 2018;09:2207–30. doi:10.4236/psych.2018.98126
      OpenUrl
    28. ↵
      1. Xia Y,
      2. Yang Y
      . CFI, and TLI in structural equation modeling with ordered categorical data: the story they tell depends on the estimation methods. Behav Res Methods 2019;51:409–28. doi:10.3758/s13428-018-1055-2
      OpenUrlCrossRefPubMed
    29. ↵
      1. Hu L-T,
      2. Bentler PM
      . Evaluating model fit; 1995.
    30. ↵
      1. Fabrigar LR,
      2. Wegener DT,
      3. MacCallum RC, et al
      . Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 1999;4:272–99. doi:10.1037//1082-989X.4.3.272
      OpenUrlCrossRefWeb of Science
    31. ↵
      1. Hair J,
      2. Black W,
      3. Babin B, et al
      . Confirmatory factor analysis. multivariate data analysis. 7th edn. Upper Saddle River, NJ, USA: Pearson Education, Inc, 2010: 600–38.
    32. ↵
      1. Hair JF,
      2. Black WC,
      3. Babin BJ, et al
      . Multivariate data analysis: Pearson new international edition. Essex: Pearson Education Limited, 2014.
    33. ↵
      1. Malhotra NK,
      2. Dash S
      . An applied orientation. Marketing Research 2010;2. doi:10.1108/S1548-6435(2010)6
    34. ↵
      1. Schmitt TA
      . Current methodological considerations in exploratory and Confirmatory factor analysis. J Psychoeduc Assess 2011;29:304–21. doi:10.1177/0734282911406653
      OpenUrlCrossRef
    35. ↵
      1. Kapoor S,
      2. White J,
      3. Thorn BE, et al
      . Patients presenting to the emergency department with acute pain: the significant role of pain catastrophizing and state anxiety. Pain Med 2016;17:1069–78. doi:10.1093/pm/pnv034
      OpenUrl
    36. ↵
      1. Mccahill R,
      2. Keogh S,
      3. Hughes JA
      . Adult pain and anticipatory anxiety assessment in the emergency department: an integrative literature review. J Clin Nurs 2023;32:4492–501. doi:10.1111/jocn.16520
      OpenUrl
    37. ↵
      1. Arıkan A,
      2. Esenay FI
      . Active and passive distraction interventions in a pediatric emergency department to reduce the pain and anxiety during venous blood sampling: a randomized clinical trial. J Emerg Nurs 2020;46:779–90. doi:10.1016/j.jen.2020.05.004
      OpenUrl
    38. ↵
      1. Chen YJ,
      2. Cheng SF,
      3. Lee PC, et al
      . Distraction using virtual reality for children during intravenous injections in an emergency department: a randomised trial. J Clin Nurs 2020;29:503–10. doi:10.1111/jocn.15088
      OpenUrlCrossRefPubMed
    39. ↵
      1. Koleck TA,
      2. Dreisbach C,
      3. Bourne PE, et al
      . Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review. J Am Med Inform Assoc 2019;26:364–79. doi:10.1093/jamia/ocy173
      OpenUrlCrossRefPubMed
    40. ↵
      1. Koleck TA,
      2. Tatonetti NP,
      3. Bakken S, et al
      . Identifying symptom information in clinical notes using natural language processing. Nurs Res 2021;70:173–83. doi:10.1097/NNR.0000000000000488
      OpenUrl
    41. ↵
      1. Lilik Lestari MP,
      2. Wanda D,
      3. Hayati H
      . The effectiveness of distraction (cartoon-patterned clothes and bubble-blowing) on pain and anxiety in preschool children during venipuncture in the emergency department. Compr Child Adolesc Nurs 2017;40:22–8. doi:10.1080/24694193.2017.1386967
      OpenUrl
    42. ↵
      1. Longobardi C,
      2. Prino LE,
      3. Fabris MA, et al
      . Soap bubbles as a distraction technique in the management of pain, anxiety, and fear in children at the paediatric emergency room: a pilot study. Child Care Health Dev 2019;45:300–5. doi:10.1111/cch.12633
      OpenUrl
    43. ↵
      1. Ferreira GE,
      2. Machado GC,
      3. Abdel Shaheed C, et al
      . Management of low back pain in Australian emergency departments. BMJ Qual Saf 2019;28:826–34. doi:10.1136/bmjqs-2019-009383
      OpenUrlAbstract/FREE Full Text
    44. ↵
      1. Robinson K,
      2. Lam B
      . Early emergency department representations. Emerg Med Australas 2013;25:140–6. doi:10.1111/1742-6723.12048
      OpenUrl

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @perfectednurse

    • Contributors JAH, NJB, KC, CD, LJ and RJ designed this study. JAH and KC obtained funding. SH and A-LL recruited the patients and collected the data. JAH, LJ and NJB analysed the data. All authors had final approval of the manuscript. JAH has overall responsibility for this work and publication.

    • Funding This project was funded through The Emergency Medicine Foundation (Australia) grant number EMLE-166R34-2020.This project was funded through The Emergency Medicine Foundation (Australia) grant number EMLE-166R34-2020.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.