Article Text
Abstract
Aims This paper describes two patient-reported measures of social contact and loneliness, which are closely related concepts. The first measure (R-Outcomes Social Contact measure) was developed from scratch, based on customer needs and literature review. It covers emotional and social aspects using positive terms. The second measure (R-Outcomes Loneliness measure) is adapted from the GSS Loneliness Harmonised Standard. Both measures are patient-reported outcome measures, based on patients’ own perception of how they feel.
Method This development started in 2016 in response to customers’ requests to measure social contact/loneliness for patients in social prescribing projects.
Both measures are compared with three other loneliness measures (the GSS Loneliness Harmonised Standard, De Jong Gierveld and Campaign to End Loneliness). Both measures are short (36 and 21 words, respectively). Mean improvement is reported as a positive number on a 0–100 scale (where high is good).
We tested the psychometric performance and construct validity of the R-Outcomes Social Contact measure using secondary analysis of anonymised data collected before and after social prescribing interventions in one part of Southern England.
Results In the validation study, 728 responses, collected during 2019–2020, were analysed. 90% were over 70 years old and 62% women. Cronbach’s α=0.76, which suggests that it is appropriate to use a single summary score. Mean Social Contact scores before and after social prescribing intervention were 59.9 (before) and 66.7 (after, p<0.001).
Exploratory factor analysis shows that measures for social contact, health status, health confidence, patient experience, personal well-being, medication adherence and social determinants of health are correlated but distinct factors. Construct validation shows that the results are consistent with nine hypotheses, based on the loneliness literature.
Conclusion The R-Outcomes Social Contact measure has good psychometric and construct validation results in a population referred to social prescribing. It is complementary to other R-Outcomes measures.
- patient reported outcome measures
- primary care
- quality improvement
- independent living
- patient-centred care
Data availability statement
Data are available upon reasonable request. This study involved secondary analysis of routinely collected data. The anonymised data used for analysis are available from the lead author.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- patient reported outcome measures
- primary care
- quality improvement
- independent living
- patient-centred care
Introduction
Patient-reported outcome measures (PROMs) capture patients’ own perceptions of aspects of their life. This includes feelings relating to lack of social contact and loneliness. PROMs are used to inform patients’ decisions and clinical care, to understand outcome variations and inequalities and for other purposes.1
Social prescribing is a way for general practitioners (GPs) and other local agencies to refer people to a link worker. Link workers give people time to help them with their concerns, focusing on what matters to them as individuals and taking a holistic approach to dealing with non-medical needs and well-being.2 3 Social prescribing can have a beneficial impact on loneliness.4
Loneliness is defined as the unpleasant experience that occurs when a person’s network of social relations is deficient in some important way, either quantitatively or qualitatively.5 It can affect all ages and may be classified in terms of social, emotional and existential loneliness.6 Social loneliness occurs when there is a mismatch between the quantity and quality of social relationships and those we want. Emotional loneliness describes the absence of meaningful relationships. Existential loneliness is less related to the specifics of relationships but is focused on a more global evaluation of disconnection from others and the wider world.
Perceived lack of social contact is treated as being synonymous with loneliness in this paper. Both differ from social isolation. Some socially isolated people do not feel the lack of social contact or feel lonely, but others do. Loneliness is related to other measures of health and well-being. Mortality rates are more than 25% higher in lonely people.7 8 During the COVID-19 pandemic awareness of the extent and impact of loneliness across the whole population has increased, although risk factors have not changed.9 10
Loneliness measures are direct, or indirect. Direct measures of loneliness ask directly about loneliness, usually with a single item. Indirect measures cover multiple aspects of the concept of loneliness, but do not use the terms lonely or loneliness. Indirect measures can indicate what might be done to reduce loneliness. Existing indirect measures include: the UCLA (University of California, Los Angeles) loneliness scale with 3 items having 3 options each11; the De Jong Gierveld Loneliness scale with 6 items having 3 options each and distinguishes between emotional and social aspects12 and the Campaign to End Loneliness scale with 3 items having 5 options each.13
In 2018, the UK Department for Digital, Culture and Sport issued a major report on loneliness, setting out a national vision to end loneliness within our lifetimes. It included a commitment to produce a new national measure of loneliness.8 The Office of National Statistics (ONS) issued recommendations on how to measure loneliness in large-scale national surveys.14 A central principle was to reuse existing measures, not to develop new ones.15 In 2020, the Government Statistical Service issued a GSS Loneliness Harmonised Standard, based on work by ONS. This is based on the UCLA loneliness scale plus a direct measure with five options.16
R-Outcomes has previously developed several PROMs covering health status,17 personal well-being,18 health confidence19 and patient-reported experience measures.20 These measures all share a common look and feel, based on the following common principles21:
Applicable to all people irrespective of their age, gender, health conditions or any other factors.
Short, with a low reading age.
Positive wording.
Cover multiple aspects of the concept recognised in the literature.
Generate a single summary score for use alongside item scores.
Mean item and summary scores for a group are reported using a 0–100 scale, where a high mean score is always better than a low mean score. This means that a before and after improvement is always a positive number.
These measures share a common format with:
Four items per measure (although this is not essential).
Four options for each item, listed from good to bad.
Brevity, as short as possible.
Simple wording with a reading age of less than 10.
Optional use of colour and emojis for each option to indicate good (green smile), bad (sad red) and so on.
Customers, who were using R-Outcomes measures in social prescribing, asked us to develop a measure of social contact/loneliness. This paper describes both the R-Outcomes Social Contact and the R-Outcomes Loneliness measures and compares these with other widely used measures for length readability and so on.
We have also validated the performance of the Social Contact measure in social prescribing. The relevant research objectives were:
Show the psychometric properties of the R-Outcomes Social Contact measure used in a social prescribing project, including descriptive statistics, internal correlations and reliability (Cronbach’s α).
Use exploratory factor analysis to examine the overlap between the R-Outcomes Social Contact measure with other R-Outcomes measures.
Construct validation of the R-Outcomes Social Contact measure, by testing hypotheses, based on the literature review.
Development
R-Outcomes Social Contact measure
Work on the R-Outcomes Social Contact measure started in about 2016, following explicit requests from customers involved in social prescribing to measure loneliness in a positive way. We studied the UCLA,11 De Jong Gierveld12 and Campaign to End Loneliness13 measures.
However, none of these existing measures meet the criteria set out above, so we developed our own measure. We used detailed reading of the literature to identify the most important aspects of the subject identified by academic research together with informal focus groups with actual and potential customers, primarily in social prescribing, to understand user needs. This took place over a 3-year period, including prototype versions.
We wanted to:
Meet the criteria set out above.
Cover both emotional loneliness and social loneliness.
Use positive wording, focusing on what prevents or ameliorates loneliness.
Avoid using a recall period, such as a week or a month or ask about frequency, because human memory is fallible and use of recall periods is a source of unreliability.22 For this reason we ask about people’s perception, using options: strongly agree, agree, neutral and disagree.
We eventually settled on four items:
I have people to talk to (companionship).
I have someone I can confide in (our need for close emotional contacts).
I have people who will help me (practical social relationships).
I do things with others (not feeling left out).
This has been briefly described previously.21 The final version is shown in figure 1.
R-Outcomes Loneliness measure
When the ONS issued early versions of what became GSS Loneliness Harmonised Standard,14 16 we set out to incorporate these into a format compatible with the criteria stated above. We noted and sought to avoid having (a) 3 items with 3 options and 1 item with 5 options, (b) improvements (reduced loneliness) shown as negative numbers and (c) different ways of scoring for different items.
In the GSS Harmonised Standard, for the first three questions the options: hardly ever or never equates to one, some of the time to two and often to three. After scoring these, they are summed to create a total, where the lowest possible total score is three and the highest nine. There is no accepted level for which to consider a person lonely. However, it is useful to observe changes in average score over time or compare the average scores of different groups.
The fourth question in the GSS Loneliness Harmonised Standard can be reported by percentage of respondents selecting each response option. For example, this could be the proportion stating that they often or always feel lonely, or the proportion that state they never feel lonely. ONS suggests defining levels of loneliness as the proportion of people reporting often or always feeling lonely.
We modified the GSS Loneliness Harmonised Standard proposals in a way that is similar to what we did with ONS4 personal well-being questions.18 Figure 2 shows the Loneliness measure we have developed.
This meets the criteria listed above, apart from positive wording. The main changes from the GSS Loneliness Harmonised Standard are the use of four consistent simplified options for each item, from worst to best (often, sometimes, occasionally and hardly ever). This has been published with minimal description21 but not validated. We include it here with more detail because we think it is a useful contribution.
Methods
Length and readability
We compared the number of items (questions), number of response options, word-count, reading age, score range, direction of improvement, best and worst mean scores for the two measures described above (R-Outcomes Social Contact measure, the R-Outcomes Loneliness measure), and the GSS Loneliness Harmonised Standard,16 the De Jong Gierveld measure of loneliness 6-item scale12 and the Campaign to End Loneliness measure of loneliness.14
Reading ages have been estimated using the Flesch Kincaid Grade (FKG), which approximates to the US school year grade; so, reading age is FKG plus 5. FKG is an optional item in the Microsoft Word Tools/Spelling and Grammar menu. The text used is based on original author’s text, including a brief preamble (eg, thinking about your friends and family), the response options separated by commas (not repeated) and the items (questions) as separate sentences. Materials for use by vulnerable patients should have a reading age less than 10 years old (FKP plus 5).23
Data collection
A convenience sample of results was analysed from Tri Locality Care, which provides social prescribing and other services for patients registered with three primary care networks northwest of Southampton. Tri Locality Care has been using R-Outcomes PROMs since 2016.
The following data are reported here, although the original surveys included more items, including free-text comments:
Type of contact (new referral, post referral, one off contact, patient died or left area).
Date of survey.
Age group in deciles.
Gender (male or female).
*Social Contact measure.
*Health status measure.17
*Health confidence measure.19
*Medication adherence measure.21
*Patient experience measure.20
*Two personal well-being items (Happy yesterday and Not anxious yesterday).18
*Two social determinants of health (SDoH2) items (I am happy about where I live and I have enough money to cope).21
Measures marked with * were developed by R-Outcomes Ltd.
The data were cleaned to remove ratings made at one-off visits and only include responses having all four loneliness items. Summary scores were calculated for loneliness, health status, health confidence, medication adherence, patient experience, personal well-being and SDoH.
Psychometric data
Analysis was performed using parametric and non-parametric statistics with the JASP statistics package24 for new referrals (before) and post referral (after), the population age and sex, loneliness items and summary score and the summary scores for other measures. Differences between before and after scores were tested for significant differences.
Intra-item correlations and Cronbach’s α was calculated. Cronbach’s α should be in the range 0.70–0.90 for this type of short survey if it is to support the use of a summary score.25
Exploratory factor analysis25 was used to assess the extent to which the loneliness measure and other R-Outcomes measures cover the same domains.
Construct validation
Construct validity is the extent to which the scores are consistent with hypotheses, based on the assumption that the measure is a valid measure of the construct being measured.26 This is assessed in this population by testing the following hypotheses, which have previously been reported:
Social contact/loneliness is improved following social prescribing intervention.4
Social contact/loneliness is worse in women than men.4
Perceived loneliness is worse in younger people.27
Social contact/loneliness has moderate correlation with personal well-being (eg, happiness and absence of anxiety).28
Social contact has moderate correlation with SDoH (eg, poor housing and poverty).29
Social contact has moderate correlation with health status.30
Social contact is correlated with health confidence (self-efficacy).31
Social contact has low correlation with medication adherence.32
Social contact has low correlation with patient experience.33
Conventional correlation thresholds are used: high over 0.5; moderate 0.3–0.5; low under 0.3.
Ethics statement
This paper uses secondary analysis of data collected as part of routine service monitoring of a social prescribing service. The data were anonymous and undertaken to monitor the current service performance without randomisation, so ethics approval was not required. No data were collected unless patients consented and there was no risk to individual participants.34
Patient and public involvement
The need for a simple loneliness measure of personal well-being was an explicit request from Tri Locality Care and other social prescribing projects. The measure was co-designed with them.
Results
Length and readability
Table 1 shows the properties of the R-Outcomes Social Contact measure, the R-Outcomes Loneliness measure, the GSS Loneliness Harmonised Standard, the De Jong Gierveld 6-item measure of loneliness and the Campaign to End Loneliness measurement tool.
The number of items and options is broadly similar for all five measures. The R-Outcomes Social Contact measure and the R-Outcomes Loneliness measures have the lowest wordcounts. The De Jong Gierveld 6-item measure has the lowest reading age.
Item wording is positive in the loneliness measures of R-Outcomes and the Campaign to End Loneliness. In the GSS Loneliness Harmonised Standard wording is negative while De Jong Gierveld has 3 items positively and 3 items negatively worded.
Scoring differs considerably. For groups of people the mean R-Outcomes scores are reported on a 0–100 scale for each item and the summary score, where a high score is good (not lonely). Improvement is a positive number.
In contrast the other measures are negatively scored (a high score is undesirable). The GSS Loneliness Harmonised Standard uses a combination of the UCLA 3-item scale with a range from 3 to 9 (least to most lonely) and a direct measure with 5 options, which is reported as the % in the category ‘often or always lonely’.
De Jong Gierveld is reported on a 0–6 range (least to most lonely) and the Campaign to End Loneliness measure is reported using a 0–12 scale (least to most lonely). Using these measures, improvement (reduced loneliness) is indicated by a negative number.
Validation data
The data shown below were collected between January 2019 and November 2020 (n=728).
Patients referred to the service are asked by a staff member to complete surveys at the time of their first visit and last scheduled visit (typically 6–8 weeks later). The staff member usually enters the data into a computer or tablet. Since the COVID-19 lockdown (23 March 2020) almost all face-to-face visits were replaced by telephone calls.
Overall missing data rates were below 5%. The number of responses is not the same for each measure, mainly due to operational changes made during the 22-month data collection period.
Table 2 shows the distribution of results by age group and gender before and after the social prescribing intervention. The mean R-Outcomes Social Contact score for each group is also shown (note that a high score indicates less loneliness).
Ninety per cent of this population were over 70 years old; 62% over 80; 62% female. There was no significant difference between before and after populations by age group (χ²=5.421, df=7, p=0.609) or gender (χ²=0.914, df=1, p=0.339).
Table 3 shows the number of responses, the mean score and SD for each group (All, Before and After) for Social Contact items and summary score and the summary scores of other measures; the difference between Before (B) and After (A) scores (A−B), and the probability that changes could be by chance. All p values are significant (p<0.05). The smallest mean changes are for I have people to talk to (social contact – companion item) and the social determinants of health (SDoH2) items.
The lowest item mean score is for I do things with others (Join in, 46.8). This may reflect the social prescribing population, many of whom have disabilities.35 The highest item mean score is for I have people who will help me (People help, 76.1), which indicates that support services are generally satisfactory. Low scores on this item would be a trigger for remedial action.
Among other measures the lowest overall score is the summary score for the two personal well-being items (PWS2 48.2). The highest is for patient experience summary score (Experience 81.3). Notably, these two measures show the greatest improvement (11.6 and 14.4, respectively). This may be because both personal well-being and patient experience respond well to the increased support offered by social prescribing services.
Figure 3 shows the mean scores for the four Social Contact items and the summary scores for Social Contact and other R-Outcomes measures used. The error bars show the 95% CIs. This chart shows substantial differences within this population in the mean scores of items.
Internal reliability
Table 4 shows the inter-item correlation matrix. In this population the item I do things with others has a lower inter-item correlation than the other items. This may be because most respondents are over 80 years old with disability, many have lost their partners and some of the data were collected during the COVID-19 pandemic.
Cronbach’s α=0.758 which is within the recommended range. This, along with the inter-item correlation matrix suggests that it is appropriate to have a summary score as well as the four items.
Factor analysis
Table 5 shows exploratory factor analysis loadings using Promax rotation for each question item.
This shows that each measure addresses a different concept. Factor 1 covers patient experience; factor 2, medication adherence; factor 3, mental distress/well-being; factor 4, functional health status; factor 5, loneliness; factor 6, health literacy and self-efficacy aspects of health confidence; factor 7, trust and help aspects of loneliness as well as the getting help and shared decisions aspects of health confidence and factor 8, SDoH.
This provides further evidence supporting the construct validity of the Social Contact measure as well as other R-Outcomes measures.
Construct validation
Table 6 shows correlations between measures. The highest correlation is between personal well-being (PWS2) and health status, which includes an item on well-being (r=0.508). The lowest is between loneliness and patient experience (r=0.254). All correlations are significant.
The results of testing the hypotheses listed in the Methods section are shown in table 7. The results are consistent with the hypotheses. This provides strong construct validation evidence for the R-Outcomes Social Contact measure.
Discussion
This is the first full paper on the R-Outcomes Social Contact measure, which is a short positively worded measure of perceived social contact. Perceived lack of social contact is a very similar concept to loneliness.
We also describe the R-Outcomes Loneliness measure, which is based quite closely on the GSS Loneliness Harmonised Standard.16 The Harmonised Standard is designed mainly for national surveys. This measure has not yet been validated.
In routine clinical and related work, such as social prescribing, it is often useful to measure several aspects of a patients’ lived experience. As a consequence, surveys need a consistent set of measures which work well together.
Both measures have been briefly described previously, where the Social Contact measure was referred to as Loneliness and the Loneliness measure as Loneliness (ONS).21
Both measures share several benefits in comparison with longer established measures: brevity and low reading age, with four items and four response options, which are aggregated to give a single summary score. Mean scores for each item and the summary score are presented on a 0–100 scale (where high is good), facilitating comparisons with other measures of health, which are usually positively scored. The R-Outcomes Social Contact measure is worded positively.
Most loneliness measures, including the R-Outcomes Loneliness measure are worded and/or scored negatively. People prefer to answer questions in a way that is more socially desirable, preferring positive to negative wording.36 Negative scoring (the score increases with loneliness) generates a negative number if an intervention improves loneliness.
The validation study used the R-Outcomes Social Contact measure. The study also showed that this measure is very acceptable to people of all ages. All inter-item correlations are within the desired range for a single measure. The Cronbach’s α result (α=0.76) suggests that it is appropriate to use an aggregate summary score.
The four items in the Social Contact measure address different aspects of loneliness and the mean scores differ. It is useful to report the mean scores for each item individually as well as the summary score. The highest mean score (76.1) is for I have people who will help me while the lowest mean score (46.8) is for I do things with others. This may reflect good neighbour relations in the first instance and patients’ disability and the impact of the COVID-19 pandemic in the second. Low scores on any items may prompt a member of staff to consider if and how they can alleviate the problems identified.
Exploratory factor analysis shows that the R-Outcomes Social Contact measure and other R-Outcomes measures address different concepts, although they are correlated. Strong construct validity evidence was demonstrated when each of the hypotheses, previously identified in the loneliness literature was confirmed.4 27–33
Completing PROMs can help patients think about themselves in ways that are beneficial and this information can also help clinicians build relationships with patients based on better understanding of how they live.37
This study has limitations. We cannot say whether any improvements in social contact/loneliness or other measures will be sustained. However, we can say that patients report reduced loneliness and better scores on patient-reported measures after social prescribing help than before. This limitation is also found in other studies.27 38
Comparisons between before and after groups are based on the cohorts (independent samples), not individuals (paired comparisons), which may have been more powerful, but this was not possible with our convenience sample. It might be expected that some people, who completed the before rating but not the after rating, may have been less lonely at the start and have less need for social prescribing, than people who did both. If so, the improvement in loneliness would be greater, but a limited analysis with matched pairs shows that the scores are similar (within ±1 point on 0–100 scale).
Parametric statistical tests results are shown, which are more familiar than their non-parametric equivalents. We have performed non-parametric tests, but these make no important difference to the results shown.
This study adds to the evidence that social prescribing delivers benefits across multiple dimensions. The largest differences between before and after ratings are in patient experience, personal well-being and medication adherence. These may have improved because social prescribing link workers have more time than GPs and other clinical staff to listen to patients' problems, help and explain issues. Some domains, which are harder for social prescribing interventions to change for most people, show smaller improvements. These include housing and poverty (SDoH) and underlying health conditions (health status). Social Contact falls somewhere in the middle.
This study is not a representative survey of the whole population, so we cannot say whether loneliness is more common in younger or older people, because only 10% of the study population were under 70 years old. However, the mean Social Contact score of younger people referred to social prescribing was more severe than that of older people.
These measures are designed for use in local projects, such as in social prescribing, where respondent burden is important and other domains need to be measured as well as loneliness. They combine well with other R-Outcomes measures, which can be picked and mixed to produce different surveys to meet local needs.21 A high score is good for all measures, so any improvement is shown as a positive number and deterioration is shown as a negative number.
These measures are released under the Creative Commons Attibution-NonCommercial-ShareAlike 4.0 International license (CC-BY-NC-SA). This means that they can be used in any non-commercial way, such as for education or small unfunded projects. If you wish to use in any other way, please contact R-Outcomes.
Conclusions
This paper describes two new measures. The R-Outcomes Social Contact measure and the R-Outcomes Loneliness measure, which is more closely based on the GSS Loneliness Harmonised Standard. Both are shorter than other loneliness measures.
The study used the R-Outcomes Social Contact measure and demonstrates good psychometric properties and construct validity in a social prescribing. This measure should be useful in other areas where social contact/loneliness measures are needed.
Data availability statement
Data are available upon reasonable request. This study involved secondary analysis of routinely collected data. The anonymised data used for analysis are available from the lead author.
Acknowledgments
We are grateful to all of the staff and patients of Tri Locality Care who helped in the development and collected the data used in this paper.
References
Footnotes
Twitter @timbenson
Contributors TB developed the measures, performed the data analysis and wrote the paper. HS, NW and PM were responsible for data collection and provided input into the design of the measures.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests TB is founder and director of R-Outcomes Ltd, which provides services to enable feedback of patient-reported information to those providing health and care services, for evaluation and routine monitoring. HS was employed by R-Outcomes Ltd. NW and PM are employed by Tri Locality Care Ltd, which collected the data.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.