Assessment of Loneliness During the Pandemic: Comparing Various Short Forms of the UCLA Loneliness Scale in South Africa Using Classical Test Theory and Mokken Analysis
Department of Psychology, University of the Western Cape, Robert Sobukwe Road, Cape Town, South Africa
† These authors contributed equally to this work.
Academic Editors: Ami Rokach and David Berman
Special Issue: The Many Shades of Loneliness
Received: May 01, 2022 | Accepted: August 09, 2022 | Published: August 17, 2022
OBM Neurobiology 2022, Volume 6, Issue 3, doi:10.21926/obm.neurobiol.2203132
Recommended citation: Pretorius T, Padmanabhanunni A. Assessment of Loneliness During the Pandemic: Comparing Various Short Forms of the UCLA Loneliness Scale in South Africa Using Classical Test Theory and Mokken Analysis. OBM Neurobiology 2022;6(3):22; doi:10.21926/obm.neurobiol.2203132.
© 2022 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.
During the COVID-19 pandemic, governments around the world implemented stringent lockdown measures making social isolation and remote work mandatory. Although these regulations were necessary to prevent the spread of the virus, they significantly deteriorated mental health conditions, characterized by increased rates of depression, anxiety, and loneliness. Loneliness was the most prevalent mental health issue associated with COVID-19 , which led the COVID-19 pandemic to be known as the “loneliness pandemic” . Loneliness is an uncomfortable emotional state arising from the perception that the social needs of an individual are not being fulfilled . Some studies have empirically shown that the prevalence of loneliness has increased in the general population. For example, a study in the United Kingdom  reported a loneliness prevalence rate of 27% in 2020, while another study  found that 28.63% and 7.20% of the participants reported feeling lonely sometimes and often respectively, in 2020. A study based on six Middle Eastern countries conducted in 2021 reported that 40.6% of the respondents expressed a moderately high degree of loneliness . Conversely, one study found no significant changes in loneliness during various phases of the pandemic . Some studies have investigated the trajectory of loneliness during the pandemic [8,9]. Others have reported an increase in the prevalence of loneliness in the general population at the onset of the COVID-19 outbreak, with especially high rates among people under quarantine . A study also found a decrease in the level of loneliness by the first year of the pandemic . This might be due to several factors, including a decrease in compliance with social distancing and quarantine measures and an increase in the use of technology to connect with others [11,12]. Some studies focused on identifying sub-groups of the population that might be more susceptible to loneliness, including adolescents , the elderly , and university students , as well as the correlates of loneliness. Changes in the romantic status during the pandemic, loss of a significant other due to infection, young individuals, living alone, and pre-existing mental health problems are associated with the risk of loneliness .
The Evolutionary Theory of Loneliness  provides a framework for understanding the increase in the perception of loneliness during the pandemic. It states that people have an innate need for social connections as it provides protection and helps people in managing adversities. In the absence of social connections, people feel lonely. When people feel lonely, they seek social contact; however, during the pandemic, a social connection was considered to be a potential threat due to the risk of infection. Additionally, the need to follow government guidelines regarding social distancing and quarantine prevented people from contacting others. Keller et al.  suggested that this negative feedback loop where social contact is considered to be dangerous and the resultant social withdrawal can produce chronic feelings of loneliness, characterized by a feeling of isolation and lack of meaningful connections .
The importance of conducting studies on loneliness has been highlighted by pre-pandemic studies, where loneliness was found to be associated with adverse mental health outcomes, including depression and anxiety, and high mortality . Similar results were found during the pandemic. Specifically, loneliness was found to be associated with significant psychological distress, lower engagement in COVID-19-related prevention measures, and an increase in maladaptive coping mechanisms, including substance use [12,17]. Thus, further studies on loneliness and its effect on mental and physical health are necessary for developing COVID-19 public health interventions.
To determine the effects of pandemic-related loneliness, its correlates, and the protective factors that might mitigate the problems, a detailed investigation using robust instruments with good psychometric properties is necessary for research and clinical purposes. Additionally, the metric equivalence of instruments across different contexts needs to be established. This might help to determine valid inter-group differences.
The most widely used measure of loneliness is the University of California, Los Angeles Loneliness Scale (UCLA-20). This scale has three versions; the original version was published in 1978 , in which all items were negatively phrased; a revised version was published in 1980 and contained 10 positive and 10 negative items ; finally, a third version was published in 1996 in which the response format was modified, and the scale consisted of 11 negative items and nine positive items . Authors have generally provided satisfactory reliability and validity indices for the different versions of the UCLA-20 [18,19,20]. Similar satisfactory reliability and validity indices have been reported in different countries, including Iran , Poland , Japan , Spain , and Denmark . However, although the psychometric properties were satisfactory, the results regarding the dimensionality and factor structure of the UCLA-20 were inconsistent. Russel considered the UCLA-20 to be unidimensional and showed through factor analysis that the UCLA-20 assesses a global bipolar measure of loneliness with two uncorrelated method factors, in which the negative items loaded on one factor and the positive items loaded on another factor . Other studies have found mixed results. One-factor , two-factor , three-factor [27,28], and bifactor solutions [29,30] have been reported.
One disadvantage of the UCLA-20 is its relative length. The scale is seldom used independently as researchers are often interested in the correlates of loneliness. When used along with other instruments, the length of the UCLA-20 can lead to a questionnaire burden on the respondents, as well as erratic responses, where the respondents respond without paying close attention to the questions. To overcome this problem, several shorter versions of the UCLA-20 have been developed, including a three-item version , a four-item version , a five-item version , a six-item version , a seven-item version , an eight-item version , a 10-item version , an 11-item version , and a 16-item version .
Das et al.  reviewed the loneliness scales used in pandemics and found that the UCLA-3, UCLA-8, and UCLA-10 were the three short versions of the UCLA-20 most frequently used, specifically in 56% of the studies they reviewed. In this study, we investigated the psychometric properties of three short versions of the UCLA Loneliness Scale, including two versions previously used, i.e., the 10-item version  and the eight-item version , as well as a five-item version developed in this study. We used the classical test theory and item response theory (Mokken analysis) in two different groups of samples, one including teachers and the other including students. Assessing measurement equivalence across different population groups is important [39,40]. Similar coherence or structure of the psychometric properties of data from multiple groups provides strong evidence of metric equivalence. This is the first study from South Africa to use the Mokken analysis to investigate the psychometric properties of the UCLA loneliness scale.
2. Materials and Methods
The participants consisted of a random sample of students from a university in Cape Town, South Africa (N = 355) and a convenience sample of teachers (N = 337) from different parts of South Africa. Most participants in the student sample were women (77.2%) and resided in an urban area (75.4%); the mean age of the student sample group was 21.95 years (SD = 4.7). The participants in the teacher sample group were mainly based in the Western Cape province of South Africa (82.3%) and resided in urban areas (61.7%). Most teachers were women (76.6%), and the mean age of the group was 41.89 years (SD = 12.42).
Both studies were approved by the Institutional Review Board of the University of the Western Cape (Ethics reference number: students, HS20/5/1, 1 June 2020; teachers: HS21/3/8, 14 May 2021). The studies were conducted following the guidelines of the Declaration of Helsinki. Participation was voluntary, and the participants provided informed consent on the landing page of the link to the survey.
The instruments used to conduct this study included the UCLA Loneliness Scale (UCLA-20] , the trait scale of the State-Trait Anxiety Inventory [STAI-T] , the Beck Hopelessness Scale [BHS] , the Center for Epidemiological Studies Depression Scale [CES-D] , and the Satisfaction with Life Scale [SWLS] . These instruments were used to assess the concurrent validity of the UCLA short-form scales. Additionally, participants completed a brief demographic survey.
The UCLA-20 is commonly used to measure loneliness . It consists of 20 items with a four-point scale ranging from 1 (Never) to 4 (Often). The estimates of internal consistency range from 0.88–0.93, as found in various studies [10,11,12,13,14,21,22,23,24,25]. Two South African studies reported acceptable reliability (0.81 and 0.92) for the UCLA-20 when used with student samples [46,47]. The structure of the UCLA-20 was also confirmed using confirmatory factor analysis (CFA) with ancillary bifactor indices in a South African study .
The STAI-T is a 20-item measure of trait anxiety with a four-point scale ranging from “Almost Never” (1) to “Almost Always” (4). An example item of the STAI-T is, “I feel that difficulties are piling up so that I cannot overcome them”. A recent meta-analysis showed that the STAI has been cited in more than 17,000 papers . The STAI-T generally demonstrates satisfactory reliability ranging from 0.87–0.93 . The scale was also used in South Africa  and showed an internal consistency estimate of 0.90.
The BHS is a widely used 20-item measure of hopelessness. All 20 items are responded to on a dichotomous scale of “True” or “False”. An example item of the BHS is, “I cannot imagine what my life would be like in 10 years”. The BHS generally shows adequate reliability, with estimates ranging from 0.89–0.98 [51,52,53]. The BHS was used in two studies on young adults in South Africa, and the reliability estimates of 0.88 and 0.82 were reported in those studies [54,55].
The CES-D is a 20-item measure of depressive symptoms and uses a four-point scale ranging from “Most or all of the time” (0) to “Rarely or none of the time” (3). An example item of the CES-D is, “I could not get going”. The CES-D has been translated into several languages, including Farsi , Chinese , German , and multiple Nigerian languages , and the reliability estimates are generally above 0.75. The CES-D was used in two studies in South Africa, with reliability estimates of 0.90  and 0.92 .
The SWLS is a five-item measure of life satisfaction. It is scored on a seven-point scale ranging from 1 (Strongly disagree) to 7 (Strongly agree). An example item of the SWLS is, “If I could live my life over, I would change almost nothing”. The reliability and validity of the SWLS are satisfactory [62,63,64], with reliability coefficients ranging from 0.86–0.91. The SWLS has also shown satisfactory reliability in a South African study .
To conduct both studies, we used Google Forms to develop a web-based version of the instruments. The student sample was randomly selected (95% confidence interval, 5% margin of error) through the Registrar’s office. For this purpose, the Registrar’s office used an algorithm on the email addresses of students that generated a random number for each email address. The link was emailed to the selected students, and reminders were sent twice per month for four months. The teacher participants were recruited by convenience sampling. An electronic link was posted on several Facebook sites for the teachers.
2.4 Data Analysis
The classical test theory analysis is the traditional way in which the psychometric properties of the UCLA Loneliness Scale are examined. IBM SPSS Statistics (version 28) was used for the classical test theory analysis, which included inter-item correlations, item-total correlations, exploratory factor analysis (EFA), descriptive statistics, and reliabilities. Inter-item correlations should range from 0.15 to 0.85 ; an inter-item correlation higher than 0.85 indicates redundancy of items. Item-total correlations indicate the extent to which each item contributes to the measurement of the latent construct and should be greater than 0.50 . IBM SPSS Amos version 27 (IBM Corp.) was used to perform a CFA. The indices that were used to examine model fit in the CFA  included chi-squared (χ2, good fit: p > 0.05), the goodness-of-fit index (GFI, good fit: >0.95), the Tucker-Lewis index (TLI, good fit: >0.90), the comparative fit index (CFI, good fit: >0.90), and the root-mean-square error of approximation (RMSEA, good fit <0.08). To support the CFA, ancillary bifactor indices were calculated using the Bifactor Indices Calculator , which included OmegaH, the explained common variance (ECV), and the percentage of uncontaminated correlations (PUC). The ECV describes the percentage of variance in the items accounted for by the total scale (also called the general factor) and the subscales (also called the specific factors). OmegaH indicates the proportion of variance in total scores that is explained by the total scale, and PUC describes the percentage of correlations between items that can be explained by the total scale [69,70]. While there are specific criteria for these indices, Reise et al.  suggested that they should be considered together. When PUC values are below 0.80, ECV is above 0.60, and OmegaH is above 0.70, the scale can be considered unidimensional.
Since the four-point response format of the UCLA loneliness scale is a weak ordinal scale, Mokken analysis, which is a non-parametric item-response theory (IRT) approach, was also used to examine the psychometric properties of the loneliness scale. IRT approaches are less concerned with total scores and focus more on the responses to individual items . Mokken analysis was conducted using the “Mokken” package  in R . Mokken analysis is a non-parametric alternative to the parametric item response theory, such as Rasch analysis. Mokken analysis provides a scalability coefficient for each item (Hi), as well as for the total scale (H). The coefficient Hi indicates the contribution of every item to the measurement of the latent trait. Mokken  suggested that an Hi coefficient greater than 0.30 indicates that the item contributes to the measurement of the scale. The H coefficient for the entire scale provides a measure of the homogeneity of items and indicates the strength of the scale. Sijtsma and van der Ark  suggested that H <0.40 indicates a weak scale, H ≥0.40 and <0.50 indicates a moderate scale, and H ≥0.50 indicates a strong scale. The Mokken analysis also provides an automated item selection procedure (AISP), which indicates whether a scale is unidimensional or multidimensional. To determine whether items can distinguish between high and low scorers on the latent trait (monotonicity), the Mokken analysis provides a Crit value for each item. Finally, another Crit value is used to identify items that respondents with the same standing on the latent trait might have responded to differently (invariant item ordering or IIO). For both monotonicity and IIO, Crit values >80 indicate serious violations .
Other types of validity of the investigated short-form scales include convergent, discriminant, and concurrent validity. The average variance extracted (AVE) and composite reliability (CR) were used to assess convergent validity. The AVE reflects the level of variance explained by the latent construct, as opposed to the variance attributable to the measurement error; CR is considered to be a less biased measure of reliability than Cronbach’s alpha. In general, AVE above 0.50, CR above 0.70, and AVE less than CR indicate convergent validity . Discriminant validity was assessed by comparing the variance in the items accounted for by the latent construct to the variance the construct shares with other constructs. Discriminant validity occurs if the latent construct accounts for greater variance in the items that contribute to its measurement (AVE) than the variance shared with related constructs (maximum shared variance or MSV; average shared variance or ASV), such as anxiety and depression . Finally, concurrent validity was assessed based on correlations with other indices of psychological well-being, including anxiety, hopelessness, depression, and life satisfaction.
The reliabilities of the five scales used in the study were as follows: UCLA-20: α = 0.92 for teachers and students; STAI-T: α = 0.91 for teachers and α = 0.90 for students; BHS: α = 0.89 for teachers and α = 0.88 for students; CES-D: α = 0.92 for teachers and students; SWLS: α = 0.90 for teachers and α = 0.89 for students. The inter-item correlations for the 10-item UCLA for the sample comprising teachers are reported in Table 1.
The descriptive statistics and inter-item correlations for the UCLA-8 are reported in Table 2.
All inter-item correlations for the UCLA-10 were within the acceptable range of 0.15–0.85  in the sample comprising teachers. In the sample comprising students, all inter-item correlations were within the acceptable range except for the intercorrelation between items 9 and 10. The intercorrelation between the items “There are people I can talk to” (item 9) and “People I can turn to” (item 10) was 0.87, which indicated redundancy between these items.
All inter-item correlations for the UCLA-8 were within an acceptable range for the sample comprising teachers except for the intercorrelation between items 4, 6, and 7. The intercorrelations between item 6 (“I can find companionship”) and both item 4 (“I feel left out”) and item 7 (“I feel isolated from others”) were below 0.15. In the sample comprising students, the only intercorrelation below 0.15 was between items 6 and 7. These findings suggested that items 6 and 7 might not contribute to the homogeneity of the items.
The UCLA-10 and UCLA-8 were subjected to EFA (principal components with varimax rotation) for the samples comprising teachers and students. The results of the EFA for the sample comprising teachers are reported in Table 3. The results were similar for the sample comprising students, and thus, only the results of the sample comprising teachers are reported here.
The two factors explained 59.27% and 58.64% of the variance in the eight-item version of the scale for students and teachers, respectively. In the case of the 10-item version, the two factors explained 66.16% and 64.99% of the variance for students and teachers, respectively. All positive items loaded on one factor and all negative items loaded on another factor. To determine whether these are substantive factors that account for a considerable proportion of variance, a CFA with ancillary bifactor analysis was conducted. The UCLA-8 bifactor model is shown in Figure 1A, and the UCLA-10 bifactor model is shown in Figure 1B.
Figure 1 The bifactor models of UCLA-10 and UCLA-8. The regression weights were standardized. Rectangles indicate measured variables, and ellipses indicate latent variables.
The model fit indices showed that the two models fit the data very well. For both UCLA-8 and UCLA-10, the fit indices were GFI = 0.98, TLI = 0.99, CFI = 0.99, and RMSEA = 0.03. For UCLA-8, χ2 = 16.84 and p > 0.05, and for UCLA-10, χ2 = 31.40 and p > 0.05. Despite the well-fit indices, the pattern of loadings was problematic in both models. There were negative pattern loadings and factor loadings below the generally acceptable level of 0.70 . In the UCLA-8, the loadings on all subscales were non-significant, but all were significant on the total scale (p < 0.01). However, items 3 (“I am an outgoing person”), 6 (“I can find companionship”), and 7 (“I am unhappy about being withdrawn”) had loadings less than 0.50 on the total scale. Along with these problematic loadings, the ancillary bifactor indices did not support the interpretation of these two versions of the UCLA-20 as multidimensional. The ECV of the total scale of UCLA-8 and UCLA-10 were 0.71 and 0.66, respectively, for the sample comprising teachers. The OmegaH was 0.75 for both versions, and the PUC was 0.43 and 0.56 for UCLA-8 and UCLA-10, respectively. The sample comprising students had similar indices. The finding that PUC was below 0.80, ECV was above 0.60, and OmegaH was above 0.70 indicated that there was some multidimensionality but not enough to rule out the interpretation that the scale was essentially unidimensional .
For the UCLA-10, the AISP found that all 10 items loaded on one scale in the samples comprising teachers and students, which indicated that the scale was a unidimensional measure of loneliness. However, for the UCLA-8, AISP found that the items loaded on two scales in the samples comprising teachers and students; hence, the eight-item version was not unidimensional. The Mokken and classical test theory indices for the items of the short-form scales are reported in Table 4.
The UCLA-10 had acceptable indices at the item level in both samples, as shown in Table 4. All item-total correlations were above 0.50, and all Hi coefficients were greater than 0.30. These item-total correlations and Hi coefficients suggested that all items of the UCLA-10 contributed to the measurement of the latent construct. All Crit values for both monotonicity and IIO were below 80, indicating minor but acceptable violations. For the UCLA-8, items 3 (“I am an outgoing person”), 6 (“I can find companionship”), and 7 (“I am unhappy about being withdrawn”) had item-total correlations below 0.50 and Hi coefficients below 0.30, indicating that these items did not contribute to the measurement of the latent construct. These items were also identified by the factor loadings in CFA as misfitting items. The inter-item correlations also showed that items 6 and 7 did not contribute to the homogeneity of the items. Additionally, there were two violations of monotonicity and four violations of IIO for the UCLA-8 (i.e., Crit value > 80 in the sample comprising teachers and three violations of IIO in the sample comprising students). After removing these misfitting items, a five-item version of the scale was obtained that had acceptable indices at the item level in both samples, as shown in Table 4; the item total correlations were above 0.50, the Hi coefficients were above 0.30, and the violations of monotonicity and IIO were minor.
The Mokken and classical test theory indices for the three versions of the UCLA-20 at the scale level in the two groups of samples are presented in Table 5. The column labeled “Criteria” indicates the acceptable levels for these indices based on previous studies.
As shown in Table 5, all scale indices met the suggested cut-offs for satisfactory indices for the UCLA-10 and UCLA-5 in both groups of samples, except for the H coefficient of the UCLA-10 in the sample comprising students (H = 0.49), which indicated that it is a moderate scale. However, the H coefficient of the UCLA-8 for the sample comprising teachers indicated that it is a weak scale, whereas the H coefficient for the sample comprising students indicated that it is a moderate scale. In both samples of the UCLA-8, the AVE was below the suggested cut-off (>0.50), which reflected that the latent construct accounted for an insufficient proportion of variance in the eight items of the scale. All scales were satisfactorily reliable (0.81–0.91).
Since the results suggested that the five-item and 10-item versions were unidimensional, a CFA was used to further confirm this using a one-factor model. For the five-item version the fit indices were GFI = 0.95, TLI = 0.87, CFI = 0.95, and SRMR = 0.04 and for the 10-item version the fit indices were GFI = 0.96, TLI = 0.95, CFI = 0.98, and SRMR = 0.04, indicating acceptable fit.
The correlations between the UCLA-20 and the indices of psychological well-being were as follows: anxiety, r = 0.65 and p < 0.001; hopelessness, r = 0.55 and p < 0.001; depression, r = 0.65 and p < 0.001; life satisfaction, r = −0.53 and p < 0.001. As shown in Table 5, the various short-form scales had similar correlations with these indices for the two groups of samples (UCLA-10: 0.50–0.61; UCLA-8: 0.47–0.70; UCLA-5: 0.40–0.63).
Although loneliness was considered to be a salient public health concern even before the COVID-19 outbreak, primarily due to its effect on mental and physical outcomes, the pandemic and its containment measures increased social isolation and loneliness. Many studies conducted in different contexts and among various population groups reached a similar conclusion [10,13,14]. Researchers generally conceptualize loneliness as either unidimensional (i.e., a unitary phenomenon that varies in perceived intensity but is similar across situations) or multidimensional (i.e., a multifaceted phenomenon that cannot be captured by a single global loneliness measure). Whether the unidimensional or multidimensional scale provides a more robust measurement of loneliness is not clear . Multidimensional scales help to determine variations in the subjective experience of loneliness and might benefit targeted intervention; however, relatively more studies have investigated unidimensional measures of loneliness [29,79].
Although loneliness can be measured using different scales, for example, the Social and Emotional Loneliness Scale for Adults  and the Differential Loneliness Scale , consensus on the factorial structure of most of these scales or the factorial structure of each of their proposed dimensions is limited. Most analyses of loneliness scales are based on Cronbach’s alpha for scale reliability, the use of correlation to establish discriminant and construct validation, as well as exploratory factor analyses for developing the scale [29,82]. The UCLA loneliness scale is the most popular measure of loneliness globally . This is mainly due to its unidimensional structure, short length, and satisfactory internal consistency reliability. However, the UCLA scale lacks a conceptual definition of loneliness, and the items included in the scale are based on face validity and inter-item correlation rather than a common conceptual understanding of the construct of loneliness .
In this study, we compared three short versions of the UCLA Loneliness Scale to extend the conceptual and theoretical approaches on which these instruments are grounded. The three scales compared were a 10-item version proposed by Russel , an eight-item version proposed by Hays and DiMatteo , and a five-item version developed in this study through the Mokken analysis. The reliability of all short-form scales exceeded 0.80 in the two sets of samples; thus, the reliability of these short-form scales was highly satisfactory.
Russel  proposed that loneliness is a unidimensional construct. For the UCLA-10 and UCLA-8, the EFA and CFA indicated that the items loaded on two factors. However, an ancillary bifactor analysis showed that the two subscales did not sufficiently account for the variance compared to that accounted for by the total scale; thus, the scale can be assumed to be unidimensional. The two-factor solution reflected a method artifact as all the positive items loaded on one factor and all the negative items loaded on another factor. Such an interpretation of unidimensionality in the case of the UCLA-8 was consistent with the findings of Hays and DiMatteo , who used EFA to identify the eight-item version of the scale. In the Mokken analysis, the AISP identified one scale underlying the 10-item and five-item versions; however, it identified that the items of the eight-item scale loaded on two scales. Additionally, unlike the UCLA-10 and UCLA-5, the H coefficient in the Mokken analysis indicated that the eight items of the UCLA-8 formed a weak scale. Thus, the results do not support a unidimensional interpretation of the UCLA-8. Our findings were consistent with those of Elphinstone  that the UCLA-8 did not have acceptable fit indices in a CFA analysis. However, this finding did not support the unidimensional structure reported in other studies [84,85].
The findings of item-total correlation greater than 0.50, inter-item correlations from 0.15 to 0.85, and Hi greater than 0.30 provided satisfactory support for the construct validity of the UCLA-10 and UCLA-5. Further support for construct validity was found based on acceptable Crit values for monotonicity and IIO. The items in these two short-form scales could differentiate between high and low scorers on the latent trait (monotonicity), and all items were treated similarly by respondents with the same level of the latent trait (IIO). The construct validity of the UCLA-8 was not strongly supported in this study. Two items in UCLA-8 had inter-item correlations below 0.15, three items had factor loadings below 0.60 on the total scale based on the CFA, and the Hi coefficients were below 0.30. Additionally, the UCLA-8 violated several assumptions of monotonicity and IIO, indicating that some items could not differentiate between high and low scorers, and some items were responded to differently by respondents with the same level of latent construct. The finding that the UCLA-10 demonstrated greater construct validity than the UCLA-8 were similar to the findings of a systematic review of the UCLA short-form scales, which indicated that high-quality evidence supported the internal structure of the UCLA-10, whereas the evidence supporting the construct validity of the UCLA-8 was of low-to-moderate quality .
The UCLA-10 and UCLA-5 demonstrated good convergent validity (AVE ≥ 0.50, CR > 0.70, and AVE < CR); however, in the case of the UCLA-8, the AVE indicated that the latent construct accounted for insufficient variance in the eight items. First, all short-form scales had satisfactory discriminant and concurrent validity. In all three short-form versions, the latent construct explained the variance in the items that contributed to its measurement (AVE) more than the variance they shared with other related constructs, including anxiety, hopelessness, depression, and life satisfaction (MSV and ASV). Second, the three short-form versions were significantly correlated with other indices of psychological well-being in the expected directions. These results supported the findings of other studies regarding the concurrent validity of the short-form scales [23,84,85,87].
More studies are needed to determine the dimensionality and validity of the UCLA-8. In this study, the UCLA-10 and UCLA-5 were identified as unidimensional measures of loneliness with satisfactory reliability and validity. The UCLA-5 might be a useful measure of loneliness when the longer 10-item or 20-item versions are not suitable, and this scale might be especially useful for telephone surveys. A limitation of the five-item version is that it only contains negative items, which makes it difficult to identify when participants exhibit acquiescence bias (i.e., they consistently select only one anchor of a scale). However, the advantages of including both positive and negative items do not necessarily outweigh the disadvantages of potential mistakes by respondents (e.g., accidentally agreeing with a negative item) or potential miscoding by the researcher, for example, failing to reverse score items [71,88]. A limitation of the study was stimulus familiarity, considering that the participants were only exposed to the 20-item version of the UCLA and not to the three separate versions, and this might lead to instrument bias.
Loneliness was described as the “signature” mental health issue of the COVID-19 pandemic. The reliable and valid assessment of loneliness should be prioritized. To avoid questionnaire burden and instrument fatigue, as well as prevent respondents from answering randomly to items due to the length of the questionnaire, short forms of the instrument to measure loneliness, such as the UCLA-10, should be investigated. A short-form instrument should have satisfactory reliability and validity that are similar to those of the original 20-item version. The classical test theory and the Mokken analysis provided satisfactory support for the reliability and validity of the UCLA-10 and UCLA-5 when used in South Africa. The UCLA-8 demonstrated satisfactory reliability regarding internal consistency; however, the validity was unsatisfactory. This was the first application of the UCLA-5, and further studies with more participants from different populations are needed to confirm the reliability, validity, and usability of this shorter version of the UCLA-20. The reliability and validity indices of the five-item version were satisfactory and comparable to those of the 10-item version in this study. The complementary evidence showing that the UCLA-5 had satisfactory psychometric properties from two different perspectives, including the classical test theory and the item response theory, provided further support for using the UCLA-5 as a suitable short-form version of the UCLA-20.
Conceptualization and methodology: Anita Padmanabhanunni & Tyrone Pretorius; Analysis and data curation: Tyrone Pretorius. All authors discussed and interpreted obtained results. All authors contributed to the writing and editing of the manuscript over several iterations.
The authors have declared that no competing interests exist.
- Killgore WDS, Cloonan SA, Taylor EC, Dailey NS. Loneliness: A signature mental health concern in the era of COVID-19. Psychiatry Res. 2020; 290: 113117. [CrossRef]
- Palgi Y, Shrira A, Ring L, Bodner E, Avidor S, Bergman Y, et al. The loneliness pandemic: Loneliness and other concomitants of depression, anxiety and their comorbidity during the COVID-19 outbreak. J Affect Disord. 2020; 275: 109-111. [CrossRef]
- Hawkley LC, Cacioppo JT. Loneliness matters: A theoretical and empirical review of consequences and mechanisms. Ann Behav Med. 2010; 40: 218-227. [CrossRef]
- Groarke JM, Berry E, Graham-Wisener L, McKenna-Plumley PE, McGlinchey E, Armour C. Loneliness in the UK during the COVID-19 pandemic: Cross-sectional results from the COVID-19 psychological wellbeing study. PLoS One. 2020; 15: e0239698. [CrossRef]
- Li LZ, Wang S. Prevalence and predictors of general psychiatric disorders and loneliness during COVID-19 in the United Kingdom. Psychiatry Res. 2020; 291: 113267. [CrossRef]
- Al Omari O, Al Sabei S, Al Rawajfah O, Abu Sharour L, Al-Hashmi I, Al Qadire M, et al. Prevalence and predictors of loneliness among youth during the time of COVID-19: A multinational study. J Am Psychiatr Nurses Assoc. 2021. doi:10.1177/10783903211017640. [CrossRef]
- Luchetti M, Lee JH, Aschwanden D, Sesker A, Strickhouser JE, Terracciano A, et al. The trajectory of loneliness in response to COVID-19. Am Psychol. 2020; 75: 897-908. [CrossRef]
- Houghton S, Kyron M, Lawrence D, Hunter SC, Hattie J, Carroll A, et al. Longitudinal trajectories of mental health and loneliness for Australian adolescents with-or-without neurodevelopmental disorders: The impact of COVID-19 school lockdowns. J Child Psychol Psychiatry. 2022. doi:10.1111/jcpp.13579. [CrossRef]
- Ray CD, Shebib SJ. Determinants of loneliness during the COVID-19 pandemic in the United States: A one-year follow-up study. J Soc Pers Relat. 2022. doi:10.1177/02654075221102632. [CrossRef]
- Losada-Baltar A, Martínez-Huertas JÁ, Jiménez-Gonzalo L, Pedroso-Chaparro MdS, Gallego-Alberto L, Fernandes-Pires J, et al. Longitudinal correlates of loneliness and psychological distress during the lockdown situation due to COVID-19. Effects of age and self-perceptions of aging. J Gerontol B. 2022; 77: 652-660. [CrossRef]
- Ernst M, Niederer D, Werner AM, Czaja SJ, Mikton C, Ong AD, et al. Loneliness before and during the COVID-19 pandemic: A systematic review with meta-analysis. Am Psychol. 2022; 77: 660-677. [CrossRef]
- Hajek A, König HH. Prevalence and correlates of loneliness, perceived and objective social isolation during the COVID-19 pandemic. Evidence from a representative survey in Germany. Soc Psychiatry Psychiatr Epidemiol. 2022. doi:10.1007/s00127-022-02295-x. [CrossRef]
- Lin T, Horta M, Heald K, Heemskerk A, Darboh B, Levi A, et al. Loneliness progression among older adults during the early phase of the COVID-19 pandemic in the United States and Canada. J Gerontol B. 2022; 77: e23-e29. [CrossRef]
- Weber M, Schulze L, Bolzenkötter T, Niemeyer H, Renneberg B. Mental health and loneliness in university students during the COVID-19 pandemic in Germany: A longitudinal study. Front Psychiatry. 2022; 13: 848645. [CrossRef]
- Cacioppo JT, Cacioppo S. The growing problem of loneliness. Lancet. 2018; 391: 426. [CrossRef]
- Keller FM, Derksen C, Kötting L, Dahmen A, Lippke S. Distress, loneliness, and mental health during the COVID-19 pandemic: Test of the extension of the evolutionary theory of loneliness. Appl Psychol Health Well Being. 2022. doi:10.1111/aphw.12352. [CrossRef]
- Stickley A, Ueda M. Loneliness in Japan during the COVID-19 pandemic: Prevalence, correlates and association with mental health. Psychiatry Res. 2022; 307: 114318. [CrossRef]
- Russell D, Peplau LA, Ferguson ML. Developing a measure of loneliness. J Pers Assess. 1978; 42: 290-294. [CrossRef]
- Russell D, Peplau LA, Cutrona CE. The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. J Pers Soc Psychol. 1980; 39: 472-480. [CrossRef]
- Russell DW. UCLA Loneliness Scale (Version 3): Reliability, validity, and factor structure. J Pers Assess. 1996; 66: 20-40. [CrossRef]
- Zarei S, Memari AH, Moshayedi P, Shayestehfar M. Validity and reliability of the UCLA Loneliness Scale Version 3 in Farsi. Educ Gerontol. 2016; 42: 49-57. [CrossRef]
- Kwiatkowska MM, Rogoza R, Kwiatkowska K. Analysis of the psychometric properties of the revised UCLA Loneliness Scale in a polish adolescent sample. Curr Issues Pers Psychol. 2018; 6: 164-170. [CrossRef]
- Arimoto A, Tadaka E. Reliability and validity of Japanese versions of the UCLA Loneliness Scale Version 3 for use among mothers with infants and toddlers: A cross-sectional study. BMC Women's Health. 2019; 19: 105. [CrossRef]
- Sancho P, Pinazo‐hernandis S, Donio‐bellegarde M, Tomás JM. Validation of the University of California, Los Angeles Loneliness Scale (Version 3) in Spanish older population: An application of exploratory structural equation modelling. Aust Psychol. 2020; 55: 283-292. [CrossRef]
- Lasgaard M. Reliability and validity of the Danish version of the UCLA Loneliness Scale. Pers Individ Dif. 2007; 42: 1359-1366. [CrossRef]
- Dodeen H. The effects of positively and negatively worded items on the factor structure of the UCLA Loneliness Scale. J Psychoeduc Assess. 2015; 33: 259-267. [CrossRef]
- Hawkley LC, Browne MW, Cacioppo JT. How can I connect with Thee? Let me count the ways. Psychol Sci. 2005; 16: 798-804. [CrossRef]
- Shevlin M, Murphy S, Murphy J. The latent structure of loneliness: Testing competing factor models of the UCLA Loneliness Scale in a large adolescent sample. Assessment. 2015; 22: 208-215. [CrossRef]
- Auné SE, Abal FJP, Attorresi HF. Modeling of the UCLA Loneliness Scale according to the multidimensional item response theory. Curr Psychol. 2022; 41: 1213-1220. [CrossRef]
- Pretorius TB. The applicability of the UCLA Loneliness Scale in South Africa: Factor structure and dimensionality. Afr J Psychol Assess. 2022; 19. doi:10.3390/ijerph19138095. [CrossRef]
- Hughes ME, Waite LJ, Hawkley LC, Cacioppo JT. A short scale for measuring loneliness in large surveys: Results from two population-based studies. Res Aging. 2004; 26: 655-672. [CrossRef]
- Lamm H, Stephan E. Loneliness among German university students: Some correlates. Soc Behav Pers. 1987; 15: 161-164. [CrossRef]
- Neto F. Psychometric analysis of the short-form UCLA Loneliness Scale (ULS-6) in older adults. Eur J Ageing. 2014; 11: 313-319. [CrossRef]
- Oshagan H, Allen RL. Three loneliness scales: An assessment of their measurement properties. J Pers Assess. 1992; 59: 380-409. [CrossRef]
- Hays RD, DiMatteo MR. A short-form measure of loneliness. J Pers Assess. 1987; 51: 69-81. [CrossRef]
- Lee J, Cagle JG. Validating the 11-item revised University of California Los Angeles Scale to assess loneliness among older adults: An evaluation of factor structure and other measurement properties. Am J Geriatr Psychiatry. 2017; 25: 1173-1183. [CrossRef]
- Faustino B, Lopes P, Oliveira J, Campaioli G, Rondinone M, Bomfim H, et al. Psychometric and rash analysis of the UCLA Loneliness Scale-16 in a Portuguese sample of older adults. Psychol Stud. 2019; 64: 140-146. [CrossRef]
- Das A, Padala KP, Crawford CG, Teo A, Mendez DM, Phillips OA, et al. A systematic review of loneliness and social isolation scales used in epidemics and pandemics. Psychiatry Res. 2021; 306: 114217. [CrossRef]
- Padmanabhanunni A. The factor structure of the normative beliefs about aggression scale as used with a sample of adolescents in low socio-economic areas of South Africa. S Afr J Psychol. 2017; 49: 27-38. [CrossRef]
- Pretorius TB, Padmanabhanunni A. Assessing the cognitive component of subjective well-being: Revisiting the satisfaction with life scale with classical test theory and item response theory. Afr J Psychol Assess. 2022; 4: 9. [CrossRef]
- Spielberger CD. Manual for the state-trait anxiety inventory. Palo Alto: Consulting Psychogyists Press; 1970.
- Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: The hopelessness scale. J Consult Clin Psychol. 1974; 42: 861-865. [CrossRef]
- Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Appl Psychol Meas. 1977; 1: 385-401. [CrossRef]
- Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. J Pers Assess. 1985; 49: 71-75. [CrossRef]
- Wongpakaran N, Wongpakaran T, Pinyopornpanish M, Simcharoen S, Suradom C, Varnado P, et al. Development and validation of a 6-item revised UCLA Loneliness Scale (RULS-6) using Rasch analysis. Br J Health Psychol. 2020; 25: 233-256. [CrossRef]
- Pretorius TB. The metric equivalence of the UCLA Loneliness Scale for a sample of South African students. Educ Psychol Meas. 1993; 53: 233-239. [CrossRef]
- Padmanabhanunni A, Pretorius T. The loneliness–life satisfaction relationship: The parallel and serial mediating role of hopelessness, depression and ego-resilience among young adults in South Africa during COVID-19. Int J Environ Res Public Health. 2021; 18: 3613. [CrossRef]
- Knowles KA, Olatunji BO. Specificity of trait anxiety in anxiety and depression: Meta-analysis of the state-trait anxiety inventory. Clin Psychol Rev. 2020; 82: 101928. [CrossRef]
- Wiglusz MS, Landowski J, Cubała WJ. Psychometric properties and diagnostic utility of the State-Trait Anxiety Inventory in epilepsy with and without comorbid anxiety disorder. Epilepsy Behav. 2019; 92: 221-225. [CrossRef]
- Pretorius T, Padmanabhanunni A. A looming mental health pandemic in the time of COVID-19? Role of fortitude in the interrelationship between loneliness, anxiety, and life satisfaction among young adults. S Afr J Psychol. 2021; 51: 256-268. [CrossRef]
- Satorres E, Ros L, Meléndez JC, Serrano JP, Latorre JM, Sales A. Measuring elderly people's quality of life through the Beck Hopelessness Scale: A study with a Spanish sample. Aging Ment Health. 2018; 22: 239-244. [CrossRef]
- Sánchez-Teruel D, Robles-Bello MA, Camacho-Conde JA. Adaptation and psychometric properties in Spanish of the Herth Hope Index in people who have attempted suicide. Psychiatr Q. 2021; 92: 169-175. [CrossRef]
- Aloba O, Awe O, Adelola A, Olatunji P, Aloba T. Psychometric adaptation of the Beck Hopelessness Scale as a self-rated suicide risk screening instrument among Nigerian University students. J Am Psychiatr Nurses Assoc. 2018; 24: 433-443. [CrossRef]
- Heppner PP, Pretorius TB, Wei M, Lee Dg, Wang YW. Examining the generalizability of problem-solving appraisal in Black South Africans. J Couns Psychol. 2002; 49: 484-498. [CrossRef]
- Padmanabhanunni A, Pretorius TB. When coping resources fail: The health-sustaining and moderating role of fortitude in the relationship between COVID-19-related worries and psychological distress. Afr Saf Promot. 2020; 18: 28-47.
- Sharif Nia H, Rezapour M, Allen KA, Pahlevan Sharif S, Jafari A, Torkmandi H, et al. The psychometric properties of the Center for Epidemiological Studies Depression Scale (CES-D) for Iranian cancer patients. Asian Pac J Cancer Prev. 2019; 20: 2803-2809. [CrossRef]
- Chin WY, Choi EPH, Chan KTY, Wong CKH. The psychometric properties of the center for Epidemiologic Studies Depression Scale in Chinese primary care patients: Factor structure, construct validity, reliability, sensitivity and responsiveness. PLoS One. 2015; 10: e0135131. [CrossRef]
- Henkel V, Mergl R, Kohnen R, Allgaier AK, Möller HJ, Hegerl U. Use of brief depression screening tools in primary care: Consideration of heterogeneity in performance in different patient groups. Gen Hosp Psychiatry. 2004; 26: 190-198. [CrossRef]
- Makanjuola VA, Onyeama M, Nuhu FT, Kola L, Gureje O. Validation of short screening tools for common mental disorders in Nigerian general practices. Gen Hosp Psychiatry. 2014; 36: 325-329. [CrossRef]
- Pretorius TB. Cross-cultural application of the Center for Epidemiological Studies Depression Scale: A study of Black South African students. Psychol Rep. 1991; 69: 1179-1185. [CrossRef]
- Padmanabhanunni A, Pretorius TB, Stiegler N, Bouchard JP. A serial model of the interrelationship between perceived vulnerability to disease, fear of COVID-19, and psychological distress among teachers in South Africa. Ann Med psychol. 2022; 180: 23-28. [CrossRef]
- Hinz A, Conrad I, Schroeter ML, Glaesmer H, Brähler E, Zenger M, et al. Psychometric properties of the Satisfaction with Life Scale (SWLS), derived from a large German community sample. Qual Life Res. 2018; 27: 1661-1670. [CrossRef]
- Lorenzo-Seva U, Calderon C, Ferrando PJ, Del Mar Muñoz M, Beato C, Ghanem I, et al. Psychometric properties and factorial analysis of invariance of the Satisfaction with Life Scale (SWLS) in cancer patients. Qual Life Res. 2019; 28: 1255-1264. [CrossRef]
- Bacro F, Coudronnière C, Gaudonville T, Galharret JM, Ferrière S, Florin A, et al. The French adaptation of the Satisfaction with Life Scale (SWLS): Factorial structure, age, gender and time-related invariance in children and adolescents. Eur J Dev Psychol. 2020; 17: 307-316. [CrossRef]
- Paulsen J, BrckaLorenz A. Internal consistency statistics. FSSE Psychometric Portfolio; 2017. Available from: https://scholarworks.iu.edu/dspace/bitstream/handle/2022/24503/FSSE17_Internal_Consistency_Reliability.pdf?sequence=1.
- Hajjar ST. Statistical analysis: Internal-consistency reliability and construct validity. Int J Quant Qual Res Methods. 2018; 6: 27-38.
- Kline RB. Principles and practice of structural equation modeling. New York: Guilford publications; 2015.
- Dueber DM. Bifactor Indices Calculator: A Microsoft Excel-based tool to calculate various indices relevant to bifactor CFA models. Lexington: University of Kentucky; 2017. Available from: https://doi.org/10.13023/edp.tool.01.
- Rodriguez A, Reise SP, Haviland MG. Evaluating bifactor models: Calculating and interpreting statistical indices. Psychol Methods. 2016; 21: 137-150. [CrossRef]
- Rodriguez A, Reise SP, Haviland MG. Applying bifactor statistical indices in the evaluation of psychological measures. J Pers Assess. 2016; 98: 223-237. [CrossRef]
- Reise SP, Scheines R, Widaman KF, Haviland MG. Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educ Psychol Meas. 2012; 73: 5-26. [CrossRef]
- Watson R, van der Ark LA, Lin LC, Fieo R, Deary IJ, Meijer RR. Item response theory: How Mokken scaling can be used in clinical practice. J Clin Nurs. 2012; 21: 2736-2746. [CrossRef]
- van der Ark LA. New developments in Mokken scale analysis in R. J Stat Softw. 2012; 48: 1-27. [CrossRef]
- R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2013. Available from: https://www.R-project.org/.
- Mokken RJ. A theory and procedure of scale analysis: With applications in political research. Berlin, New York: De Gruyter Mouton; 2011.
- Sijtsma K, van der Ark LA. A tutorial on how to do a Mokken scale analysis on your test and questionnaire data. Br J Math Stat Psychol. 2017; 70: 137-158. [CrossRef]
- Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res. 1981; 18: 39-50. [CrossRef]
- Almén N, Lundberg H, Sundin Ö, Jansson B. The reliability and factorial validity of the Swedish version of the Recovery Experience Questionnaire. Nordic Psychology. 2018; 70: 324-333. [CrossRef]
- Manera KE, Smith BJ, Owen KB, Phongsavan P, Lim MH. Psychometric assessment of scales for measuring loneliness and social isolation: An analysis of the household, income and labour dynamics in Australia (HILDA) survey. Health Qual Life Outcomes. 2022; 20: 40. [CrossRef]
- DiTommaso E, Spinner B. The development and initial validation of the Social and Emotional Loneliness Scale for Adults (SELSA). Pers Individ Dif. 1993; 14: 127-134. [CrossRef]
- Schmidt N, Sermat V. Measuring loneliness in different relationships. J Pers Soc Psychol. 1983; 44: 1038-1047. [CrossRef]
- Ebesutani C, Drescher CF, Reise SP, Heiden L, Hight TL, Damon JD, et al. The importance of modeling method effects: Resolving the (uni)dimensionality of the loneliness questionnaire. J Pers Assess. 2012; 94: 186-195. [CrossRef]
- Elphinstone B. Identification of a suitable short‐form of the UCLA‐Loneliness Scale. Aust Psychol. 2018; 53: 107-115. [CrossRef]
- Yildiz MA, Duy B. Adaptation of the short-form of the UCLA Loneliness Scale (ULS-8) to Turkish for the adolescents. Düşünen Adam. 2014; 27: 194-203. [CrossRef]
- Xu S, Qiu D, Hahne J, Zhao M, Hu M. Psychometric properties of the short-form UCLA Loneliness Scale (ULS-8) among Chinese adolescents. Medicine. 2018; 97: e12373. [CrossRef]
- Alsubheen SA, Oliveira A, Habash R, Goldstein R, Brooks D. Systematic review of psychometric properties and cross-cultural adaptation of the University of California and Los Angeles Loneliness Scale in adults. Curr Psychol. 2021. doi:10.1007/s12144-021-02494-w. [CrossRef]
- Wu Ch, Yao G. Psychometric analysis of the short-form UCLA Loneliness Scale (ULS-8) in Taiwanese undergraduate students. Pers Individ Dif. 2008; 44: 1762-1771. [CrossRef]
- Sauro J, Lewis JR. When designing usability questionnaires, does it hurt to be positive? Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 2011 May 7-12; Vancouver, Canada. New York: Association for Computing Machinery. [CrossRef]