Adding value to UK graduate labour market statistics: The creation of a non-financial composite measure of job quality - Section 3: Methods

Section 3: Methods

The development of our composite measure comprises of three stages. The first of these relates to exactly how we treat the three indicators relating to the extent to which an individual believes their work is meaningful, aligns with their future plans and utilises their skills. Despite measures of wellbeing often being ordinal in nature, it is often the case that researchers will treat the data as being continuous with equal distance between the values (Blanchflower and Oswald 2011; Dynan and Ravina 2007; Hetschko et al. 2014). A question that merits further consideration at this point is whether such an assumption can be justified. Van Praag (1991) discusses an experiment carried out in a context-free setting in which participants were asked to assign numbers to five verbal labels (very bad, bad, not bad/not good, good, very good). As Kaiser and Vendrik (2022) highlight, the results suggest that respondents treat such labels on a linear scale with Van Praag (1991) stating that such verbal label sequences can be transformed into a numerical format. We therefore convert our three variables accordingly such that they range from 1 to 5 (1 – strongly disagree, 2 – disagree, 3 – neither agree nor disagree, 4 – agree, 5 – strongly agree). In line with the wellbeing literature, our three indicators are presumed to be types of interval (i.e. continuous) data.

The second phase of work assesses the degree to which the statements are all part of the same underlying (latent) concept, which in this instance is the design and nature of the job carried out by the individual. This can be examined by calculating Cronbach’s alpha, which lies between 0 and 1. A statistic closer to 1 indicates greater shared covariance between the items. We determine this value for our overall sample and by academic year to evaluate the robustness of the results. Among the entire sample, alpha is found to equal 0.83 with little variation found if the analysis is run separately for those qualifying in 2017/18 (0.82) and 2018/19 (0.83). Acock (2013) suggests that an alpha above 0.70 implies that the variables are likely to be part of the same underlying concept. Consequently, we conclude that the three statements all sit within the same component. 

The final element of our exploration evaluates whether these three statements can be reduced into a single dimension. As we are assuming the three variables are continuous in nature and that any composite measure is also of this form, the most appropriate reduction technique to implement would be factor analysis. There are a number of potential extraction methods one can apply when using this approach. As Fabrigar et al. (1999) point out, in instances where the distribution of the variables is not normal, it may be preferable to apply a principal component factor analysis (PCFA) as opposed to utilising the maximum likelihood (ML) approach. With our data exhibiting distributions that are not normal, we begin by utilising the PCFA approach, before carrying out a sensitivity analysis using ML. It is often the case that different extraction methods will produce similar results (Tabachnick and Fidell 2019) and we find that to be true in our investigation too. For the overall sample across the two years, we find the factor loadings from running a PCFA (i.e. the correlation between the statement and the latent dimension) to be high with a range from 0.84 to 0.89 (similar results emerge if we look at each academic year in turn). The first factor has an eigenvalue of 2.25 and thus accounts for 75% of the overall variance, while the second and third factors have eigenvalues of 0.44 and 0.31 respectively. These values suggest that the three items can be reduced into a single dimension. The MJQWG argue that the design and nature of work is one component of job quality comprising of aspects such as skills utilisation, progression opportunities and whether employment provides a sense of purpose. Our empirical findings appear to support this theoretical formulation.

In conclusion to this section, we consider exactly how the composite measure should be constructed. Acock (2013) remarks that in instances where the factor loadings are all very similar, there will be little difference in a composite measure formed from PCFA to that composed by taking a mean of the three items. Indeed, we find the correlation between two measures generated in this way to be 0.999. Additionally, deriving a composite variable using ML shows a similarly high association with a measure produced by either taking an average or using PCFA. Given the interest in utilising the created variable in the public domain, our preferred approach is to use a composite measure that is based on the mean value from the three statements and thus ranges from 1 to 5.

Next: Section 4: Exploring associations with subjective wellbeing and ‘highly skilled’ employment