Dissemination - Comparisons with DLHE
The former Destinations of Leavers (DLHE) survey was the predecessor to the Graduate Outcomes survey. It operated for many years and, subject to some changes in content and coding frames, provided a good time-series for data users over the period of operation. The Graduate Outcomes survey replaces it and the review process that led to the formation of Graduate Outcomes, together with the key differences in the two surveys are explained in detail in the section of the Survey methodology dealing with its history and background.
It is very important that users of data from the Graduate Outcomes survey understand that it is a fundamentally different survey from DLHE, since this has significant implications for those who wish to compare data from the surveys and create time-series analysis.
Jisc has taken the decision that we will not undertake, publish or otherwise disseminate any direct comparisons of data between the Graduate Outcomes survey and the DLHE survey. This is due to the fact that the data from the two surveys are not directly comparable.
Jisc advises all data users against attempting to directly compare data between Graduate Outcomes and DLHE. Any such comparisons are likely to generate highly questionable results that are open to misinterpretation.
The two surveys represent a discontinuity in time-series which, although inconvenient for data users, is necessary for the reasons explained in the Survey methodology.
The key survey differences which explain the lack of comparability are:
- Timeframe change - the census point in Graduate Outcomes is at approximately 15 months after gaining qualifications whereas for DLHE the census points were at 6 months and 3.5 years (for the follow-up sample). Graduates surveyed are therefore at fundamentally different stages in their progression after gaining their qualification(s).
- Centralisation - the contact centre, fieldwork and coding of Standard Industrial Classification (SIC) and Standard Occupational Classification (SOC) are undertaken centrally, no longer by the provider, allowing for greater consistency.
- The survey instruments (questionnaires) used for DLHE and Graduate Outcomes are different by design. The questions established for Graduate Outcomes were designed to address similar themes to DLHE but in many cases are worded slightly differently or cover more detail. An example is that there is much more detail on graduates who are self-employed or running their own business in Graduate Outcomes than was the case in DLHE, with greater insight into the inter-relationships between self-employment and other activities. Graduate Outcomes questions are designed to provide information about graduates’ experiences that is relevant in the current graduate labour market, as compared with DLHE questions which were designed to address the labour market and associated data needs many years ago. Lack of direct comparability between the surveys was accepted as necessary in order to achieve this, and information requirements for Graduate Outcomes were defined independently of DLHE. Comparability of graduates’ perceptions and responses to questions that are worded differently between DLHE and Graduate Outcomes were not cognitively tested because this was not a design criterion for Graduate Outcomes. Data users therefore cannot assume that graduates’ responses to such questions are comparable between the surveys.
In addition to topics covered in the DLHE survey, Graduate Outcomes includes a range of new questions such as ‘Graduate Voice’ questions which provide a novel understanding of success in outcomes as judged by graduates themselves. There are also new questions to which shed light on the work and study pathways graduates may have taken between gaining their original qualifications and the survey reference week. Banks of opt-in questions give providers the opportunity to ask graduates additional questions.