Numerous reviews had taken place over time to adapt the DLHE and LDLHE to fit changing needs.
The Graduate Outcomes survey marked an extensive evolution of the DLHE/LDLHE approach to meet the longstanding needs outlined above. Through public consultation, HESA observed a very high level of agreement that a census survey was needed, and that its profile should be raised further, especially given the development of the Longitudinal Educational Outcomes (LEO) dataset and its application in higher education. It appeared that users perceived the LEO dataset becoming more important in the future.
Between July 2015 and June 2017, HESA conducted a major review of our destinations and outcomes data, referred to as the NewDLHE review. The review began using a piece of policy landscape analysis referred to as the “remit” for the review. This document identified four key drivers of change, that ought to be addressed through an exploratory review.
Drivers of change
- Future-proofing – a fundamental reconsideration of the kinds of data that will be required for the foreseeable future, taking into account a labour market that is changing at a structural level and increasing demands for rich information about graduate outcomes.
- Efficiency – taking advantage of new capabilities to link data sources and use modern survey technology to increase value for money and reduce the cost of acquisition.
- Fitness for purpose – ensuring the data collection methodology allows the data to be used in new and emerging contexts, with confidence.
- Supporting legislation – by taking into account the legal gateway opened by the Small Business, Enterprise and Employment Act (SBEE Act) and supporting developing government purposes for destinations and outcomes data, across the UK.
The remit proposed a series of research questions that the review was to answer. The review was conducted as a piece of deliberative public engagement, with a transparent approach to discussion, evidence-gathering, and decision-making.
Needs were identified using the following main mechanisms:
- Policy analysis.
- Establishment of an appropriate governance structure and use of HESA’s organisational governance.
- Involvement of key stakeholders in a working group.
- Running events and attending third-party events and meetings to discuss needs and methods with stakeholders.
- Commissioning two separate research reports.
- Undertaking two public consultations and publishing both summaries of responses and detailed findings from each.
- Undertaking an administrative data quality assurance self-assessment exercise in line with the expectations of the Code of Practice for Statistics.
These approaches led to findings that were synthesised into the publication of a final model.
A full archive of the review materials is available on the HESA website: NewDLHE: Destinations and outcomes review.
Accompanying the model, we published a rationale for the decisions that we made during the review. The rationale offered our answers to the review questions set in the remit document and provided the reasoning behind design decisions, such as the development of the graduate voice questions and the decision to centralise the survey.
Requirements for the survey outputs were principally to contextualise data from new sources such as LEO, which have very complete data on a small number of variables, but lack the rich breadth of data in the Graduate Outcomes survey. Many of the outputs developed from the DLHE data were seen to have value, and so our approach to designing outputs was to iterate from the DLHE approach, making improvements where richer data allows, and to follow established standards where possible.
The model developed was the business case for the ‘design’ stage to take place, which would then go on to develop the model into a fully-featured design for a new survey. The detailed design stage challenged some aspects of the model and caused some elements to be modified or delayed. These are detailed later on in this report.