Skip to main content

Evaluation

On this page: In progress | To be explored

Continuous evaluation and improvement are among the main features of the design and delivery of Graduate Outcomes, as evidenced throughout this methodology statement and in our regular communications with providers. We are keen to build on the foundations from the first two years and enhance our data collection, processing and dissemination systems as we move into the next year and beyond. To this effect we launched a review of Graduate Outcomes at the end of 2020. Changes emerging from this review will affect the processes in years three and four and will be communicated alongside the corresponding statistical releases for these years.

The following are a number of survey enhancements that are either in progress or will be considered for future collections of the survey. These will be described once they have been fully explored in future iterations of this methodology statement:

In progress

  • Enhance the survey experience for users of mobile devices, considering recent technological developments and increasing use of such devices.
  • Explore the use of web survey design features such as progress bars and information buttons.
  • Investigate and make recommendations for closer alignment with UK and international labour market data standards.
  • Measure the quality of graduate contact details supplied by providers and its impact on response rates.
  • Identify factors contributing to variability in response rates and, where possible, determine ways of reducing it.
  • Determine the extent of measurement error introduced by questionnaire design, mode effects, interviewer bias and respondent bias.
  • Identify ways of improving the quality of free text responses.
  • Undertake an investigation into linking Graduate Outcomes data to subsequent years of the HESA student record, in order to quality assure further study outcomes data, and enhance our understanding of undergraduate to postgraduate transitions.
  • Solicit feedback from users on our first Statistical Bulletin and Open Data release, to help HESA refine and develop our outputs.

To be explored

  • Understand what encourages individuals to engage with emails and SMS messages and use it to inform the content of our communication / graduate engagement materials.
  • Assess the costs and benefits of various approaches to incentives.
  • Obtain access to raw LEO data, in order to quality assure collected salary data, and to contribute to our understanding of non-response bias in the survey.

Previous: Sector engagement     Return to Graduate Outcomes user guide