Skip to main content

Evaluation

On this page: Data collection | Data quality analysis | Data processing and dissemination

Continuous evaluation and improvement are among the main features of the design and delivery of Graduate Outcomes, as evidenced throughout this methodology statement and in our regular communications with providers. We are keen to build on the foundations from the first year and enhance our data collection, processing and dissemination systems as we move into the next year and beyond. This is partly being achieved through a series of post-implementation reviews with our key stakeholders, including suppliers.

The following are a number of potential survey enhancements that HESA will consider for future collections of the survey. These will be described once they have been fully explored in future iterations of this methodology statement:

Data collection

  • Understand what encourages individuals to engage with emails and SMS messages and use it to inform the content of our communication / graduate engagement materials.
  • Enhance the survey experience for users of mobile devices, considering recent technological developments and increasing use of such devices.
  • Explore the use of web survey design features such as progress bars and information buttons.
  • Explore the use of a single number for telephone interviews versus geo-referenced phone numbers to increase uptake of telephone calls.
  • Assess the costs and benefits of various approaches to incentives.
  • Investigate and make recommendations for closer alignment with UK and international labour market data standards.

Data quality analysis

  • Measure the quality of graduate contact details supplied by providers and its impact on response rates.
  • Identify factors contributing to variability in response rates and, where possible, determine ways of reducing it.
  • Determine the extent of measurement error introduced by questionnaire design, mode effects, interviewer bias and respondent bias.
  • Identify ways of improving the quality of free text responses.
  • Obtain access to raw LEO data, in order to quality assure collected salary data, and to contribute to our understanding of non-response bias in the survey.

Data processing and dissemination

  • Solicit feedback from users on our first Statistical Bulletin and Open Data release, to help HESA refine and develop our outputs.
  • Undertake an investigation into linking Graduate Outcomes data to subsequent years of the HESA student record, in order to quality assure further study outcomes data, and enhance our understanding of undergraduate to postgraduate transitions.
  • Undertake further research into weighting methodologies incorporating the second year of survey results (18/19), when available, to ascertain whether more nuanced approaches can be identified that improve quality of statistics derived from the survey.

Previous: Sector engagement     Return to Graduate Outcomes open data homepage