It is important to ensure that response rates do not fall significantly over time, especially if they are likely to reduce the usability of survey results (e.g. through the generation of less precise estimates for smaller groups of population). Target response rates could therefore be one possible measure of survey performance.
Graduate Outcomes response rate targets
In 2021 it was agreed (in consultation with the Steering Group) that the response rate target for non-EU overseas graduates will be reduced by 5 percentage points (from 25% to 20%). This was done in response to a Government directive to reduce costs to the sector of receiving data services from HESA. After careful consideration of all possible options it was concluded that the least damaging solution to the problem was the discontinuation of telephone data collection from non-EU overseas graduates. Details about this change and implications were communicated to the sector in November 2021, with the change taking effect from the following month, at the start of the new survey year.
UK domiciled full-time: 60%
UK domiciled part-time: 60%
Research funded: 65%
EU domiciled: 45%
Non-EU domiciled: 20%
These targets are mainly applicable for the entire set of survey results across a year but are also used to monitor performance of individual cohorts. This is to acknowledge the diverse composition of different cohorts in terms of graduate characteristics and how it impacts the response rates that can be realistically achieved.
The ‘headline’ response rates for the Graduate Outcomes survey are defined as:
- Numerator: Count of records with a valid response to a minimum (pre-determined) set of core questions (classified with a status of ‘survey completed’).
- Denominator: Count of all records in the target population
In addition to responses classified as ‘survey completed’, a status of ‘partially completed’ has been assigned where some of the core questions are missing but the first two questions have been answered. Although partially completed responses do not contribute to the ‘headline’ response rates, these are used alongside ‘survey completed’ responses in statistical outputs and form part of the base population for this purpose. Data from such responses will appear in published statistics with unknown values for questions that were not answered.
It should be noted that achieving a high response rate is not sufficient in itself to ensure good quality data has been collected. In later parts of this methodology statement, we will be discussing the other checks and processes we have undertaken in assessing and ensuring the quality of the data
 In the first year we excluded graduates notified to us by providers as being dead or seriously ill from the denominator. These individuals are now included in order to enable our data users to accurately compare the published population totals with data available to them as the information about deceased or seriously ill graduates is only available to HESA and not shared with users, hence leading to misaligned totals.