Skip to main content

"The true method of knowledge is experiment" - why Graduate Outcomes statistics are experimental

As HESA is preparing the first set of Graduate Outcomes data for publication from late April, we are sharing some key points about the nature of the statistics being released. 

HESA has collected data on the destinations of leavers from higher education, and released statistical bulletins under National Statistics designation for just under 20 years. However, Graduate Outcomes is a very different data collection, so the way we’re classifying the statistical releases under the banner of official statistics is going to be different too.

Very early on, we received guidance from the Office for Statistics Regulation (who provide independent regulation of official statistics produced in the UK) that the National Statistics designation of our previous DLHE-based bulletin should not simply be carried forward to our Graduate Outcomes bulletin due to the fundamental changes in the survey. Having received this guidance, we decided to release all Graduate Outcomes data outputs as ‘experimental statistics’ for this year. 

What are experimental statistics?

Experimental statistics are an existing class or subset of official statistics and are defined as newly developed or innovative official statistics undergoing evaluation. They are published with the aim of involving users and stakeholders in the assessment of their suitability and quality. Users should exercise caution when using data from experimental statistics, and evaluate the quality and coverage of any data they intend to use in the context of the intended application to ensure that it is fit for the user’s purpose.

As this is the first year that the Graduate Outcomes data has been collected, with methodologies and outputs still under review and evaluation, we have made a collective judgement in labelling the resulting data releases as experimental. 

For HESA, one of our key strategic objectives is to ensure our data is open and accessible. Ensuring that all of our outputs and statistics are suitable for our users is key to this. We’ll continue to explore this as we understand more about the quality of the data, and review our survey methods and approaches.

How do they become National Statistics?

Experimental statistics are, by definition, also official statistics. This label is temporary and can be removed when HESA believes it is appropriate to do so. However, experimental statistics cannot be classed as National Statistics simply through the decision to remove the label of experimental statistics. The only way for a data release to become a National Statistic is for it to undergo a full assessment against the Code of Practice for Statistics by the Office for Statistics Regulation. National Statistics status tells users that the statistics fully comply with the Code and meet the highest standards of trustworthiness, quality and value.

We are hoping to work with the Office for Statistics Regulation over the summer to assess whether our Graduate Outcomes statistical bulletin complies fully with the Code of Practice, opening the possibility that future editions could be re-designated as National Statistics.

Do experimental statistics come with any health warnings?

We have made the decision to label our Graduate Outcomes data releases as experimental specifically to highlight the fact that it is derived using data from a new survey. Some aspects of the methodology are still under evaluation and development, and data quality continues to be explored. Additionally as the statistical products themselves are new, we will wish to engage with users on their structure, content and functionality to reach a point where we can demonstrate that they are meeting user needs.

One very important point here is that the experimental statistics designation does not imply that the statistics or source data are of poor quality, nor does it provide any signal to suggest that the data is not fit for any operational purposes. 

Does this label affect the way data users should make use of the statistics?

Data users should pay close attention to the quality information that HESA will provide alongside the statistics and assess fitness for purpose in the context of their intended uses. As this data is completely new and not directly comparable to DLHE, it is worth noting that users will not have a time-series of data (a series of comparable data from previous years) at their disposal until future years of Graduate Outcomes data become available. 

Stay up to date

You can keep up to date on the timings for Graduate Outcomes releases via our publication schedule - we will provide specific dates no later than four weeks before each release. 

We’re always keen to hear from potential users of these new data outputs, as well as existing users of our other data outputs. Contact us on [email protected] or call us on 01242 211 494 (01242 211 120 for press enquiries).

If you would like to receive email alerts when HESA publishes new data and official statistics about UK higher education, please sign up here.

Rebecca Mantle

Rebecca Mantle

Head of Official Statistics