Accessibility and clarity

In this section, we discuss issues of accessibility and clarity relating to the Graduate Outcomes dataset and the statistical outputs which are based upon it. In assessing statistical quality, accessibility refers to the ease with which users are able to obtain the data, including the format or formats in which the data is available and any supporting information which may be needed. Clarity refers to the availability and comprehensibility of any metadata which its users may need to understand the statistical data fully.

Confidentiality and disclosure control

Given that the Graduate Outcomes survey requires the collection of contact details and other personal information about respondents, issues of data protection, confidentiality, and disclosure control have been important throughout the design and implementation phases of the survey.

HESA receives contact details for most graduates from providers.[1] Students are informed that their contact details will be passed on to HESA via HESA’s Student Collection Notice, which informs students that, after graduation, providers will pass graduate contact details on to HESA and any organisations contracted by HESA to enable the collection of Graduate Outcomes data. The Student Collection Notice further informs students of the legal basis for the processing of their contact details for use in Graduate Outcomes, stating that contact details obtained from providers will be processed by HESA on the grounds that such processing is necessary for the performance of a task carried out in the public interest and for research and statistical purposes.[2]

Data protection policies and privacy notices available both on the HESA website and on the separate Graduate Outcomes website inform both providers and respondents of the uses which will be made of graduate data. Providers and respondents are informed that graduates may opt out of completing the survey, but that responses to the survey will be processed in accordance with GDPR on the basis of public interest, not consent.[3] Respondents are informed that their survey responses will be passed on to their HE provider, but that, unless they explicitly agree to be contacted by their provider about their survey responses, providers will only use survey responses for statistical and research purposes; while providers receive SWB data, they do not receive SWB responses for individual graduates, but instead receive aggregated statistical information about all their graduates’ responses. Similarly, while Graduate Outcomes responses are passed on to a variety of other public and private bodies (including HE funding and regulatory bodies, public authorities, and others who have a legitimate interest in using the data for research and statistical purposes), survey responses are not used to make decisions about individuals. Where Graduate Outcomes data is passed on to third parties for use in research about higher education and the student population, the data is supplied under contracts which ensure that individuals cannot be identified from the data.[4]

When Graduate Outcomes data – or any other HESA data about people – is used in statistics published by HESA or any other users of HESA data, the data is subject to HESA’s rounding and suppression strategy, which aims to reduce the risk of identifying individuals from published statistics. There are three main aspects to HESA’s rounding strategy, each of which contributes to the protection of individual data: first, all counts of people are rounded to the nearest multiple of five; second, percentages based on fewer than 22.5 people are suppressed, and, third, averages are not published if they are based on seven or fewer individuals. Rounding counts of people prevents the use of multiple tables to identify small numbers of individuals, while the suppression of percentages based on fewer than 22.5 individuals and averages based on fewer than seven individuals prevents users from working back from an average or a percentage in order to obtain individual data.[5]

HESA’s rounding strategy is designed to protect personal data, while still enabling HESA and other users of HESA data concerning individuals to publish useful statistics. In this vein, to prevent the compounding of inaccuracy which would occur if calculations were based on rounded figures, the rounding strategy is applied to the data only after any calculations have been carried out. Likewise, the specific thresholds applied in the rounding strategy represent an attempt to strike a balance between disclosure control and the production of detailed statistics; while rounding to multiples of 50, for example, would make it even harder to identify individuals, such a strategy would reduce the usefulness of the statistics which could be published.

Statistical products and supporting information

As has been discussed in the Production timeline and Frequency of production sections, data for the fourth year of the Graduate Outcomes survey was published in May 2023, with subsequent data releases occurring annually in late spring or early summer. Like other HESA statistical releases, Graduate Outcomes data is not subject to scheduled revision; revisions to statistical releases are only carried out in the event of errors in HESA’s data collection and production processes.[6]

Historically HESA produced two main statistical outputs based on the Graduate Outcomes data. The first, a Statistical Bulletin, which contains a range of tables, charts, and summary analysis of headline figures drawn from the data and the second, a release of open data, published about a week after the Statistical Bulletin, containing a wider range of tables and charts, including provider-level for some variables. Both the Statistical Bulletin and the Open Data are available for free on the HESA website, and each chart is accompanied by a freely available data download, allowing users to conduct their own analysis of the data.[7] From 2022, HESA produces an annual single combined release of Graduate Outcomes. This combined release consists of summary statistics and detailed information, including by provider, in the Graduate Outcomes open data repository.

In addition to the Graduate Outcomes data release, HESA has also published a variety of outputs, including this quality report, designed to help users understand the Graduate Outcomes survey and the statistical outputs derived from it. In March 2020, HESA published a Survey methodology concerning the Graduate Outcomes survey, along with an accompanying blog post explaining the main points covered in each part.[8] The Survey methodology outlines the predecessors to the Graduate Outcomes survey, DLHE and LDLHE, the need for a new survey, and the process by which the new survey was developed. It then goes on to discuss in detail the most important aspects of the design and implementation of the survey, with sections on survey coverage, survey design, data collection, data processing and analysis, data dissemination, sector engagement, and the evaluation of the survey.[9] The Survey methodology has been updated to reflect changes which have taken place since the initial publication; the revised version of the Survey methodology was released alongside the 2018/19 data. In May 2020, HESA published a dissemination policy for the Graduate Outcomes survey, setting out HESA’s policy, approaches, and standards for the dissemination of Graduate Outcomes data; for 2022 an updated version of this document was integrated into the Survey methodology, and was published along with the year three data. The dissemination section of the Survey methodology 4 (which applies both to HESA’s publications and to those which may be produced by other users) includes sections on key users and uses of the data, legal and ethical considerations, and HESA’s policy on misrepresentation of data; it also contains sections on HESA’s statistical outputs based on the survey and supporting information for users of the data.[10] Supplementary information on our approach to data concepts and standards can be found in the following section of this report on coherence and comparability.

HESA also makes a range of other metadata available to users of the survey. The Graduate Outcomes section of the HESA website includes general information about the project and the survey, a link to the information page for students and graduates, a link to the information page for providers, and links to the Graduate Outcomes coding manuals; the survey results coding manual contains a variety of detailed metadata, including information on survey coverage, survey routing, and the variables used in the dataset.[11] The information page for providers includes a variety of resources, including detailed operational survey information; the operational survey information page includes detail on how the survey is being carried out, as well answers to FAQs about survey operation, response rates, and the delivery of data to providers.[12] In April 2021, HESA published a summary of the work done to quality assure the SOC coding of year two data; alongside that summary report, HESA also published a separate report on the results of an independent verification exercise in which SOC codes returned by the primary coder for Graduate Outcomes were compared with those returned by the Office for National Statistics.[13]

Further information about HESA’s data can also be found on the ‘Definitions and data standards’ page of the HESA website. This page includes a glossary which defines terms and acronyms frequently used in HESA outputs; information about the coding of subjects, disciplines, industries, and occupations; data intelligence notes which describe specific issues in the HESA data; and lists of definitions relevant to each HESA data stream.[14] The ‘Definitions and data standards’ page also includes answers to a number of FAQs which are relevant to multiple HESA collections, including a specific page covering Graduate Outcomes.

To help users navigate the range of supporting materials available, HESA has developed a single user guide bringing together all the materials described above, as well as this quality report. From the second year of Graduate Outcomes publication, this user guide has taken the form of a detailed table of contents, which will inform users about the various resources available to them and what is contained in each of those resources. In subsequent years, we aim to act on user feedback on the presentation of supporting documentation, and we will adapt the user guide as necessary to meet user needs.

Access and use

The Graduate Outcomes data release is freely available and downloadable on the HESA website under a Creative Commons Attribution 4.0 (CC BY 4.0) license.[15] Users of the data are free to copy, use, share, or adapt it for any purpose, provided that they give appropriate credit to HESA, provide a link to the Creative Commons license, and indicate if any changes have been made to the data.[16]

Since Graduate Outcomes data is freely available for public use, HESA cannot be responsible for the uses made of its data by external parties; HESA neither has the resources to police external uses of its data nor desires to be an arbiter of truth in the domains in which it publishes data. At the same time, HESA is aware that use of its data to support invalid conclusions or interpretations could entail a risk to the perceived trustworthiness, quality, and value of HESA’s statistical outputs. With this risk in mind, the dissemination section of the Graduate Outcomes methodology includes HESA’s policy on potential misrepresentations of the Graduate Outcomes data, outlining the steps which HESA may take if a factual misrepresentation is perceived to have taken place.[17]

In addition to the Graduate Outcomes data which is available on the HESA website as open data, other datasets relating to the Graduate Outcomes survey are available to certain categories of users. HESA’s statutory customers receive quality-assured microdata covering HE providers in their constituencies and a range of data fields aligned with their statutory powers and public functions; individual providers also receive microdata for their own graduates on an individual basis, except for the SWB data, which is released to providers only in aggregated form.

Tailored datasets are also available for users who have data needs which are not met by the Graduate Outcomes open data. Datasets are provided under licence for a fee and can be commissioned through the Jisc Tailored Datasets service.[18] Graduate Outcomes data will be available for use in tailored datasets as soon as possible after the release of the open data.[19]

Further information about Graduate Outcomes data and publications is available from HESA’s Official Statistics team ([email protected] or (0)1242 388 513 [option 2]).

Next: Coherence and comparability