Skip to main content

Relevance

When considered as a dimension of statistical quality, relevance refers to the extent to which statistical outputs meet the current and potential needs of users. In order to assess relevance, it is necessary first to identify likely users of the data and their needs. The data sources and statistical concepts used in the production of a statistical output are also a factor in determining relevance; depending on user needs, different data sources and classification schemes will be appropriate. Finally, it is important to identify any gaps between the statistical output and known user needs and to assess how those gaps may be filled in future.

Users and user needs

A wide variety of users, in the HE sector and beyond, work with HESA data on graduates. HESA has obligations to a range of statutory customers in all four UK nations, including the funding and regulatory bodies for higher education in each nation; our statutory obligations to these customers require us to provide them with the data which they need to carry out their public functions[1]. As Designated Data Body for England, HESA is further required by law to publish ‘appropriate information relating to registered HE providers and the higher education courses they provide’. According to the Higher Education and Research Act 2017 (HERA), the category of ‘appropriate information’ includes information which may be helpful to students in higher education, potential higher education students, and HE providers; HERA also specifies that the designated data body must provide appropriate information to the Office for Students (OfS), UK Research and Innovation (UKRI), and the Secretary of State for Education[2]. In addition to those users whose needs we are required by law to consider, we also wish to consider the needs of others for whom high quality data on graduates will be useful, including HE funding and regulatory bodies, local and national governmental agencies, graduate employers, and academic researchers[3].

Different users have different needs for the Graduate Outcomes data. Prospective HE students may look to Graduate Outcomes in order to make informed choices about providers and courses, while HE providers may use the data for strategic planning purposes. Funding and regulatory bodies may use Graduate Outcomes data to assess the performance of providers and courses, while government agencies—both local and national—and graduate employers may look to the data to provide information both about the regional supply of graduates with different skills and about the roles played by graduates in society more generally. Since the publication of the first statistical outputs based on the survey, HESA has tracked citations of Graduate Outcomes data, and seen it used a variety of publications, from stories on higher education in the national media, to publications designed to support student choice, to analyses conducted by funding and regulatory bodies[4].

Throughout the design and implementation phases of the Graduate Outcomes survey, HESA has been engaging with the various potential users of the survey data, actively. In the early stages of the NewDLHE review, a Strategic Group and a Working Group were convened; these groups, which were comprised of representatives from a wide variety of HE providers and other sector bodies, were responsible for setting the remit for the review and developing a workplan to pursue this remit[5]. Later in the review, HESA carried out two consultations, the first to determine user needs for the successor to DLHE, and the second to solicit feedback on the draft model for the new survey[6].

Responses to the second consultation suggested a high level of stakeholder approval for the proposed model, giving HESA a mandate from potential survey users to proceed with the implementation of the new survey. The model proposed in the second consultation called for the establishment of the Graduate Outcomes Steering Group; this group, like the earlier Strategic and Working Groups, is designed to reflect the diversity of stakeholders for the Graduate Outcomes survey and is comprised of representatives from HE providers and HESA statutory customers from across the UK. The Steering Group met quarterly during the development and implementation of the survey to advise HESA on all aspects the Graduate Outcomes survey. HESA values the expertise and input which has so far been contributed by the Graduate Outcomes Steering Group, and it is envisaged that the group will continue to operate in an oversight capacity and to help guide further improvements to the survey[7].

In addition to the regular meetings of the Graduate Outcomes Steering Group, HESA continues to solicit feedback from the sector on particular issues. While the charts and tables to be included in the first Graduate Outcomes Statistical Bulletin and open data release were being developed from conceptual designs into logical wireframes, HESA convened a group of sector representatives to ascertain whether the planned outputs met with user needs. This engagement with stakeholders informed the initial publication of Graduate Outcomes data, and has continued to guide our decision-making process as we prepared for the second year of outputs. for the second year of outputs[8]. Over the course of the second year of surveying, HESA has invited key users to participate in a review of the survey questionnaire, and has also continued to consider user feedback about the survey and its associated outputs submitted to the Agency via other channels[9].

Next: Data and statistical concepts


[1] A list of the statutory customers who require data from HESA can be found at https://www.hesa.ac.uk/about/what-we-do/statutory-customers

[2] Higher Education and Research Act 2017, sections 64 and 65. http://www.legislation.gov.uk/ukpga/2017/29/section/65/enacted

[3] A list of likely users of graduate outcomes data, based on known users of the DLHE survey, can be found in the Graduate Outcomes Survey methodology: https://www.hesa.ac.uk/data-and-analysis/graduates/methodology/understanding-outcomes

[4] See, for example, a recent BBC article on graduates leaving Wales (https://www.bbc.co.uk/news/uk-wales-57528943), the 2020/21 ’What do graduates do?’ report from Prospects (https://luminate.prospects.ac.uk/what-do-graduates-do), and the experimental ’Proceed’ measure published by the Office for Students in May 2021 (https://www.officeforstudents.org.uk/publications/proceed-updated-methodology-and-results/).

[5] The NewDLHE review was a major review of HESA’s destinations and outcomes data which ran from July 2015 to June 2017; ‘NewDLHE’ was the working title for replacement for DLHE, which has since become the Graduate Outcomes survey. For a complete record of the review, see https://www.hesa.ac.uk/innovation/records/reviews/newdlhe
Further detailed information on the NewDLHE Working Group and Steering Group can be found on the HESA website: https://www.hesa.ac.uk/innovation/records/reviews/newdlhe/working-group
https://www.hesa.ac.uk/innovation/records/reviews/newdlhe/strategic-group
 

[7] Further information on the remit and composition of the Graduate Outcomes Steering Group: https://www.hesa.ac.uk/innovation/outcomes/about/steering-group

[8] HESA. 2020. How to publish Graduate Outcomes data? Our consultation on open data release. https://www.hesa.ac.uk/blog/15-04-2020/how-publish-graduate-outcomes-data

[9] Users can submit feedback to the Official Statistics, Liaison, and Communications teams; they can also provide feedback on the HESA website, or direct queries to Jisc, HESA‘s data analytics partner.