Quality methodology

HESA builds the Code of Practice for Statistics into all aspects of its work. At HESA, quality management is an overarching practice that is prioritised in each part of the statistical business process. We operate appropriate quality regimes for each aspect of our work, and although delayed by the pandemic, we are committed to bringing these practices together in a single overarching quality policy. For this quality report, we have taken the following approach.

First, we base our approach on the guidance offered by the National Statistician on survey quality measurement, by structuring our report around the five dimensions of quality outlined in the European Statistical System[1].

Second, we have already created a range of supporting materials, now organised into a single user guide, including a Survey methodology[2], which covers our dissemination policy for Graduate Outcomes. These materials are cross-referenced as needed in this quality report, as they form part of the evidence base for it. Sometimes, for ease of reading, there will be some repetition between this report and others we have published, though we have attempted to keep this to a minimum. Our stated intention was to bring these several resources together in a single user guide for the Graduate Outcomes survey; a goal endorsed by the Office for Statistics Regulation (OSR) in their assessment of the first year of Graduate Outcomes against the Code of Practice for Statistics[3]. We are therefore pleased to present this third edition of the quality report as an integrated section of the new user guide. We are keen to get users’ feedback on the user guide, as we expect to develop the approach further in coming years.

Third, our aim and purpose in writing this report is to offer the most up-to-date assessment of the quality characteristics of the Graduate Outcomes survey. In doing so, we have necessarily prioritised our own uses and outputs first, as these take into account the many user requirements we have already elicited. However, at this relatively early stage in the Graduate Outcomes survey’s development, a quality report cannot be as comprehensive as one that follows a period of extensive usage by other users. Notably, our own initial uses are mainly for the release of aggregated data, which is filterable by multiple characteristics, but ultimately still a summary of findings. We encourage users of survey microdata to carry out and publish their own quality assessments, especially in areas where our own work does not provide them with the understanding they need to have confidence in the validity of their analysis. This approach will extend and enhance our own work, for the benefit of all users.

Fourth, although this is a technical report about statistics, it follows a narrative format. Our assessments and evaluations of quality characteristics are presented using a predominantly narrative approach, with tabular information included as static data tables to illustrate our findings.

Next: Quality description