Skip to main content

Graduate Outcomes statistical releases - detailed questions and answers

As part of our approach to the dissemination of 17/18 Graduate Outcomes statistical outputs, we delivered a series of webinars about the contents of the releases. This page shares a recording of one of the webinars and provides answers to detailed questions raised.

Webinar recording

One of the webinars was recorded and this video provides a comprehensive guide to the Statistical Bulletin and its contents. There are various sections of the webinar, but should you wish to skip to the live tour of the Bulletin, it starts at 20:30 and the question and answer session at 37:31.

Questions and answers

The following questions and answers represent the main topics covered across the three statistical release webinars.

Questions about the Statistical Bulletin

Can you clarify which type of graduates this data relates to? Is it all of the students in the survey or a subset? Are they UK full time undergraduate or graduates?

We include all graduates who either fully completed or partially completed the survey; these graduates have known outcomes. In addition, some tables include counts of non-respondents by personal or course characteristics. Some of the tables and charts within the release will relate to a specific restricted population but this will be clear from the titles on each table or chart.

Will there be detailed definitions of what is included in each figure or table, for example, what fields have been used and what filters are applied?

The filters are labelled in a consistent way across the tables and charts and they will be consistent with concepts like mode of former study or level of qualification. We have also published definitions and expandable question boxes that explain more about what's included in particular filters.

In response to feedback, we have now published additional supporting information (Excel).

The Excel file specifies the data fields used within tables and charts in the releases. This includes the field names and restrictions applied to base tables from which the tables and charts were created and information about additional derivations we have applied to the data for the purpose of the data releases. Each base table references a list of tables and charts to which they relate.

We believe that the breadth and depth of material that we have published about Graduate Outcomes surpasses any previous release of data from HESA. We've really tried to be as open and transparent as possible about every aspect of the survey and how we present these statistics. But if users feel that there's further information needed, please get in touch with us at [email protected].

Is Figure 1 based on XACTIVITY?

Yes, we have used XACTIVITY to describe the activities in Figure 1.

Is Figure 7 (Employment basis of graduates working for an employer in the UK by level of qualification obtained) restricted to graduates where work was their most important activity?

Yes, this is restricted to graduates with a most important activity of either paid work for an employer or voluntary work for an employer. The work type marker can filter between the most important activities.

Do Figures 13 and 14 (salary tables) include self-employment/freelancing, running a business and developing a portfolio as paid employment, or just paid work for an employer?

The salary question in the survey is only asked of graduates who are in paid work for an employer or self-employment / freelancing, so it is just these that have been included in Figures 13 and 14.

The salary data in the Statistical Bulletin only includes UK based employment. Are there any plans to include international salaries in the statistical outputs?

There are currently no plans to look at international salaries due to issues around identifying relevant currencies and exchange rates.

Why wasn’t salary data checked during the survey? Under DLHE, any outliers would have been queried during the submission process.

There is validation within the survey itself and we ask graduates to double check their figures if they think they fall outside of what we would accept as the normal range. If a graduate confirms that it is correct, then they are free to enter that response and move on in the survey. We will look at where we can make improvements, where necessary, over time.

Have HESA published how the 95% confidence intervals are calculated?

Further information on how confidence intervals are calculated is included in the footnotes to the tables that display confidence intervals. HESA uses the Goodman method.

Are there any concerns about the quality of partial responses?

We’ve included partial responses in our statistical outputs because they include usable survey data. We have no reason to believe that the quality of the responses is any less than the responses of graduates who have a completed response.

Will partial responses be included in all future statistics e.g. performance indicators and league tables?

This depends on the particular measures in question. With partial responses, there will be high levels of unknowns in certain questions. If the measures in question are based on survey questions where we don’t have a full coverage of responses, then that will be taken into consideration in terms of how those are defined.

Is SOC analysis only performed on complete responses?

In addition to complete responses, partial responses are also SOC coded as long as we have usable data in the JOBTITLE and JOBDUTIES fields.

Is ‘work is an activity’ filter derived from XACTIVITY or ALLACT?

The work population marker filter displays two options: ‘work is an activity’ and ‘work is most important activity’. ‘Work is an activity’ is derived from all graduates who report an ALLACT 01-05. ‘Work is most important activity’ is further restricted to all graduates who report a value for MIMPACT within 01-05.

How is significant interim study identified?

Our definitions page describes how we work out if somebody was in significant interim study or not. Essentially, we include any graduate who has completed a full-time higher education course or professional qualification between the points of graduating and the census week. On particular tables, we allow you to exclude that subset of people, depending on what you wish to do with the data. We have provided an expandable box within the release that provides this definition plus an example of why you might like to exclude these types of graduates.

Is significant interim study derived from other HESA returns or is it self-reported?

At the moment, this is derived from the Graduate Outcomes survey itself. There is a section within the survey which asks graduates about any further study, training or research they were doing in the interim period. In the future we would like to start linking data sets to find students in our range of student records. This is not something that we’ve been able to do for this particular release.

Why would a user want to exclude or include those with significant interim study?

This depends on the user’s purpose for the data. For example, a user may want to exclude those in significant interim study to assess the rates of unemployment among graduates. This is because among other examples, there will be some first degree graduates who go onto full-time master’s courses which they complete just before the census survey week begins. These graduates will have had less time to be looking for work compared to their peers who completed only the first degree. When looking at unemployment, it may be more suitable to exclude those who report to be in significant interim study.

What subject classification scheme are we using?

We are using the JACS classification for the subject data within this release.

Is HESA going to release a provider-level subjective wellbeing table?

No, we are not planning on doing this during our first year of dissemination.

Will HESA be publishing unrounded data?

No, we will not due to data protection reasons. If users require unrounded figures, please get in touch with our colleagues at Jisc who compile bespoke datasets specific to your needs. There is a fee for this service.

Future use of the Graduate Outcomes data

How will data for competitor analysis be presented for providers to use and analyse?

The main vehicle for delivering data for competitor analysis will be via Heidi Plus which is delivered by Jisc. We are in discussions with Jisc about defining the set of appropriate Graduate Outcomes data to be released through Heidi Plus. This is a complex process and we are looking to focus on this work over the coming weeks.

Is HESA engaging with the OfS to ensure that TEF and UKPIs are consistent?

Yes, HESA will be picking up this work over the coming weeks and months. We will be looking at emerging thoughts from the OfS particularly around the definition of TEF metrics and we need to ensure that the metrics are either aligned or complementary with the UK Performance Indicators (UKPIs). Find out more about UK Performance Indicators.

Is the data robust enough to inform the TEF and if so, why, or why not?

This is not something that HESA can answer. It will be a decision for particularly the OfS and the other administrations in the UK who feed into that discussion. Organisations like the OfS will need to make assessments on quality and decide whether they feel that they fit for a particular purpose.

How will HESA work with league table compilers?

Our work with league table compilers has already commenced. We will be meeting with as many league table compilers as possible to ensure that they fully understand the data, what it’s capable of providing and that we understand what their expectations are and what they’re trying to achieve from the data.

Our survey population contains a set of graduates on part-time courses that we think shouldn’t be in the survey because they are not graduates. Could they be excluded from future survey populations?

In terms of the courses that are in scope for Graduate Outcomes, it’s not just degree courses. There may be HNC or HND type courses that are not degree courses. A full list of qualifications that are in scope for the survey can be found in the coverage page of the coding manual.

Will the C19071 Graduate Outcomes survey consider the impacts of COVID-19?

All graduates surveyed in a particular cohort are asked to reflect on their activities in the census week. If they were in employment in the census week, then they would be encouraged to record that activity. If they were not in employment, they should record that. We have made allowances for the recording of ‘furlough’ as a result of COVID-19 and graduates will still be able to record this as employment.

Regarding how we present the statistics, we are acutely aware that the pandemic and the after effects will have a significant impact on graduate experiences as they enter the labour market. We will look to ensure that we explain this as fully as possible to users of the data.