Data intelligence

Among graduates from 2017/18, there were 175 from Northern Irish providers that were duplicated in the dataset. These graduates were all taught at a Further Education College (FEC) but were awarded their qualification by a Higher Education Institution (HEI). Both the FECs and HEIs returned contact details for each graduate. Of the 175 graduates, 130 should have been returned by the FECs only and 45 of the graduates should have been returned by the HEIs only. Because these graduates were duplicated, they would have been invited to survey twice for their qualification.

In 2017/18, among Northern Ireland Further Education Colleges approximately 110 graduates were included in the Graduate Outcomes population in error. These graduates completed a short course (ILM Level 5 Making Professional Presentations) that was taken alongside their main qualification but had a different date of completion.

The most comprehensive assessment of data quality in the Graduate Outcomes survey currently available can be found in the Graduate Outcomes quality report.

In 2019/20, The Open University underreported the number of qualifications awarded due to administrative delays as a result of changes to assessment timetabling. There is a reduction of around 8,225 qualifications awarded in the 2019/20 data return in comparison to 2018/19. Most of this drop was seen among part time undergraduate students. The reduction is reflected in a lower population of survey respondents for the Graduate Outcomes survey of an estimated 7,900 responses. These qualifications were returned in the 2020/21 student data, the 2020/21 Graduate Outcomes population reflects this.

In 2020/21, there was an error in the census year referred to in the Welsh language version of the Graduate Outcomes survey.

There were just over 3,280 responses received from graduates who had the option to complete the survey in Welsh at these providers. Of these responses, approximately 300 were completed in Welsh. Around two thirds of these Welsh responses were collected via Computer Assisted Telephone Interviewing (CATI) and, following extensive checks of recordings, we could confirm that interviewers corrected the census year at the time of interview. These responses could therefore be confirmed as referring to the correct census year and retained as part of the dataset. The remaining third of these responses were completed online and could not be verified as relating to the correct census year. As a result, due to the data quality concerns, all the online survey responses to the Welsh language version of the survey have been removed from the dataset.

These graduates were not removed from the surveyable population dataset used as a base population to calculate response rates; they will appear as ‘non-respondents’ to the survey.

In 2019/20, an error existed in two of the opt-in research questions for the Welsh language version of the Graduate Outcomes survey (Collaborating with others in different subject and Collaborating with others outside HE research). Whilst the question text was displayed correctly, the selectable answer options displayed blank (empty) boxes. Therefore, graduates who were eligible to answer these questions and chose to complete the survey in Welsh would not have known what answer option they were selecting. This issue affects 5 graduate responses.