Our approach to surveying non-EU international graduates
As we approach the start of year four of Graduate Outcomes, we’re providing an update on the approach we will take for non-EU international-domicile graduates*, starting in December 2021.
Back in July we made the decision to cease calling non-EU international graduates from year four. Since then, we have been working to put this into operation, engaging with key stakeholders and finding ways to alleviate the impact for some users and uses. While the work is still ongoing I’m pleased to share more on this below.
What is changing and why?
Firstly, let’s quickly reflect on why and how the decision to cease calling international graduates was made.
In July we announced the decision to stop calling non-EU international graduates, a decision which was made in consultation with the Graduate Outcome Steering Group. The key driver for this is the need to reduce cost on Graduate Outcomes following the government’s commitment to reduce burden in the higher education sector, in which the DfE and BEIS asked HESA to make a 10% efficiency saving by 2022/23.
Graduate Outcomes survey is a significantly large component of the overall HESA subscription. After considerable reflection of the options available to make such a significant cost saving, HESA and the Graduate Outcomes Steering Group backed discontinuation of calling non-EU international graduates. In making the decision, HESA and the Steering Group were conscious of the need to preserve our focus on the UK domicile population, in view of the constraints we must operate within. I want to emphasise that this wasn’t an easy decision for HESA or the Steering Group as we firmly believe that outcomes of all graduates are important, regardless of domicile. Therefore, up until now we have implemented a singular data collection methodology across all graduates, at all providers. The decision to stop calling international graduates is a practical response to a strategic, external directive.
The removal of the telephone mode for international graduates means that the engagement plan for this target group will consist of email and SMS contact only. This means international graduates can complete the survey online only via the secure personalised links provided via these modes. There will be no change to the engagement strategy (the sequence of invitations and reminders) across these modes, but they will not receive calls from our contact centre to complete the survey over the phone.
How will this impact our understanding of international graduates’ outcomes?
The short answer is, we don’t know yet.
This is because useability of the data will depend on the quality and quantity of data we collect from these graduates, under the new system. This in turn depends on several factors: coverage and quality of contact details, graduates’ willingness to take part in the online survey, possible differences between responses collected online vs over the phone; to name a few. Through a combination of mitigating actions, continuous monitoring and quality assurance exercises we will be able to understand the true impact of this change on the data and our understanding of outcomes for this group.
Based on our experience of running Graduate Outcomes over the last three years, it is fair to assume that the removal of telephone interviewing will result in a reduction in response rates for international graduates. To reflect the change to the engagement strategy for these graduates, the response rate target will reduce from 25% to 20% from year four. It’s important to remember that response rates are not the only measure of survey performance – we will be monitoring any impact of this change on the usability of the data and report our findings in due course.
We understand the impact of this decision on the sector and have been exploring ways in which the consequential reduction in the non-EU international domicile response rate could be mitigated for some users and uses.
We will be going back to basics with our messaging to ensure our survey communications satisfy the key question these graduates will be asking themselves - ‘why should I take part in the survey?’ as the reason may be different to the other target groups. Historically sector response rates for this group have been much lower compared with the UK domiciled graduates. In collaboration with providers, we must continue to make Graduate Outcomes visible to all prospective respondents by improving brand awareness.
In addition to this we are exploring ways in which we can gather insights into respondent and non-respondent behaviour to adapt our style and process of engagement. Finding new strategies and methods to resonate with this audience will be our continued focus over the coming cohorts and collections.
As ever, it is essential that contact details for all graduates are up to date and accurate, especially UK mobile numbers. These are particularly effective for online responses because they can be used to access the survey directly while also serving as a reminder to a previous email. Our analysis of contact details suggests effectiveness of non-ac.uk email addresses is far greater than ac.uk. We therefore expect providers to collect at least one personal email address from all their graduates and most importantly from non-EU international graduates. This would maximise our chances of contacting them and collecting their response to the online survey.
No impact on EU and UK graduates
We will continue to survey EU and UK graduates across all modes (telephone, SMS and email). It is essential that providers continue to gather email addresses and telephone numbers for these graduates as accurately as possible. Response rate targets for these groups remain unchanged.
It is also vital that providers continue to implement their communication strategies to increase brand awareness for the survey. The decision to cease calling non-EU international graduates in no way impacts providers’ engagement with graduates about the survey. Providers should be using the supplied brand materials and implementing the suggested graduate contact plan to ensure all graduates receive “warm up” communications prior to being surveyed.
Survey changes and year four subscription model – coming soon
As the decision to cease calling international graduates was driven by a savings directive, we know providers will want to see the impact of this change on the Graduate Outcomes subscription model and we will be sharing this very soon. The resulting model will reflect the costs of running the survey more accurately and provide the best possible service for the sector as a whole.
In the coming week, I’ll also be sharing the results of the first phase of an extensive survey review that we’ve been carrying out in consultation with the Graduate Outcomes Steering Group. We’ve been able to make a number of changes that will positively impact data quality, reduce burden and deliver a cost saving. I look forward to sharing more about this in due course.*Graduates with a HOMEEUOS = 3. HOMEEUOS is a flag used to identify graduates’ home domicile using the derived fields in the C20051 Student and C20054 Student Alternative records.