Improving Graduate Outcomes response rates: why quality contact details matter

It may seem obvious, but it needs to be said: having good contact details for a survey will affect the response rates to that survey. The better the contact details, the better the chances of contact with prospective respondents and the better the insights we can get. 

Graduate Outcomes is no different. The journey to success starts with contact details. 

During the course of Graduate Outcomes, we have made significant progress (working alongside higher education providers) in improving the quality of the contact details we obtain. This has helped us to see an increase in successful contact, and improvement in both the online and telephone response rates.          

But the quest to improve the quality of contact details is never over. We have learned a lot over the past two years, and we wanted to take this opportunity to share our insights so we and HE providers can continue to improve contact details for Graduate Outcomes, enabling us to improve response rates.

We also want to share some best practice pointers that have arisen out of our work.

But first to what we have learned so far.

Mobile phone numbers are important for response rates

• Email and mobile numbers are the most important contact details we need (our two main response modes are online and telephone). 
• Mobile numbers are influential because they work across both online and telephone response modes.
• Online completes contribute around 40% of total responses. 70% of those come via email, 30 % via SMS.
• We think SMS may be acting as a reminder to complete via email (in addition to its primary role to elicit online responses via a link in the SMS). 
• For the other main mode (telephone) mobile numbers play a critical role in achieving good response rates: the connection rate for mobile using Computer Assisted Telephone Interviewing (CATI) is higher than for both landline and international phone numbers. 
• CATI represents a majority of our total response (60%). In recent cohorts over 80% of telephone surveys are completed on a mobile number, so having a mobile number is crucial for obtaining good responses in this mode.

Email reaches nearly all graduates – but that’s not the whole story

• Email delivery rates are usually at least 97% – and we have email addresses for almost all graduates. So, in theory, it’s only a very small proportion that we can’t contact via email.
• Email delivery doesn’t always translate into survey response. We have found that by the end of a cohort a significant proportion of graduates will not have opened any of the emails we sent (we usually send six to seven rounds of emails each cohort). It is possible that although the emails get delivered to an address the account itself is no longer accessed by the graduate.  
• ac.uk email addresses have a much lower than average open rate. The bounce rate for ac.uk email address is also much higher. Although only around 2% of the graduates in the largest cohort of respondents had only an ac.uk email address, this proportion can be much higher when looking at individual HE providers.
• Even where emails are opened, people may be hesitant to click on the survey link. This can be addressed through increased brand awareness and communication by providers aimed at legitimising the survey.

There is a link between high rates of online response and higher response rates overall

When looking specifically at online response rates we have found that:

• Providers with the highest response rates also usually have a good online response rate. There is less variability in telephone response rates between providers.
• Online response rate is significantly higher when graduates have both a valid email address and mobile number (rather than just email) we can use to contact them on.

Contact details best practice

To sum up:

• As far as possible submit a mobile number for every graduate.
• At least one mobile number and email address should be supplied for UK graduates. These should help obtain good online and telephone response rates.
• An ac.uk email address is generally not important and is less likely to perform well. Provide a personal email address for every graduate and as far as possible check it is the current address for the individual.
• Contact details should be continuously updated during the survey field period to give us the best chance to contact your graduates.