Survey, systems and subscriptions: An update on Graduate Outcomes
A great deal of progress has been going on behind the scenes in HESA over the past few months, ensuring we are on track to deliver the first Graduate Outcomes survey at the end of the year.
I’ve chosen some of the key areas to share some additional detail with you about what we’ve been doing, and the delivery of Graduate Outcomes.
We will be using the Confirmit system to manage the surveying of graduates.
We’re pleased to be announcing today that we will be using the Confirmit system to manage the surveying of graduates. The Confirmit system is widely used to conduct surveys by leading sector bodies, including the Office for National Statistics. It conducts the management of survey contact with graduates both online and for telephone interviewers, meaning there is live interaction between the different methodologies. This was crucial to us when selecting a system, to ensure as soon as a graduate has completed online they will be pulled from the call queue and vice versa. HESA will be operating this software to conduct the survey with graduates.
The system also includes a sophisticated online system which is instantly smartphone compatible. Given that in the final LongDLHE survey, 61% of graduates who completed online did so using a smartphone or tablet, we see this ease of use on a smartphone as crucial. We are working with Confirmit to ensure we design the online survey in a way which will engage graduates.
Confirmit are now working on building the Graduate Outcomes survey into the system, which sets us up to take forward the procurement process to get a call centre in place, who will use the Computer-Assisted Telephone Interviewing (CATI) segment of HESA’s Confirmit system. The overall running of the survey through Confirmit and by the call centre will be overseen and managed by us at HESA.
Our final survey design has now been confirmed and we’ve published this, along with the outputs from the cognitive testing which we contracted IFF research to undertake. This gave us a great opportunity to not only test the new Graduate Outcomes questions, but also to get those taken from DLHE properly tested. This is helping us to ensure Graduate Outcomes utilises best practice in survey design and methodology.
The outcomes from the cognitive testing were very positive, with 95 face-to-face interviews and 1094 online surveys undertaken.
The outcomes from the cognitive testing were very positive, with 95 face-to-face interviews and 1094 online surveys undertaken. Graduates were supportive of the new name, the flow of the survey and length of time for completion (which on average took only 47 seconds longer than DLHE). The area which received the most positive reception from graduates was the newly developed graduate voice questions, which reiterates to us the importance of these new measures for providing a wider view on what makes up ‘graduate success’.
Many of the recommendations were for low level changes throughout the survey to aid graduates’ understanding, which we have implemented. One area that came up as requiring some more work is the description of what a creative, artistic or professional portfolio means – there wasn’t a consistent understanding across the graduates surveyed. Based on the feedback from the cognitive testing, we have sharpened up some of the language around this area, however we will also be looking to further develop these using portfolio career experts and further testing beyond the 2017/18 survey.
I am often asked how students who have recently left post-graduate study will be captured in the survey. You’ll see from the final survey design that although we focus on graduates’ activities in the census week, we also capture what they were doing over the last 15 months and whether they are about to start a course or work. This allows us to show a complete picture for these graduates. Once data has been collected we will review this to see what impact it should have on our outputs.
The published survey and reports are now available. This includes both the core and opt-in question banks.
We have taken the decision to postpone the inclusion of the provider questions in the survey for the first year, due to the legal, financial and operational complexities which surround these and to allow providers to manage the potential demand for these. These will be included from the 2018/19 survey.
Alongside getting the Confirmit system set up, HESA are also working on developing our provider portal. This system will allow providers to access the Graduate Outcomes population file (currently available through the Student record), upload provider logos for the survey, select which opt-in banks to be included in the survey and, most importantly, provide the contact details for us to contact the graduates.
Today we have also published an update to the data collection schedule for the contact details, intended to minimise the burden of the sign off process for this element.
We’ve been working on the subscription model for Graduate Outcomes. The cost of HESA delivering the Graduate Outcomes service will be met through subscriptions from HE providers. This reflects the centralisation of the survey operations that forms a key element of the new collection.
More information about how this subscription will operate is now available.
What comes next?
The Graduate Outcomes team’s work over the next few months is going to be largely focussed on the development and integration of systems and the communications and engagement activity to familiarise graduates with the Graduate Outcomes brand and encourage response rates. Look out for more information, and questions, about how we can best work together to produce joint communications activity to ensure we all achieve the response rates needed.
And most excitingly – the first ever Graduate Outcomes conference is less than a week away. For those attending, I’m looking forward to seeing you there.