Resources and guidance archive

There are only 7 days between the confirmation of the final population for cohort A and the sign off of contact details. Can we have more time?
This is limited by the restriction of the Student/AP Student record sign off dates and the start of the survey, but please be assured this is the only cohort when the timelines will be this tight.

The system opens early to provide more time to complete any activity that can be done prior to the Student record sign off. We suggest you work closely with Student record contacts and ascertain if the leavers from August – October last year can be prioritised. You can go through the QA process before sign off and it is worth remembering that cohort A is not the whole population – only around 20%.

Can the population be downloaded in Excel/CSV?
You can currently get the population file for cohort A from the Student data collection system in CSV. At the moment, within the Graduate Outcomes provider portal, the population file is available as XML only – if once the system is open there is demand for CSV, we will look to develop CSV versions. We have produced a short video demonstrating how you can use Microsoft Excel to open the XML as a CSV format, available in the user guide.

How will we know what cohorts we are eligible to survey? How do I access our current Cohort A population?
This is now available in the Graduate Outcomes provider portal.

Is the provider portal still set up for overrides for .xml files and not cumulative for updates to contact details?
It is cumulative so you can add to the population and update the file.

If we are making changes, do we have to submit an .xml that includes everyone or can we just submit a subset of the population?
It is possible to just send us a subset.

What would trigger a response rate query? Is it being below the actual average provider response rate for a cohort? Or below the estimated 60-65%?
A response rate below the estimated 60-65%. Note that we will be reviewing the response rate targets after the first year.

What happens if no update to contact details is available on a repeated attempt after the contractor notification during the survey contact period?
In these instances, we will accept that we cannot get a response for that graduate. They will not be surveyed.

The predicted response rates are lower than previously predicted, will heads of institution be notified of this revised expectation?
There will be no targeted communications to heads of institution regarding this matter. Our point of contact is the Graduate Outcomes record contact - it is up to these contacts to circulate this as required.

What would constitute 'high quality' contact details?
We expect contact details to be provided for all graduates and to be details that we can use to get responses from graduates e.g. valid email addresses and phone numbers.

How many times will the third party try and make contact before referring back to the provider?
The contact centre will use all of the contact details submitted in order to obtain a response. They will not be ‘referred back’ – they will appear in the web form on the provider portal with a status showing that we have not yet been able to contact the graduate using the methods supplied.

Will the survey provider still be taking valid answers from third parties (i.e. the graduate's parents, spouse, etc)?
Yes, telephone third party responses will be accepted, provided they can answer enough of the survey.

What will happen in terms of analysis should we have a significant failure on the response rate? 
Individual circumstances will need to be evaluated if this becomes an issue.

Can you provide any information on non-UK response rates and the Research Council response rate?
The response rates for these are as yet unconfirmed but we will be able to share more after the steering group meeting in October.

What will you do when we reach a 60% response?
HESA will be monitoring response rates through a range of hard and soft target response rates to ensure they are being met amongst course, levels of study and different personal characteristics. We will stop attempts at contact once we are happy these soft and hard targets are met across these groups.

If one institution gets a 60% response and another an 80% response it will make a big difference to the result for TEF, league tables etc. How will you handle this?
As per the the above answer, we will stop attempts at contact once we are happy these soft and hard targets are met across these groups. We cannot comment on the onward use by the DfE for TEF, or league table providers.

How will a refusal to participate be treated?
If a graduate refuses to participate they will need to inform the call centre or HESA (through the Graduate Outcomes website) that they do not want to participate and they will then marked as explicit refusal.

If a statutory body opts us in, do we only have one opt-in question bank left as a provider?
A total of three opt-in question banks are allowed. For the 2017/18 collection, providers are able to select any opt-in question banks as no statutory customers have chosen to mandate any of them. We created a limit to prevent the survey getting too long but we will be evaluating the demand for these in year one.

Are we able to devise our own questions to add to the survey rather than use the opt-in questions already set?
We call these ‘provider questions’ – we are looking to develop these for year two due to the legal and operational complexities of inclusion.

What is the deadline for selecting opt-in question banks?
Providers can opt-in to additional question banks in the provider portal. The deadline for confirming provider opt-in question banks is 26 October 2018 and these will be added to the survey for all cohorts for C17071.

An addendum to the HESA subscription agreement will be issued after this date and will need to be agreed and signed by your Accountable Officer/Head of Provider on or before 9 November 2018 for the provider opt-in question banks to be included in the survey.

How will we define a set of students that we want to receive a specific opt-in question bank? Can we have different sets receiving different questions?
If an opt-in question bank is selected, all relevant graduates will answer the questions.

IWhen are opt-in question banks available from?
Opt-in question banks are available for all cohorts. If you select an opt-in question bank for cohort A, it is applied across all cohorts.

Are the general subscription costs still being released in September as previously advised?
The Graduate Outcomes subscription and opt-in question bank costings are now available on the HESA website. We have notified your Vice-Chancellor of these costings.

How have the costs per graduate been calculated?
The cost per graduate is a blended rate which takes into account the different response methods and associated costs.

Will HESA sell opt-in question bank responses or do they belong to the HEI? Who owns these responses and can third parties use this data without permission of the HEI?
The data from these will be available through publications i.e. in the same way as the core survey data.

For ITE providers, will DfE pay for that question bank?
They will confirm soon but yes they will pay if they wish to mandate it.

When will we know if any opt-in question banks have been mandated by other parties?
There are no mandated question banks in year one.

How have institutions been notified of the opt-in questions and the option to opt-in?
The opt-in question banks and ability to opt-in have always been part of the Graduate Outcomes model. The provider specific questions are out of scope in the first year but will become part of the model in year two.

What is a “switch”?
Where a validation / quality rule error is triggering and you have a justified reason for why it might be, we can apply a switch which means the error will be turned off. A switch can be applied by contacting Liaison.

Will structural rules be switchable? As it would be conceivable that a student may change their surname between submission in the AP return and inclusion in the Graduate Outcomes population (e.g. post-graduation marriage).
Surname change is allowed, it just must be returned in the SNAMECHNGE field rather than in SURNAME field. Switches will be possible for provider and record errors only. We expect that the rules are loose enough to fit most circumstances.

Within the quality rules, do warnings need to be corrected? If a student has got the same number for mobile number and home number, will we have to remove one?
The warnings require checking, the errors need fixing or switching. With regards the same number this will trigger a warning where the UK telephone number is the same as the mobile number so will not need removing however where a UK mobile number returned is the same as the UK telephone this will trigger an error (If the mobile does not start 07).

When will the quality rules be finalised?
The note against the quality rules on the manual identifying them as in development is reference to the fact that the rules may change slightly after cohort A if we feel there is a need.

How will you contact us regarding quality issues?
Providers will be made aware of quality issues on the contact details through on-screen alerts within the provider portal.

Is there a validation kit?
There will be no validation kit for Graduate Outcomes. The data that is now captured and the rules that are in place to validate are much simpler than in the past so it was felt there was no requirement for a validation kit.

Do the different stages of errors happen in stages or at the same time? Do we need to get past first stage of errors to see second stage?
There are three types of rules: Structural / Schema, Provider and Record level. The structural rules are the first stage of validation and prevent the file from being successfully loaded. If a single issue is present then the file cannot be loaded. The provider errors and record level errors occur at the same time.

Will there be validation to show if UKMOB should be returned as INTTEL? In our records system, we don't differentiate, we just hold one mobile number.
Yes – the rules will trigger if a UKMOB is not a UK number. It is worth looking at the contact details rules to see how they fit with the contact details you hold and whether it’s worth doing any data cleansing. This is something you can be doing immediately.

Do we only populate address entity when GradStatus is equal to 2?
Yes, only populate PostalAddress entity when GradStatus is 02 as there is no other way for us to make contact with them. We don’t take address information for other graduates as we won’t use it. You do not need to return the PostalAddress entity and associated fields unless GRADSTATUS is coded 02 (not contactable by email or telephone).

If a record has a Graduate status of 01 or 03 and does not require contact details that we as a provider uploaded, do we need to manually remove them or does the system take this into account automatically?
Graduates with a GRADSTATUS of 01 or 03 can be submitted to HESA and we will remove/not send these to the contracted surveyor. You do not need to remove them from your file.

Can telephone numbers have hyphens or spaces?
No.

The international phone numbers are not possible to validate and include dialling codes, it is a free text field so we have no idea if they are correct or how to add a correct dialling code. What should we do?
This is a data cleansing exercise that you should complete to ensure all contact details are usable.

What if two students genuinely have the same landline number i.e. housemates or married couples?
In this instance you would need to request a switch (see definition in Quality rules section below) with Liaison.

What if the student has an institutional email as their main address because they have continued onto a different course at the same institution?
This is allowed – we only have a warning if more than 10% of email addresses end in .ac.uk.

If international number is in the same field as UK numbers in our student record system, will these appear under UK numbers and then need moving to the correct field within the web form?
Yes, you will then need to undertake data cleansing. This will also be reported back to you as a quality rule warning on submission.

Are FNAME and SURNAME taken (without amendment) from the Student return?
Yes, these are pre-populated in the provider portal web form.

When will the website www.graduateoutcomes.ac.uk go live?
The aim is for this to go live in October. We will be emailing record contacts when this has gone live as this is a key milestone. It provides an information source to link to from any communications you may have planned.

Will leavers being contacted in the survey be able to update their contact details via www.graduateoutcomes.ac.uk or can this only happen through institutions?
Yes – they will need to email [email protected] (this will be listed on the Graduate Outcomes website). Please note that any contact details sent via the Graduate Outcomes website will not be shared with the provider due to data protection issues.

What's the rationale behind not asking respondees request an update to contact details on the webpage if these can be passed back to providers? Would make sense to update details in golden source. Could they then be over-written again if we are sending revisions during the contact period window?
We would not be able to cover all of the potential uses of all different providers. From a data protection perspective it was not viable. Contact details will not be overwritten as updated versions will sit within Confirmit.

What materials are available to help me promote Graduate Outcomes to colleagues and students?
We have worked with a design agency to develop communication materials for Graduate Outcomes. We will provide a number of additional materials in October which will include materials to use on LinkedIn and Twitter.

How should I refer to Graduate Outcomes?
The name is Graduate Outcomes. When referring to the record, the survey or the data collected by the record, capitalise Graduate Outcomes. If you are referring to general data, use lower case. For example:

• Graduate Outcomes is HESA’s newest record.
• The Graduate Outcomes survey collects information on the activities and perspectives of graduates.
• The first Graduate Outcomes collection will be C17071.
• The first Graduate Outcomes data will be published in January 2020.
• HESA publishes data about students, staff and graduate outcomes.

Arolwg Hynt Graddedigion is the Welsh version. We are doing further work on how we will write about the survey in Welsh materials.

We do not foresee a need to shorten Graduate Outcomes, and so we are not going to introduce a new acronym.

Where a shortening is absolutely necessary, it should be sufficient to use just ‘the record’, ‘the ‘survey’, or ‘the data’. If clarity is needed, use ‘outcomes’. For instance:

• Graduate Outcomes data will be published in January 2020. The data will be released on…
• The Graduate Outcomes record captures details of graduates’ activities 15 months after completing their studies. The record comprises a survey and the return of contact details.
• We return data to the Student, Staff and Graduate Outcomes streams. The outcomes data is returned four times a year...

We may allow the use of ‘the GO survey’ to brand and promote the survey to graduates if we feel this will enhance engagement. But this is the only scenario in which the ‘GO’ acronym should be used.

Who are the third parties hired by HESA to run Graduate Outcomes?
As announced earlier in the year, Confirmit will be providing our contact management system, which includes management of the CATI and online survey system. We are in the final stages of agreeing contracts with the suppliers who will be operating the call centre and SIC/SOC coding. We will announce the final two suppliers as soon as contracts are in place.  

Will the career provider link be read out over the phone if graduate doesn't respond via email?
No, the URL is just for the graduates completing the survey online.

When are the census weeks?
The census weeks can be found at the below link: https://www.hesa.ac.uk/innovation/outcomes/providers/timing

When your call centre gets through to a Mum / Dad and they provide another new number for the graduate, how will this new number be captured?
This will be captured within the Confirmit system.

How do I acess the provider portal?
The provider portal is now live and we have communicated this to record contacts and on the JISCMAIL email group.

How many records can be contained in a 25MB file?
We cannot advise with any certainty on this but we have undertaken research into the existing DLHE files received and we know this to be more than sufficient.

Does access to the provider portal need to be requested by one institutional contact or can multiple access requests be made in bulk? We have several careers and alumni colleagues who will need access.
Record contact and admin contacts can approve access to the provider portal. You can give out as many roles as you need.

Will there be a separate dashboard for graduates' current activity (for example), at course level and the subsequent SOC and SIC allocation?
Yes – dashboards will be made available through the Graduate Outcomes provider portal but we’re unable to provide a demo at this stage. We will be releasing training material on this as soon as we can.

What support is HESA providing for institutions whose systems don’t simply "spit out" an .XML file in the correct format? 
For those providers that are unable to generate an .XML we have the web form tool located within the Graduate Outcomes provider portal. From there you can see the eligible population and use it to start adding contact details. The web form can also be used for ad-hoc amendments to contact details.

When can I assign additional Graduate Outcomes roles via HESA's Identity System?
Graduate Outcomes identity system roles can now be assigned within the provider portal.

How do I gain access to the Student data collection system? I just logged into the Data Collections - is it C17051 I need access to?
Yes. You will need to liaise with the Student record contact at your provider for access to this.

How can we check who is the approver of contact details within the institution? The sign off on DLHE it was the VC - is this the same?
There is no sign-off as such with Graduate Outcomes contact details. Instead, the data needs to be approved by the contact details user.

Is there a cost for including the providers' careers link into the survey?
There is no cost for providing this. Similarly there is no cost for adding a logo.

Can the web form be used for small updates, rather than resubmitting the whole .xml file?
Yes, the web form can be used for small ad-hoc updates.

You mentioned that the survey cohort will only be available in .XML format. HESA have said that the first cohort A will be available in csv format also. Does this only apply to this year’s cohort A, and that when cohorts are only available via the Graduate Outcomes provider portal, they will be .XML only?
Correct, the Graduates Outcomes provider portal will only have .XML population files. To mitigate this, we have produced a short video, available in the user guide, outlining how you can use Microsoft Excel to open the population file and then save as a CSV format.

There is a JACs field in the .xml, will this be updated to HECOS/CAH in the future?
Correct, we are intending to move to HECOS for 2018/19 graduates.

How many logos can we upload into the portal? Graduate Outcomes one plus our own?
A single logo must be uploaded by a provider. The Graduate Outcomes logo is included in the survey automatically by HESA, so there is no requirement for you to upload it. The provider logo should be uploaded once and will then be used across all cohorts.

If we make changes, can we resend the whole file rather than a subset?
Yes, but beware that in doing so you will overwrite the contact details previously supplied. As per the question above, it is possible to just upload a subset.

When will we get updates on non-completion? Will these be in real time or will they be released at specific intervals / dates?
Updates will be provided weekly through automated emails.

If the contact centre asks a provider to investigate further contact details is this communicated and updated via the provider portal?
Yes, HESA will be prompting this further communication and we will be notifying you of this through the provider portal.

Will we receive our entire survey file for our own analysis, and when will we receive these?
At the end of each cohort we will be supplying a data supply file via the provider portal. The format of which is still to be determined. The entire survey file for all cohorts will then be provided after the final cohort (D).

I’m concerned that resubmission overwrites any data entered via the provider portal. There is now a disconnect between HESA and the provider and also the scenario that portal data will be overwritten. Is it possible you can respond to the resubmission question?
When changes are made to the Student record we ‘soft delete’ contact details, so if these graduates reappear in the provider portal their contact details will reappear with them (if they don’t reappear, they will be fully deleted). You will need to work with Student record contacts to plan this work.

If we want to update contact details after an initial file has been successfully uploaded (but before the survey has been carried out), is it possible to make these changes by resubmitting a file rather than using the web form?
Yes a new file can be uploaded.

Are institutions expected to chase non-respondents? This is not something that I've picked up before, I thought that the contractor would be responsible for chasing. Could you confirm what is expected please?
HESA will report back to you, through the provider portal, the non-responders to the survey. We suggest that you do not try and contact graduates as there is a likelihood of crossover in contact and this could put graduates off. Where there are bad contact details (e.g. email addresses which bounce back) we will expect providers to seek to supply additional contact details. We have a comprehensive engagement strategy that is designed to encourage optimum responses..

Will we be able to bulk download errors? If yes, what format will this be in?
Due to the nature of the data that is collected, we are not envisaging lots of errors. There will be no download of the rules triggered available.

We do not store serious illness on the Student or Alumni database, is it ok for this to be blank? Serious illness is subjective too.
Yes, the GRADSTATUS, FNMECHNGE, SNAMECHNGE are only intended to provide additional information to support the survey process if you hold this information.

Is it best to keep updated graduate contact details on Alumni or Student database? Previously they would be transferred to Alumni and the student database would rarely be updated. Is this not best practice now?
This is a decision to be made by each provider. All we require is that you return the most up to date details.

If the Alumni database is used we would need to convert an Excel file to XML, can you provide or recommend a tool?
Unfortunately we cannot recommend a tool. Contact details can be added through the web form so an XML does not need to be uploaded.

Our graduates can opt out of contact from us (provider) via our alumni team by phone, email, etc. Does this count as a legitimate opt out via GRADSTATUS 02?
No, a graduate cannot opt-out of Graduate Outcomes before the survey takes place.

If a student graduates and then starts a masters course, they will be surveyed twice. Are there any plans to account for this in the collection?
Throughout the survey they will always be asked to respond in reference to the relevant course.

Can we contact graduates during the contact period to encourage them to complete the survey? Does this have any implications for GDPR?
We suggest that you do not try and contact graduates as there is a likelihood of crossover in contact and this could put graduates off. You should seek advice from your data protection officer on the GDPR status of your contact with graduates.

Under GDPR, when we contact graduates in the period between graduation and your data collection, are we limited to only asking them to update contact details? Can we check what they're doing now?
This is restricted by your own data protection notices.

Is there going to be any advice on how to make other systems (such as Raiser's Edge) to create a submission file?
HESA has shared information with software houses, however it is for providers to make contact with your software houses and ensure you can submit up to date contact details in the correct format.

Data Futures states that information such as placement information is required for Graduate Outcomes. Can you advise why this is?
Placement information is collected through the Student record rather than retrospectively as this is information that will be available while the student is studying. Therefore, it should have higher quality than asking a graduate to reflect back. We do not ask any placement information in Graduate Outcomes so there will be no conflicting information.