Methodology

Methodology

Version 1.0 Produced 2011-02-18  

Table of contents

Arrangements for the 2010/11 DLHE survey

There are two census dates for each year. For the majority of students completing a programme in either autumn or summer the reference date will be six months after completion.

Students who complete their courses between 1 August and 31 December 2010 have a census date of 18 April 2011. Students who complete their courses between 1 January and 31 July 2011 have a census date of 09 January 2012.

Institutions should report firmly determined destinations in place at the reference dates.

Graduates should be surveyed in either April or January as appropriate: institutions are advised to maintain the discrete survey populations as it is possible that the numbers of leavers being surveyed at the wrong time will be monitored through audit. Surveying students unnecessarily may lead to survey fatigue and damage response rates in the long term. However, it is recognised that for some students the final confirmation of award by exam boards may be several months after the student completes the course, such that the qualification obtained is not known in time for the student to be included in the April survey. Under these circumstances, and only these circumstances, institutions should include these leavers in the next January survey.

Note that despite there being two data capture reference dates, there is only one data collection, the return date for which is 31 March 2012.

The DLHE exercise spans two years and data collected from the April survey period is not returned until the following March and so it is imperative that the surveys are stored safely until the time. The DLHE schedule diagram explains the stages and timeframe of the DLHE process.

Improving and maintaining a good response rate

Institutions are reminded of the response rates for the DLHE survey, namely:

80% for UK-domiciled (i.e. home) leavers who previously studied full-time;

70% for UK-domiciled leavers who previously studied part-time

50% for all other EU students

80% for Research Council funded students

Data Protection

What to do before and during the DLHE surveys

Before the survey institutions should:

• Update each student's contact details
• Let all students know that the DLHE will be happening and give them an opportunity to withdraw from it, for example by providing the HESA student collection notice.

During the survey institutions should:

• Use accurate contact details
• Use only the contact details provided by the leaver (except where given by a third party, when the details should be used as described below)
• Take notice of any objections the leaver has to being contacted
• Either not collect, or delete, information from any leaver who does not wish their destinations data to be held by the institution.

When collecting data from a third party the interviewer should be aware that:

• They should ask that the third party informs the data subject that data has been collected. The interviewer should also indicate that they can contact the institution for more information or with any concerns they might have
• If the third party gives an interviewer a contact telephone number for the leaver, this may be used, providing the interviewer states where the number has come from and gives the leaver an opportunity to object to the call at the beginning of the conversation. The telephone number may only be recorded for future contact purposes outside of DLHE if the leaver gives their permission.

Objections to being contacted or providing data can be given at any time before the data is returned to HESA. Such objections must always be recorded.

Standard text, collection notices and informing students

In order for processing to be fair, data subjects should be informed about the uses made of data. A collection notice is a piece of text which fulfils these criteria.

Student collection notice

The HESA Student collection notice should be made available to all students at each institution. This tells the students that:

• They may be contacted for first stage DLHE
• There is the possibility of follow up to the first stage DLHE
• They have the opportunity to object to further contact.

DLHE questionnaire

There is a general statement on the questionnaire, referring to the covering letter for more information.

DLHE covering letter and email

The covering letter and email for the DLHE survey

• Tells the leaver that they will be contacted again if they don't reply
• Informs the leaver that data may be collected from a third party if they can't be contacted themselves
• Gives the leaver an opportunity to refuse to give data or be contacted again
• Tells them that they may be contacted for the longitudinal survey and gives them the opportunity to object.

DLHE collection notice provided by HESA

This is included at the bottom of the covering letter and email, and describes the uses made of the data by HESA and its Statutory Customers.

DLHE collection notice to be written by institutions

Space has been allowed within the covering letter for each institution to include their own collection notice text about the use they make of the data. HESA strongly recommends that each institution composes suitable wording for, and includes its own collection notice in the covering letter, in order to protect the institution’s legal position.

Further information about informing students

Institutions cannot guarantee leavers read the covering letter and collection notices, but this does not necessarily cause a problem. The Data Protection Act 1998 says that information concerning what happens to personal data must be made available to the individuals concerned. Organisations must make every reasonable effort to provide this, but not so that it is disproportionate to the purposes for which the data is collected. Therefore, if institutions follow the above procedures, there should be no further need for concern.

Further information about data protection issues

Students can obtain further information on data protection issues from the HESA website at www.hesa.ac.uk/dataprot. The document 'Data Protection Guidance for the HESA Records' can be obtained from this page. For further information please contact the Data Protection Manager at data.protection@hesa.ac.uk

DLHE pre-survey preparation

Centrally-hosted online DLHE questionnaire

HESA offers a web-based DLHE questionnaire, which is centrally-hosted by HESA. If an institution wishes to offer this method of taking part in the DLHE survey to its graduates, then it will need to have completed the registration exercise prior to the data capture period. Details about how to register are available in the Guidance notes.

Postal mailing questionnaires and telephone scripts

Prior to each DLHE survey the HESA Institutional Liaison team will email all DLHE contacts with a set of questions about the size of their institution's DLHE target population and the mailing/telephone arrangements that they plan to use. Also, as part of this email, the Institutional Liaison team will ask for the name, address, telephone number and job title of two people at the institution to whom the DLHE questionnaires and telephone scripts should be addressed. This does not have to be the DLHE contact but can be whoever is deemed most appropriate. The distributor will attempt delivery to the first recipient and, if unsuccessful, a second delivery attempt can be made to the back up recipient.

Not supplying responses to these questions will adversely affect the smooth running of the DLHE process. If an institution does not provide addressee details for each survey, the distributor will either deliver to the address HESA provide (this will be of the last known DLHE contact as stored on the HESA system) or take the consignment back to the depot. Both situations can cause delays, which impact on the time institutions have to prepare for mailing. This also poses unnecessary administrative burdens on the printers, on HESA and on institutions, in trying to trace a missing consignment and/or arrange for re-delivery.

It is important that contacts in institutions inform their colleagues, e.g. security staff (or anyone else who may be likely to be in receipt of the delivery) that delivery of questionnaires and telephone scripts is expected. Institutional Liaison will email DLHE contacts close to the delivery date to inform them between which dates delivery can be expected (this will usually be no longer than during a period of five working days). If boxes of questionnaires and telephone scripts are accurately addressed, and institutions expect the delivery, consignments should be received within this stated delivery period.

If an institution's DLHE survey is administered by another institution/organisation, and the questionnaires and telephone scripts need to be delivered there, please provide the full mail address of this institution/organisation. In such cases the address label will include both a named 'care-of' contact and the address of the institution/organisation where the survey is administered and also the name of the institutional contact for reference. The back up contact details given should also be someone from the administering institution/organisation, so that a second attempt at delivery can be made.

The consignment will be clearly labelled - indicating the total number of boxes making up the delivery, and also the contents of each box, e.g. 1000 English questionnaires. It is important that an institution checks the consignment on delivery, as shortfalls cannot be rectified later.

HESA stock only a very small number of questionnaires and telephone scripts as contingency. This would be sufficient to provide to an institution in the event of a minor and unforeseen circumstance, should part of their own stock become unsuitable for mailing, e.g. if mailing equipment malfunctions and questionnaires are lost as a consequence. HESA does not store sufficient numbers to replace part of or an entire single institution's consignment. It is therefore imperative that institutions check the POPDLHE estimates provided by HESA and either; confirm that these are correct, or provide accurate revised estimates.

DLHE survey

Initial contact

In this period (18 April 2011 to 09 May 2011 for the April survey and from 14 November 2011 to 09 January 2012 for the January survey) the only acceptable method of data capture is the standard questionnaire and it should be completed by the graduate only. Institutions should not be undertaking any destinations surveying prior to this 'Initial contact' period.

The standard questionnaire is available in the following formats:

HTML version: An HTML version of the questionnaire is available for institutions to host on their own websites. The HTML version is available in English and Welsh and can be downloaded from the DLHE 2010/11 collection page. Institutions should not add any questions to or change the questionnaire in any way. Institutions are responsible for programming any ‘back-end’ or database structures that are required to support the use of this version of the questionnaire. Graduates should be directed to the web reference for this locally-hosted online questionnaire using the email or covering letter texts provided by HESA. Leavers should complete, date and return this version of the questionnaire on the institution’s web site. Please note that this method will no longer be available after the January 2012 survey.

Centrally-hosted online DLHE version: The centrally-hosted online DLHE questionnaire is available at dlhe.hesa.ac.uk. Graduates should be directed to this web-reference in the email or covering letter and can then follow the instructions and submit their questionnaire.

PDF version: A PDF version of the questionnaire is available in both English and Welsh and can be downloaded from the DLHE 2010/11 collection page. Institutions can email this version of the questionnaire to graduates for completion. Leavers should print the form in order to complete, sign, date and return it to their institution.

As there are many ways a leaver can complete and return a questionnaire, it is possible that an institution may receive more than one questionnaire for the same individual. In this case the institution should return to HESA the first information it receives.

Initial contact schedule

At the beginning of the first week of the 'Initial contact' period it is recommended institutions email a link to the centrally-hosted online questionnaire or their locally-hosted online questionnaire to those graduates for whom a valid and up to date email address is available. A postal questionnaire should then be sent to all those who did not complete and submit an online version, and to all those for whom an email address was not available. To allow as much time as possible for graduates to complete an online version, and therefore reduce the number of postal mailings, it is recommended the postal questionnaires be sent out towards the end of the second week. All graduates must have been contacted by the end of the 'Initial contact' period.

If institutions are not offering either of the online versions, then postal mailing could take place any time during the fortnight 'Initial contact' period.

If a leaver returns the questionnaire without completing one or more of the core questions they can be contacted by the institution in order to try and obtain the missing information and so ensure a valid return.

HESA encourages institutions to make use of the centrally-hosted online questionnaire, and it hopes over time use of this version will increase.

Covering letters/emails

HESA has produced the text of the covering letters and emails that should accompany all versions of the questionnaire. HESA also provides instructions on how to use these texts and institutions should refer to this guidance. The letter and email texts are available in both English and Welsh. The covering letter text should be printed onto institutions' own letter headed stationery and this should be sent with all postal questionnaires. The text for email contact should be used for sending a link to either the centrally-hosted version or a locally-hosted version of the questionnaire. All letter and email texts with instructions for their use can be downloaded from the DLHE 2010/11 collection page.

The text of the letters and emails should NOT be changed apart from where it is indicated either in italics or where there is an instruction. Institutions should particularly note the instruction to insert their own data protection collection notice (See Data Protection and Collection Notices section below).

Institutions can add to the introductory text in order to address particular groups of students. For example, for students who have continued to study at the institution the following text may be used:

• We are asking everyone, even those who have continued to study at the institution, what they will be doing on 18 April 2011/09 January 2012, so that the information is comparable.

When sending out postal questionnaires institutions should mail out the questionnaire, letter, and, if they use them, a reply-paid envelope. In order to offset some of the additional costs that are incurred in implementing the survey, institutions can, if they wish, find sponsorship for the survey. Statutory users and their auditors are also content that institutions should be able to include other items in the survey mailing, for example, leaflets about institutions’ graduate services. It is acceptable for institutions to include other surveys in with the DLHE questionnaire. However in deciding what might be reasonable to include in the mailing, institutions will need to be conscious of the requirement to obtain the best possible response to the survey and to meet the response rate targets.

Similarly, the text for the email should be sent with a web reference to either the centrally-hosted online questionnaire or a locally-hosted version of the questionnaire. Institutions can include web references, e.g. to their own graduate services, or to other surveys, again bearing in mind the requirement to obtain the best possible response to the DLHE survey.

Direct contact with the leaver during the initial contact period

If the institution has direct contact with the leaver during this period e.g. they visit the careers office, the leaver can be handed a standard questionnaire to complete whilst they are there. A copy of the covering letter must be handed out with the questionnaire.

If an institution's graduation ceremony falls during this period then graduates can be handed a questionnaire at graduation. During the initial contact period the standard questionnaire along with a copy of the covering letter must be used.

Follow-up contact

Follow-up contact can be made between 09 May and 27 June 2011 for the April survey and between 09 January and 09 March 2012 for the January survey. Emailing a web reference or sending out a postal questionnaire for a second time is optional, as institutions may find other methods generate a better response, e.g. institutions may find that the practice of one postal mailing together with telephone follow-up is the best way for them to meet the response rate targets. If an institution intends to conduct a second postal mailing then it will have needed to inform HESA of its intention to do so at the pre-survey preparation stage of the survey. Again, the texts for covering letters and emails for follow-up contact are available in English and Welsh. Possible alternative follow-up methods are:

• Reminder postcards (which can be designed and printed locally to reflect the institution’s own branding).
• SMS text messages to graduates to remind them (if the institution believes that this is appropriate).
• Email graduates with a reminder.

Each time a questionnaire is sent, either by post or electronically, it must be accompanied by the information contained in the covering letter or email.

Direct contact with the leaver during the follow-up contact period

If an institution has direct contact with the leaver during this time, e.g. they visit the careers office or the graduation ceremony falls within this period, the leaver can be interviewed using the telephone script or handed the printed version of the questionnaire to complete whilst they are there. A copy of the covering letter must be handed out with the questionnaire.

Telephone survey follow-up

Any telephone follow-up must be conducted during this period. As institutions are required to undertake initial contact using the standard questionnaire, it is not possible to only conduct a telephone survey, which features a reduced set of questions.

Copies of the telephone script together with postal questionnaires are delivered to institutions during March and September and so institutions should inform HESA (as part of the pre-survey preparation) of their intention to conduct telephone follow-up. If an institution finds it is short of telephone scripts, photocopying of the DLHE telephone script is permissible with the HESA's consent, provided the forms are used for DLHE purposes only.

Where a telephone call is made the caller needs to explain by way of introduction that:

• They are ringing on behalf of the institution to find out whether the former student has found employment and/or further study following completion of their course.
• Information is used both to advise current students about opportunities and also to feed into a national statistical survey conducted each year to see what happens to students upon leaving higher education.
• The information on this form will be used by your institution and coded information will be sent to the Higher Education Statistics Agency. Further details and contact information are supplied as part of the covering letter.
• They may, if consent is given, be contacted again in the future as part of a follow-up survey. This survey will be conducted by a third party and so their details may be passed on to this third party.
• They need to find out what the former student was doing on the census date.

Where a telephone call is made, obtaining the response directly from the leaver is desirable; if contact is made with someone other than the leaver, this is acceptable, provided that the contact is sufficiently well informed to be able to answer the core questions, which are indicated by a ring around the number on the telephone script. It is not so much the source of information that is important as the quality of the information itself. However, care must be taken when obtaining information from such third parties (see Data Protection and Collection Notices section below).

In determining whether or not information is robust enough for inclusion in the 2010/11 DLHE return, the contact must be able to:

• Supply all the relevant information (i.e. answers to the core questions ringed on the telephone script) AND
• Must be confident that it relates to the leaver’s position on the census date.

Where information for completion of the core questions on the telephone script comes from a source other than the leaver or a contact at the leaver’s home, then additional audit proof is required to show that the individual conducting the survey has collected or confirmed their source of data after the initial contact period. For example, a confirmatory signature against the date that each individual student’s details were confirmed, since a standard questionnaire completed by the leaver is the only acceptable method of data capture prior to this time.

Survey accessibility

Graduates with visual impairments or other disabilities that may restrict them from completing the paper questionnaire or an online version can complete the survey by way of telephone interview, or by using the PDF version of the questionnaire. The telephone interviewer can use the questionnaire instead of the telephone script to enable the graduate to answer all the questions, and not just the reduced set of questions on the telephone script. Alternatively, a graduate can print the PDF version of the questionnaire as a large print version or in a way that suits their requirements in order to complete, sign, date and return it to the institution. The PDF version of the questionnaire can be downloaded from the DLHE 2010/11 collection page.

Additional/sponsored questions

Institutions’ own or sponsored questions can be used with the telephone script but institutions will need to be conscious of the requirement to obtain the best possible response to the survey and to meet the response rate targets. The sponsored questions should perhaps be asked at the end of the telephone call, after institutions have ensured they have obtained sufficient information (at least all of the core questions answered) to provide a valid DLHE return.

Other informed source

Information obtained from other informed sources, including academic departments or employers, is also acceptable as a method of data capture. However the contact must be able to:

• Supply all the relevant information (i.e. answers to the core questions ringed on the telephone script) AND
• Must be confident that it relates to the leaver’s position on the census date.

The individual conducting the survey must be able to provide additional audit proof to show that they have collected or confirmed their source data during the follow-up contact period. For example, a confirmatory signature against the date that each individual student’s details were confirmed, since a standard questionnaire completed by the leaver is the only acceptable method of data capture prior to this time.

Information from third parties should only be used where the institution has tried to make contact with the student directly in both the initial and follow-up contact period without success.

Note that the core questions require that the contact will know what the leaver is doing with respect to both employment and further study on the census date.

Please note also that for any records where information has been provided by a third party the graduate should be excluded from the longitudinal follow-up. 

Issues associated with the collection of data from third parties

The two main issues associated with collecting data from third parties are ensuring the data is accurate, and that leavers are informed of what happens to their data.

The accuracy principle is not contravened because:

• Steps are taken to ensure the data are accurate for the purposes, e.g. validation and audit
• The DLHE collection allows a record to be marked as containing data obtained from a third party.

So that the collection of the data is fair, institutions should follow the procedures above and use the standard collection notice text.

HESA wrote to the Information Commissioner's Office describing the position. The IC's office replied with two recommendations: to list the statutory bodies in receipt of the data on the collection notices, and to allow leavers to refuse to participate in the DLHE by replying in writing. Copies of this correspondence can be obtained on request from the Data Protection Manager (data.protection@hesa.ac.uk).

Explicit refusals

An explicit refusal is where a leaver either sends back the questionnaire marking it as a refusal, emails, or writes a letter including a statement that they do not wish to be included in the survey. It can result from a telephone conversation where the leaver verbally states that they do not wish to be included in the survey. A leaver who does not respond in anyway or is not contactable should not have a DLHE record returned to HESA, and is not regarded as having explicitly refused. Leavers with DLHE records identified as METHOD = 9 ‘Reply received explicitly refusing to provide information’ will be counted towards an institution’s response rate.

Audit trail, record keeping and monitoring

All data capture methods used must result in a robust audit trail. This will consist of one of the following:

Standard questionnaire (printed and PDF versions): evidence comprising the completed questionnaire, signed and dated by the leaver himself or herself.

Standard questionnaire (locally-hosted version): evidence demonstrating that the questionnaire was completed and dated by the leaver. This might for example mean that the institution’s system is set up in such a way that a read-only copy of the data keyed by the leaver is retained.

Standard questionnaire (centrally-hosted version): institutions should keep a copy of the files downloaded from the HESA archive.

Telephone survey: evidence comprising a completed telephone script, which was completed during the follow-up contact period. Where information for completion of the core questions on the telephone script comes from a source other than the leaver or a contact at the leaver’s home, then additional audit proof is required to show that the individual conducting the survey has collected or confirmed their source data after the initial contact period. For example, a confirmatory signature against the date that each individual leaver’s details were confirmed, since a standard questionnaire completed by the leaver is the only acceptable method of data capture prior to this time.

Other informed source: procedural evidence to demonstrate that the data

• Answers the core questions ringed on the telephone script AND
• Relates to the leaver’s position on the census date.

The individual conducting the survey must be able to provide additional audit proof to show that they have collected or confirmed their source data after the initial contact period. For example, a confirmatory signature against the date that each individual leaver’s details were confirmed, since a standard questionnaire completed by the leaver is the only acceptable method of data capture prior to the follow-up contact period.

Keeping DLHE data

The data should be kept only as long as is necessary. If the purpose is research then data may be kept for a long time. The completed questionnaires need not be kept beyond the time that the possibility of an audit might be carried out. Institutions should therefore keep either a hardcopy or electronic version of the questionnaire or telephone script for each individual leaver for three years after the data is returned to HESA. There is no need to anonymise the data. However, the data should not be used in a way that will affect the individual concerned

Longitudinal follow-up

An explicit refusal at the early DLHE stage automatically excludes the graduate from longitudinal follow-up. Institutions should however establish a central record for any graduate who indicates in Section F of the DLHE questionnaire that they do not wish to be contacted in the future as part of a follow-up survey. Also as a graduate can contact the institution at any point between the early DLHE and the longitudinal follow-up three years later to indicate they do not want to be further involved in DLHE, this record should be accessible to a number of institutional colleagues in Alumni, Careers Service etc., as there will be more than one institutional point that a graduate may contact. This information will then be readily available for institutions to refer to when the sample selection is drawn, and these graduates can be excluded.

Please note also that for any records where information has been provided by a third party the graduate should be excluded from the longitudinal follow-up.

Who to contact

If you have any queries about DLHE methodology, please contact the Institutional Liaison team at HESA (liaison@hesa.ac.uk).