Graduates who select that they are engaged in a certain type of employment, and that their place of work is in England, Scotland, Wales or Northern Ireland, are asked to provide the postcode for their place of work. To improve the quality of postcode data, validation was implemented in year two to check the first two digits of the postcode entry and to prevent graduates from entering an invalid postcode in the free-text field. It is important to ensure that this validation is improving the data, without increasing non-response. After postcode, graduates are asked to provide the city/town of their place of work. If they have selected that they do not know their postcode, or if they provide a short postcode, then city/town is asked as a mandatory question. The quality of this data is therefore linked to postcode, as it is important to know if graduates are dropping out due to this question. Both elements of employment location are analysed in the following section.
Assessment of postcode validation
Year one data was run through an artificial validation to determine the levels of invalid postcodes that were received. This allows for a more thorough analysis of the data to determine if there have been quality improvements. The following table highlights the postcodes received in Cohort D. It compares year one against year two and combines the postcodes for all employment types. Additionally, the split by completion mode allows a comparison of mode effects, which will be useful in determining if there are mode related quality issues at play. Invalid postcodes from both the short and long groups are compiled into one section/group. The rightmost column gives the percentage difference, colour-coded to indicate positive (green), neutral (yellow) or negative (red) outcomes of the validation changes.
Table 24 Summary of the differences between postcodes collected in Cohort D of year one and year two of the Graduate Outcomes survey, split by completion mode
|Mode||Year 1||Year 2||% Difference|
*Invalid postcodes in year two are anomalies. These are as a result of either telephone interviewers changing responses after validation or graduates dropping out of the online survey at the time of answering the postcode question and their responses therefore not passing validation. Of these responses, of which numbers are small, many would have passed validation if formatting was corrected.
Differences between the postcode results in year one and two highlight generally positive outcomes as a result of the implementation of postcode validation. There has been an increase in the number of long postcodes provided and a reduction in the number of invalid postcodes, as it was not possible in most circumstances for a graduate to enter an invalid postcode after the implementation of validation. For telephone, this difference is larger and is potentially due to validation reducing interviewers entering incorrect postcodes, either as a result of mishearing graduates or graduates providing an incorrect postcode. There were small numbers of postcodes in both modes that did not pass validation due to either interviewers changing formatting after the validation or graduates not moving forward in the survey after entering their postcode, but these numbers have still reduced significantly. In terms of short postcodes, more were provided in the telephone mode in year two, whereas numbers decreased very slightly for online completions. Both outcomes are marked as neutral in the table, as although it is positive to see an increase in postcode provision overall, these values could be influenced by either more graduates providing a long postcode or more graduates deciding not to answer at all. Indeed, the percentage of graduates who did not answer the postcode question increased slightly in both modes, but this is outweighed by the increases in postcodes provided in total. There has also been a decrease in ‘Don’t know’ responses in both modes, with a significant decrease of more than 5 percentage points for telephone interviews.
Assessment of town / city question
After the postcode question, graduates are asked to provide the town, city or area of their place of work. This question was simplified in year 3 of the survey to ask about the graduates nearest town or city, but the following assessment is based on the responses received to the question as it was previously worded. If the postcode question is not answered, or a short postcode is provided, then the town/city question becomes mandatory. The table below indicates whether town/ city was filled in and whether it was a mandatory or optional question for the graduate. This helps to give an indication of the overall quality of location data and will help in ensuring that changes to the postcode question have not had a negative impact on other areas of the survey.
Table 25 An indication of the response provided to the town/city question in cohort D of year one and year two, and splits by mandatory or optional, where postcode was answered
|Year 1 Cohort D||Year 2 Cohort D||Difference|
|City mandatory and answered||48.12%||46.21%||-1.91%|
|City mandatory and NOT answered||0.23%||0.23%||0.00%|
|City optional and answered||51.39%||53.19%||1.79%|
|City optional and NOT answered||0.26%||0.37%||0.11%|
Town/city became optional for more graduates in year 2, again highlighting that more graduates provided a long postcode than the previous year. Positively, most of these graduates still provided a location regardless of the question being optional for them. Of the mandatory group, the percentage of graduates who did not answer remained the same and at a low percentage, highlighting that although the percentage of graduates who answered dropped there was no increase in non-response to the question for mandatory graduates, and this was instead due to an increase in long postcodes being provided.
Conclusions on location data quality
Overall, the quality of location data has improved. Although there appeared to be a very slight increase in non-response to postcode, the values are very small and the improvements in the quality of data being received is clear, with large increases in long postcodes being provided and a reduction in responses of ‘Don’t Know’. Equally, there was no increase in the percentage of graduates who did not provide a town/city when it was a mandatory question, highlighting that dropout was not increased at other questions due to validation. City also became optional for more graduates, which is helpful, as postcode data is more efficient to analyse and reduces the risk of bias introduced during processing, compared to the free-text town/city question. It is therefore clear that overall, the quality of the postcode data has seen major improvements since year one, both due to validation improvements and perhaps also as a result of changes in the working patterns of graduates.