AOR major review for Scotland and Northern Ireland - summary of responses

HE providers that are or may be required to submit data to meet the proposed requirements were invited to submit burden assessments as part of their responses to the consultation. These scores are grouped into bands with the following definitions:

Setup:

0: No impact

1-3: Almost no change to processes or systems. Can be fitted into business as usual against a known release window. Limited training or guidance required.

4-7:  Minor to medium changes to systems and processes. Difficult to resource. Date for go-live sub-optimal. Requires bespoke training.

8-10: Significant change to systems and/or processes. Detrimental impact on ability to do other activities. Cannot be serviced from existing capacity. Release date very challenging. Requires training / re-skilling.

Run:

0: No impact

1-3: Business as usual (BAU). Operated through processes and systems. No discernible impact on current operating model.

4-7: Increased resources. Sustained opportunity cost. Complexity in planning. Requires specialist support.

8-10: Sustained increased resources required. Conflict with other business priorities. Cannot be systemised – lots of manual checking. Requires subject matter expert (SME).

54% of respondents agreed that the scenarios were an accurate reflection of TNE activity, 38% suggested that this was partly the case, and one respondent was unsure. 69% felt that the registration, delivery, and awarding arrangements were as expected, against 31% who were unsure. 77% agreed that the proposed definitions offered a useful framework for classifying TNE provision, against 23% who neither agreed nor disagreed.

Respondents discussed a number of models of TNE provision that fell outside of the example scenarios in their responses, including the following:

• Blended learning
• Distance learning with only occasional attendance, whether in the UK or overseas
• Flying faculty arrangements where provision is delivered overseas by UK provider staff
• PhDs jointly supervised by multiple providers
• Provision involving multiple partners overseas
• Distance learning with an approved local partner
• Variations on multiple and combined awards
• TNE provision where aspects of the programme are based in the UK but it is not considered TNE due to falling into the coverage of the existing Student record

One respondent agreed that the proposed definitions matched their current TNE provision, while another was concerned that they use different definitions within their provider. It was also noted that some of the above examples of provision were missing from the definitions and scenarios. One respondent noted in very general terms that it was important for the definitions of types of TNE to be clear and consistent across the UK for the purposes of sector comparisons and benchmarking. It was suggested that the definitions be used across non-TNE collaborative provision as well.

The sector body respondent agreed that the proposed framework of programme and venue types were broadly representative of TNE arrangements in the sector, but noted that different awarding structures and articulation arrangements were not fully captured in the scenarios.

Registration and awarding body

62% of respondents expected the registration and awarding body arrangements to be consistent for all students on a given course, versus 23% who disagreed and 15% who were unsure. 46% of respondents supported the proposal for the data to be collected on the Course entity, 23% opposed the proposal, and 31% were unsure.

38% of respondents scored the set-up burden of submitting data on who registers TNE students in the “4-7” category and 46% of respondents scored the run burden in the “1-3” category.

54% of respondents could see a benefit to submitting data on the specific organisation(s) who register TNE students, with 38% unable to see a benefit and one respondent undecided.

46% of respondents scored the set-up burden of submitting data on which specific organisation registers TNE students in the “4-7” category and 38% of respondents scored the run burden in the “1-3” category.

Comments in support of the proposal noted the utility of data on registration and awarding bodies for TNE students, with one respondent noting that a UK-wide approach would be preferred to enable comparisons and benchmarking. The concerns raised by respondents included the risk that the same course could be offered by distance learning for both TNE and non-TNE students on different timescales; that some collaborative programmes could be set up with different registration arrangements; and the fact that the proposals were reliant on the free-text venue name field, raising issues of consistency between providers and years.

There were calls for clarification around requirements when a student is registered with more than one provider, consideration of whether the same requirements could be added to the Student data model for non-TNE collaborative provision, and for additional detail on the intentions of the users of the data.

Course delivery

46% of respondents scored the set-up burden of submitting data on who delivers the course to TNE students in the “4-7” category and 46% of respondents scored the run burden in the “4-7” category as well.

Respondents flagged concerns that the data is not necessarily centrally stored or managed and that development work would therefore be required; that variation in courses offered would make the data burdensome to collect; and that the precise definitions around different types of programme could necessitate duplication of data.

One respondent asked that the data be aligned more closely to how it is structured in the Course role entity in the Student record, noting that it would be burdensome to add separate values solely for the purposes of the AOR.

Distance learning

38% of respondents scored the set-up burden of submitting data on the location of distance learning TNE students in the “4-7” category and 46% of respondents scored the run burden in the “4-7” category as well.

One respondent noted in support of the proposal that they already collect and hold the necessary data in their student record system. One noted that this was in line with the current approach for the AOR as well. Others expressed concerns to the effect that the location of students is difficult to track, particularly for TNE students, and that work would be needed to improve the corresponding processes. One respondent noted that they have no internal use for the location of these students and that they felt the burden was disproportionate to the benefit. Two providers highlighted the difficulty of collecting location data for distance learners due to the arm's length nature of their relationship with their provider.

Additional clarity was requested around the intended use of the data and the guidance for distance learners undertaking collaborative provision.

Venue name

46% of respondents scored the set-up burden of submitting data on venue names in the “4-7” category and 46% of respondents scored the run burden in the “4-7” category as well.

Respondents made several comments in support of the proposal, including their confidence that this would be straightforward to maintain for new and continuing students, and that it could easily be derived from the student location. Concerns were raised by respondents as well, primarily that this data would not be comparable without quality assurance or a list of valid options from which to select – and consequently, that data without such safeguards would be inconsistent to the point of being unusable, and therefore difficult to provide a use case for. There were also concerns that some form of venue name would be required for distance learners.

Qualification subject

38% of respondents scored the set-up burden of submitting data on qualification subjects in the “1-3” category and 54% of respondents scored the run burden in the “1-3” category as well.

A significant proportion of respondents noted that they already held subject level data on programmes or awards and could meet this requirement. Other respondents highlighted the burden of readying systems and processes to provide the data, with one noting their concern that research activity could be more difficult to code.

Replacing the TYPE field

46% of respondents scored the set-up burden of submitting data on programme and venue types in the “4-7” category and 46% of respondents scored the run burden in the “4-7” category as well.

The majority of respondents expressed their support for the proposal to classify TNE provision by programme and venue type, including comments to the effect that the definitions were useful and would bring additional consistency and comparability to the data.

One concern raised by respondents was around the limitations of these definitions, noting that some types of TNE activity are excluded. Other respondents referred to the burden of collecting the data on an ongoing basis, the work involved to redefine their suite of programmes in these terms and to ensure that they were used consistently within their provider, and concerns that the data they currently held in this regard was neither systemised nor sufficiently robust.

There were also requests for additional clarity around the definitions and the permissible combinations of programme and venue type. One respondent suggested that the definitions would be of use for categorising non-TNE provision as well.

The sector body that responded to the consultation agreed that the proposals would allow for more accurate classification of TNE partnership activity and better data on the locations where this activity takes place.

Headcounts

46% of respondents scored the set-up burden of submitting data on separate headcounts in the “4-7” category and 69% of respondents scored the run burden in the “4-7” category as well.

Comments in support of the proposal included that some respondents held the necessary information in their systems already, and that the decision to continue to exclude dormant students was welcome. One respondent noted their dependence on software suppliers provisioning the necessary systems.

A number of respondents raised concerns, including:

• The burden of quality assuring the different headcounts, noting that TNE provision follows a different academic year cycle than domestic provision, and that processes around registration, enrolment, continuation, and assessments would all need to be tightened
• Difficulties interpreting and tracking dormancy, particularly for distance learners, as well as shifts from full to part-time modes or vice versa
• The need for more justification around the requirement for the individual headcounts and omission of data on withdrawals
• That including students in multiple headcounts may complicate quality assurance efforts and use of the data

Course data

46% of respondents scored the set-up burden of submitting data on courses in the “4-7” category and 46% of respondents scored the run burden in the “1-3” category.

Several respondents noted in support of the proposal that they already collected the necessary data, or that it was aligned with the fields collected in the main Student return and would therefore not present an issue. A few concerns were raised as well, with one respondent stating that some work would be required to understand and embed the new definitions and tighten up processes, and others highlighting the potential difficulty around determining the timings of awards and the circumstances under which new versions of the Course entity will be required.

Provision data

Respondents were evenly split on the set-up burden of submitting data on provision, with 31% landing in each of the “1-3”, “4-7”, and “8-10” categories. 38% of respondents scored the run burden in the “4-7” category.

One respondent stated that they held the necessary data centrally already and could meet the requirements, while another raised concerns that some work would be required to understand and embed the new definitions and tighten up processes. One respondent questioned the justification for the requirement, highlighting their reservations about the burden of supplying the data versus the benefit. There was also a call for additional guidance on how to approach the coverages of the respective student records, particularly in relation to the movement of students between the AOR and the main Student return.

Respondents showed a clear preference for continuing to collect TNE data through the Aggregate Offshore record, with 77% selecting this option versus 15% who wished to combine the TNE data with the Student record.

54% of respondents scored the set-up burden of continuing to submit TNE data through the AOR in the “4-7” category and 54% of respondents scored the run burden in the “4-7” category as well.

62% of respondents scored the set-up burden of combining TNE data with the Student record in the “8-10” category and 46% of respondents scored the run burden in the “8-10” category as well.

Respondents who favoured retaining the AOR for data on TNE provision raised a number of points:

• Changes to the AOR would entail a smaller degree of burden; submitting the data via the individualised return would require significant work, not least because of the differences in academic year structures and how TNE students register or enrol, and the challenges of capturing personal characteristics data
• There are too many differences between the requirements for Northern Ireland and Scotland and those for England and Wales to workably combine the returns
• The burden of system changes to enable submission of individualised data on TNE students, including where data is not centralised
• Retaining the separate returns makes resource management more straightforward
• Confidence that an aggregate data model can meet requirements around quality assurance of TNE provision, is more adaptable to differing overseas academic contexts, tried and tested with overseas partners and authorities, and able to provide information on sector trends for comparison and policymaking purposes
• The need for a clear justification of any requirement for individualised data and clear benefits to the sector from its availability
• The need for a pause in changes to the Student record to allow the sector to adapt to the changes from Data Futures

Respondents who preferred moving TNE provision to the Student record in line with the approach proposed for England and Wales offered the following arguments in favour:

• Having two separate returns with conflicting submission deadlines creates additional work; combining the records would allow for a streamlining of processes and reduction in burden
• Although submitting individualised data will entail more work than the aggregate return, including changes to infrastructure, this would not have a particularly significant impact on processes

One respondent suggested that they were open-minded about revisiting the decision once the Data Futures changes to the Student record had been thoroughly embedded across the sector.

69% of respondents indicated that their provider would be able to submit data on TNE provision once year, compared to 23% who indicated that they would be able to submit the data twice a year to match the Student submission timescales.

38% of respondents scored the set-up burden of submitting TNE data once per year in the “4-7” category and 46% of respondents scored the run burden in the “4-7” category as well. 62% of respondents scored the set-up burden of submitting TNE data twice per year in the “8-10” category and 54% of respondents scored the run burden in the “8-10” category as well.

Those respondents who preferred to submit the data once per year highlighted their concerns that multiple submissions would entail significant additional burden and that consequent resourcing constraints may compromise data quality; that most of the proposed aggregate data would not confer any additional benefit by being collected and submitted twice; and that without an obvious benefit to increasing the frequency of returns, the justification for any requirement to submit twice per year would have to be clear.

Those respondents who preferred to submit the data twice per year commented that their TNE data is already held in the same system as the Student data, and that if the decision were taken to combine the TNE data with Student data, it would make the most sense to submit all of that data with the same frequency. They did however note that aligning the submission timescales would require work to align processes and timescales for academic activity.

62% of respondents agreed with the statement that it is important to keep a UK-wide approach, against 31% who neither agreed nor disagreed, and one respondent who disagreed.

Those respondents who agreed with the statement made a number of points in support of maintaining a UK-wide approach:

• Key data items, for example on types of TNE provision, need to be consistent across the UK to allow for meaningful comparisons
• High-level contextual data on TNE provision is sufficient for comparisons of activity and benchmarking across the sector
• Deviation across the UK has the potential to result in exclusions or negative implications with regard to league tables for providers in Northern Ireland and Scotland
• Although UK-wide equivalence is important and has benefits, it does not justify the adoption of an individualised data model across the UK. Instead, care should be taken to ensure that any individualised dataset can be aggregated for equivalence to the approach taken in Northern Ireland and Scotland

One respondent who agreed with the statement made the case for an individualised data model across the whole UK, arguing that retaining the aggregate data model would limit the future capability of the TNE dataset, despite the increased burden associated with submitting individualised data. They also expressed an interest in submitting a subset of the individualised student data requirements, rather than in line with all of the proposals made for providers in England and Wales.

Those respondents who neither agreed nor disagreed with the statement noted that although sector comparisons of data had the potential to be useful, returning individualised data would place considerable burden on submitting providers and their partners overseas. One respondent expressed the view that sufficient cross-sector comparability could be maintained with an aggregate submission, and another was concerned that the benefits of submitting individualised data had not been adequately articulated.

The respondent who disagreed with the statement emphasised the need for equivalence at a high level, but gave the view that this could be achieved by amending the existing aggregate record, and therefore without placing disproportionate burden on submitting providers. They also noted that the proposals have arrived during a time of considerable burden and change across the sector.

Comments in support of the proposals noted the greater usability of the resultant data, highlighting the benefits that subject-level data and improved definitions for types of TNE provision would bring. It was also noted that this would need to be consistent with data on non-TNE students in order to be of value.

A number of concerns were raised by respondents:

• More detail is needed on the data specifications, the justification for the data, and the intended onward use of the data
• That the current level of change across data returns more generally means that expanding the AOR presents an additional burden to submitting providers, Statutory Customers, and Jisc. It would be better to delay the implementation of new requirements until the changes to systems and business processes under Data Futures have been embedded across the sector
• The burden of changes to systems and processes is significant, including where aggregate data is compiled by manual processes. Extending the time allowed for the sector to implement the changes would give providers and software suppliers more room to develop and test solutions and therefore reduce burden

One respondent noted their dependence on their software supplier to provision a solution that could meet the requirements. Another reiterated their opposition to requiring individualised data on TNE students, emphasising the associated burden and their concerns that the benefits would not be proportionate.

The sector body which responded to the consultation was broadly supportive of the proposals to enhance the existing Aggregate Offshore record, arguing that the same approach should be taken across the UK. They gave the view that an aggregate record could be used to monitor the quality of transnational education, while retaining a UK-wide approach and ensuring familiarity on the part of the overseas organisations expected to submit much of the data. They noted their concern that requirements for individualised data run the risk of straining and potentially creating barriers to international partnerships.

Several respondents requested that the data requirements be finalised and communicated to the sector – and software suppliers – with as much notice as possible.

Question 3: What is your organisation?

Question 6: In what capacity are you responding to the survey?

Question 7: Does your organisation currently submit data to the Aggregate Offshore record?

The sector group response was from the Quality Assurance Agency.

One provider in Scotland does not submit to the AOR so they along with QAA answered “No” to question 7.

Question 8: Do these scenarios accurately reflect TNE provision in your experience? For those completing a response on behalf of a provider, are the combinations of programme type and venue type ones that you recognise at your own provider?  

Question 9: Are the registration, delivery, and awarding arrangements as expected in each case?

Question 11: To what extent would you agree that the proposed definitions provide a useful means of classifying TNE provision?  

New data items required

Registration and awarding body

Question 13: Would you expect these arrangements to be consistent for all students on a given course?

Question 14: Do you agree that registration data should be collected on the Course entity?

Question 15: Please complete your provider’s burden assessment for returning data on who registers TNE students

Question 16: Would you see a benefit to submitting data on the specific organisation(s) who register TNE students?

Question 17: Please complete your provider’s burden assessment for returning data on which specific provider(s) register TNE students

Course delivery

Question 19: Please complete your provider’s burden assessment for returning data on who delivers the course to TNE students

Distance learning

Question 21: Please complete your provider’s burden assessment for returning location data for distance learning TNE students

Venue name

Question 23: Please complete your provider’s burden assessment for returning venue names

Qualification subject

Question 25: Please complete your provider’s burden assessment for returning data on the qualification subject(s) of TNE students 

Replacing the TYPE field

Question 28: Please complete your provider’s burden assessment for returning data on programme and venue types for your TNE provision

Headcounts

Question 30: Please complete your provider’s burden assessment for returning data on separate headcounts for your TNE provision

Existing data items required

Course data

Question 32: Please complete your provider’s burden assessment for returning the required course data for TNE students

Provision data

Question 34: Please complete your provider’s burden assessment for returning the required provision data for TNE students

Aggregate versus combined TNE and Student record

Question 36: What is your preferred method of collecting TNE data? 

Question 37: Please complete your provider’s burden assessment for continuing with the Aggregate Offshore record

Question 38: Please complete your provider’s burden assessment for combining TNE data in the Student record

Timing of the return

Question 40: How many times a year would your provider be able to submit TNE data?  

Question 41: Please complete your provider’s burden assessment for submitting TNE data once a year

Question 42: Please complete your provider’s burden assessment for submitting TNE data twice a year

Closing feedback

Question 44: To what extent do you agree or disagree with this statement:

“It is important to keep a UK-wide approach to the collection of TNE data”