Skip to main content

Beta operational documents

This page provides key operational information to Beta participants and will be updated with any new information throughout the Beta pilot.

If you have any queries please contact the Liaison team by email: [email protected], or phone: +44 (0) 1242 211 144.

Timescales

Collections

The Beta pilot will be divided into two collections, each collecting data for one reference period. The timescales for each of the collections are below.

 

 

Collection 1: Beta 2018/19 RP1

Collection 2: Beta 2018/19 RP2

Reference period start date

1 August 2018

1 May 2019

Reference period end date

30 April 2019

31 July 2019

Sign-off deadline and dissemination date

31 May 2019 (no sign-off required)

5 September 2019 (sign-off required)

 

Reference period start and end dates

The reference period start and end dates define the period of interest for quality assurance, sign-off and dissemination for a collection.

Only data that is in-scope during the reference period start and end dates will be included within the reports for each collection and so only this data will be signed-off and disseminated for the collection.

 

Sign off deadline and dissemination date

Quality assurance should be undertaken throughout the collection but must be completed, and any quality rules outside of tolerance resolved, by the sign-off deadline and dissemination date.

Providers can resubmit data as many times as is neccesary ahead of this date.

Data must be signed-off by the head of provider by the sign-off deadline in order to be included within the dissemination.

For collection 1, the sign-off functionality will not be available and so any data that has reached the ‘Complete’ stage within the HESA data platform will be included in the dissemination.

For collection 2, the sign-off functionality will be available and providers will need to sign-off their reference period 2 data by 5 September 2019.

For the purposes of the Beta pilot, data will be sent to statutory customers but will not be used for any regulatory or statutory purposes.

The HESA data platform

Accessing the HESA data platform

The HESA Data Platform Collection Portal can be accessed here: https://collection-beta.hdp.hesa.ac.uk

The HESA Data Platform Specification Portal can be accessed here: https://specification-beta.hdp.hesa.ac.uk

Please note that if your head of provider has not signed the Beta Pilot Participation Agreement, access to the HESA Data Platform will not be granted.

Access to the HESA Data Platform is managed through the HESA Identity System (IDS). Users within the IDS have roles assigned to them which govern their access to the HESA Data Platform.

Below is a list of the roles within IDS that are relevant to the Beta Pilot with associated descriptions.

There are two categories of user as outlined in the Pilot Participation Agreement: Lead User and Data Futures Pilot Provider Participant User. The terms and conditions for these users are detailed in Schedules 3 and 4 of the Pilot Participation Agreement respectively. The below summary descriptions of the roles also indicate which terms applies to which role.

The HDP Student Lead User is responsible for assigning roles to any colleagues within their provider. The available roles are designed to give different levels of access to the HESA Data Platform meaning that access can be restricted. Users should be given the minimum access required in order to perform the tasks allocated to them by their provider.

Further details regarding each role, including terms and conditions can be found in the HESA Identity System roles for the HESA Data Platform document.

If further roles are required, information regarding these will be added and this will be communicated with providers.

For guidance on using IDS, including how to grant roles to colleagues, please see the HESA Identity System user guide.

 

Role

Description

Terms and Conditions for the role

HDP Student Lead User

The role holder represents the main point of operational contact for the HESA Data Futures Beta Pilot. Whilst this role doesn’t provide the ability to access any HESA system it allows the user to grant the HDP Student Admin, HDP Pilot Participant, HDP Data Returns Officer, HDP HEP Issue Resolver and HDP Data Returns API roles which would grant access to the HESA Data Platform.

 

It may not be necessary for each provider to have an assigned user for each of the above roles. Roles that are not essential for participation in the Data Futures Beta Pilot are marked as optional below.

 

There must only ever be one HDP Student Lead User per provider.

Schedule 4 of the Pilot Participation Agreement.

HDP Student Admin

(Optional)

This role can be granted by the HDP Student Lead User to undertake the role administration on behalf of the HDP Student Lead User. Whilst this role doesn’t provide the ability to access any HESA system it allows the user to grant the HDP Pilot Participant, HDP Data Returns Officer, HDP HEP Issue Resolver and HDP Data Returns API roles which would grant access to the HESA Data Platform.

Schedule 3 of the Pilot Participation Agreement.

HDP Data Returns Officer

The role holder can supply data to and retrieve data from the HESA Data Platform Student record via the Collection Portal. They have full access to the data submitted by the provider and the ability to make changes to it.

They are able to raise, respond to and resolve data issues on behalf of the provider.

Schedule 3 of the Pilot Participation Agreement.

HDP HEP Issue Resolver

(Optional)

The role holder is not able to supply data to or edit data on the HESA Data Platform but is given read only access to the data in raw form and via reports.

They are able to raise, respond to and resolve data issues on behalf of the provider.

Schedule 3 of the Pilot Participation Agreement.

HDP Data Returns API

(Optional)

This role allows the role holder to upload data to the HESA Data Platform Collection Portal via the API.

The role holder cannot access the HESA Data Platform and so cannot view the data submitted as raw data or in reports.

Schedule 3 of the Pilot Participation Agreement.

HDP Pilot Participant

This role allows users to access the HESA Data Platform Specification Portal which contains the specification of the data to be returned, including data model, data dictionary, xsd and quality rules.

Schedule 3 of the Pilot Participation Agreement.

HDP Accountable Officer

The role holder is not able to supply data to or edit data on the HESA Data Platform but is given read only access to the data in raw form and via reports.

When a collection within the HESA Data Platform reaches the sign-off stage, holders of this role will be issued the sign-off form.

There must only ever be one Accountable Officer user per organisation.

Schedule 3 of the Pilot Participation Agreement.

HESA data platform functionality

At the start of the Beta pilot the HESA data platform will have the following functionality:

Portal

Functionality

Description

Specification portal

Data items

The specification for all fields to be submitted by providers including coverage, field specification and guidance.

This also includes the specification for all derived fields.

XSD

The schema file for the collection can be downloaded from the specification portal.

Quality rules

A list of all active quality rules including technical expression and plain English description.

Collection portal

File upload

Providers can upload files to the collection portal via an API or manual upload.

Data quality assurance

A quality rules report will be available containing the validation results.

Each quality rule will have a defined tolerance and where your data is outside the tolerance for a rule, this will need to be resolved in order to progress with your submission.

If you need a change to the tolerance you will be able to raise an issue requesting a tolerance change which will be approved either by HESA or the relevant statutory customer.

Data supply

The data supply files are a set of downloadable CSV files showing the data that is in-scope for the collection. Data supply is a zip file containing a csv file for each entity.

Credibility reports

The credibility reports are a series of online tables which summarise the data submitted by providers using different filters. For example, a table showing student numbers by mode and level of study. A subset of these will be available at the start of Beta and will be added to throughout the pilot.

Upcoming HESA data platform functionality

Additional functionality will be added to the HESA data platform as the pilot progresses.

Portal

Functionality

Description

Collection

Unused record report

This report shows all data that is not in-scope for any open collection; for example, data submitted with the dates indicating the activity will take place in the future.

Duplicate primary keys report

This report shows the records submitted in a file with the same primary key; for example, two student records with the same student identifier (SID).

Sign-off functionality

The electronic process that allows the heads of providers to sign-off data will become available for the second reference period.

Data Futures Beta pilot known issues

Performance issues: Due to current performance issues on the system files may not process as quickly as you are used to, HESA will monitor the system for any errors.

Open

 

 

Data Supply: Any date field contains a superfluous timestamp in the output. For example - BIRTHDTE

12/04/1980 01:00

Open    
Data entry tool: For union type fields, the tool prepopulates the field with 'None'. If this is left, it generates a 'None' value in the XML which causes a validation error on upload. If the field is not in coverage for the Student, providers will need to delete the 'None' in the tool. Open    
File upload: There is an intermittent connection problem that may mean that an error message is received on upload of a file stating that a connection to the database could not be made. If this occurs, please refresh the page and retry. Open    
In-scope to expressions: The in-scope expressions for the Leaver, Student course session, Financial support offer, Module and Contact purpose entities are currently incorrect which will in particular circumstances cause referential integrity issues in the Quality Rules report. Open    
Data issues: The system is defaulting to display 'System' as the 'Last updated by' value.      

Provider testing

Provider testing overview

Throughout the Beta pilot HESA will be asking participants to focus on and provide feedback on particular areas and peform certain tasks on the system.

This section will provide information regarding the tasks we would like you to perform and how to provide your feedback to us.

Preparation phase

The Beta pilot begins with a preparation phase designed to allow users to familiarise themselves with the system and complete some initial testing.

At the end of this preparation phase, any data submitted in the system will be removed, prior to the early Beta phase beginning. This means that if you are uploading any test files or dummy data, you will not need to delete this data as it will be removed by HESA at the end of this phase.

During this phase we ask that you undertake the below tasks. At the end of the phase, HESA will be seeking feedback from providers regarding their engagement with these tasks.

Preparing for using the HESA data platform

Task

Description

Why is this required?

Gaining access to the HESA data platform.

Ensure all colleagues who will participating in the Beta pilot have the relevant roles within HESA’s identity system. This access is managed by the lead user at each provider, see ‘Accessing the HESA data platform’ above for more information.

Ensuring the right colleagues have access during the preparation phase will mean they have the chance to familiarise themselves with the HESA data platform and that they are set up and ready to engage with the pilot after this initial phase.

You will be able to update access throughout the pilot if necessary.

Complete the e-learning course for the HESA data platform

The e-learning course will guide you through how to use the HESA data platform and the functionality within the system.

It will also demonstrate the behaviour of the system when uploading different files.

The HESA data platform works differently to any HESA systems you may have engaged with before. We strongly recommend that you complete the e-learning course to understand how the system works before engaging with it.

Download the HESA XML data entry tool and review the guidance

The HESA XML Data Entry Tool and associated guidance can be found on the Student 2019/20 coding manual: https://www.hesa.ac.uk/collection/student/datafutures/default

This allows you to manually create xml files for submission to the HESA data platform in the right format for a given specification.

Depending upon your internal system, you may need to use the data entry tool to submit to the Beta pilot, either at the beginning of the pilot or throughout.

Ensuring you familiarise yourself with this process will mean you are ready to create files for submission.

Using the HESA data platform

Task

Description

Why is this required?

Upload a file

Use the file manager to upload a file to the HESA data platform.

Start with the basics, ensure that you understand the process for uploading a file.

Engage with schema checks

When you first upload a file, the system will check that it is in the correct format. If it is not, a schema error will be triggered.

Upload a file that has a schema issue (provided as part of the e-learning in Module 4), review and resolve the issue.

This is the first stage of validation that your data will need to pass; make sure you understand where these rules will appear and how to interpret them.

Process a deletion

In order to remove data from the HESA data platform, you must upload the data with a delete action in the xml.

Further details can be found on this here: https://www.hesa.ac.uk/collection/student/datafutures/deleting_data

We would recommend having a go at this process during the preparation phase to ensure you understand the process and system behaviour when deleting data.

This is a new process required for Data Futures which most providers will not have previously engaged with.

Going through this process as part of the preparation phase will ensure you understand the process and system behaviour for deleting data ahead of doing more rigorous testing.

Engage with referential integrity checks

A referential integrity quality rule will appear if you are attempting to link to another entity that does not exist. For example, if you return a Student identifier (SID) for an Engagement and there is no student that has been returned with that SID.

It would also occur if you deleted a parent entity without also deleting any child entities.

Upload the test files causing the referential integrity issues (provided as part of the e-learning in Module 5), review and resolve the issues. The files will need to be uploaded in the right order to cause the issue.

Submissions to the new HESA data platform work on an incremental basis, therefore you will need to keep track of data submitted in previous files in order to ensure referential integrity.

Understanding how entities link together across multiple files is an important part of engaging with the HESA data platform.

Engage with quality rules

Quality rules will run on your data in the context of a collection. Upload a file that passes schema checks but fails other quality rules (provided as part of the e-learning in Module 3). View your quality rules report, can you understand and resolve the quality rule?

Quality rules are presented in a plain English and technical expression format, it’s important to be able to interpret and resolve quality rules presented in these formats in order to finalise your submission.

Engage with issues

You can create issues within a collection. You may do this when, for example, you want to request a tolerance change on a quality rule.

Raise an issue within your submission.

Raising and editing issues will allow you to request tolerance adjustments as well as communicate with HESA and your statutory customer(s).

Use this time to get to grips with the process.

Any other tests you would like to run

If you have any data you wish to upload or tests you would like to run, feel free to take this time to do this.

Be aware that at the end of the preparation phase, any data in the system will be removed.

This is your opportunity to familiarise yourself with the HESA data platform. Any data you submit will be removed so you can use this to get to grips with the different functionality and processes without needing to delete any test data you have submitted afterwards.

 

Other things to consider and feedback on

Area

Description

Why is this required?

Specification portal

This is the area of the HESA data platform which contains details of the specifications relevant to each collection.

This includes the data model, XSD, data items and associated guidance documents.

The specification portal will replace the coding manuals published on the HESA website. We would appreciate any feedback on the portal prior to these becoming available outside of the Beta pilot.

General user experience of the system

This includes the user interface, navigation and your general experience as a user of the HESA data platform.

The above tests encourage you to run through some key functionality of the system, however the general usability, ease of navigation and user experience are all also important.

We would appreciate any feedback you have on these elements.

Providing HESA with feedback

HESA will require Beta participants to provide feedback throughout the Beta pilot. There will be oppurtunities to provide general feedback as well as feedback on specific tasks we have asked you to undertake.

We will provide survey templates that you can complete in order to give HESA feedback.

Once these are available the links will appear here.

Further guidance

Data model and specification guidance

Beta participants must ensure that they refer to the specification portal on the HESA data platform for details and further guidance regarding the data model and specification.

Please be aware that the specification portal and coding manual on the HESA website may be on different version numbers. For the purposes of Beta testing, the version on the specification portal must be referred to when creating files for submission.

The data entry tool

The HESA XML Data Entry Tool can be used to manually create files for submission to the system.

Please make sure you use the latest version of the xsd from the specification portal to create your files so that they adhere to the correct schema.

The data entry tool and a user guide can be downloaded from the Student record - Data Futures coding manual.

Data Futures programme information

Further information on the Data Futures programme can be found in the Data Futures area.

This includes Data Futures resources which contains key information including FAQs, data scenarios and key concepts.