Skip to main content

Beta phase

This page provides key operational information to Beta participants and will be updated with any new information throughout the Beta phase

If you have any queries please email [email protected], or call +44 (0) 1242 388 531.

Beta phase: key points

  • Our aim for the Beta phase is to support testing and provider readiness ahead of the HESA Data Platform (HDP) opening for the 2022/23 collection in early 2023 with an experience closer to real-world usage than Alpha.  
  • There is no cap on applications: as communicated in our September 2021 update, we would encourage all providers to participate even if they may not be ready to submit files from the start of Beta. 
  • The HESA Data Platform will remain open throughout Beta for providers to use in whatever capacity they can - we aren't restricting access only to perform specific tasks/functions. This includes but is not limited to:
    • Uploading a new file.
    • Reviewing quality rules and reports.
    • Raising issues in the Issue Management System.
    • Progressing to sign-off.
  • Beta testing will ensure the HDP is robust and responsive and enable providers to test their systems and processes: providers will submit real and test data files and engage with the quality assurance process to replicate a real-world data submission; there will be no limits on file size.
  • Data used in Beta testing will not be used for regulatory or statutory purposes.
  • We will provide support and training throughout the phase and we will communicate when this is available.

Beta phase: register your interest

You can join the Beta phase at any time. We have closed the public online survey, so please contact [email protected] for further information on how to join.

Reference copy of Beta participation agreement

A PDF reference copy of the participation agreement is now available: we hope this is helpful. Please note, this version of the agreement is for internal reviewing purposes only.

The actual participation agreement will be sent via DocuSign to your head of provider and must be signed before you can be onboarded to the Beta phase.Data Futures Beta phase participation agreement (PDF)

Beta phase overview

Page updated: 23 August 2022.

Please note that during Beta phase, we are still testing and refining systems and processes, and this is to be expected. Your valuable input throughout Beta will help us to resolve issues and any outstanding functionality ahead of the HESA Data Platform (HDP) go-live in early 2023.    

Once participants have signed their participation agreement, they will be onboarded with Beta IDS roles. Participants can engage with the HESA Data Platform (HDP) once they have accepted their Beta IDS roles. 

Participants will complete the Beta phase using 2021/22 data (the Beta 21056 coding manual) with a single reference period that starts from 1 August 2021 until 30 July 2022, with an extended sign-off period from 1 August 2022 until November 2022. 

Participants will complete a more realistic submission using 2021/22 data, as the data will not have been through HESA's quality assurance process.

  • Beta coding manual version 1.3.0 (including XSD version 1.4.0).
  • Collection Activity log.
  • Additional Collection Reports:
    • PGR Transfers In/Out
    • IRIS (Statutory Customer)
    • Continuity report
    • Frequency counts.
  • Migrated data loaded into HESA Data Platform (HDP) for quality assurance (QA).
  • Access to 20056 migrated data (enriched file only) from HDP.
  • Online validation tool (OVT).
  • Credibility Issues can now be managed in the Issue Management System (IMS). As the reference period for Beta has ended, all issues will be automatically raised for a submission.
  • Tolerance approval ranges - allowing providers to self-approve some requests, while others are routed to either HESA or Statutory Customers.

Contact [email protected] if you have any queries related to phases or requirements. 

Real data

For Beta we are encouraging providers to use real student data in the HESA Data Platform (HDP), to closely mimic a normal collection and provide a more realistic submission.

This would ideally use 2021/22 student data. This data will not have been through any of HESA’s quality assurance process, so will provide a more accurate picture on HDP behaviour. By using real data, providers can engage with the HDP and its embedded quality assurance processes. That will help to test their systems and processes and replicate a real-world data submission.

Use of live data

At HESA, we are committed to implementing industry-leading data privacy and security practices, processes and technology to protect the rights and freedoms of data subjects in the Higher Education sector.  

Beta is the second phase of the Data Futures Programme in which real personal data is permitted to be processed with the HESA Data Platform (‘HDP’). HESA does not require Providers to use live personal data during this phase. Therefore, the choice to use either synthetic or personal data rests with the Provider.

Nonetheless, we are aware of the challenges of generating large scale complex synthetic data. To this end, we have ensured that HESA controls are in place to ensure Providers can safely use real personal data in the HDP during the Beta phase. As a result, the environment has been set up to ensure the secure processing of personal data, as if it were live and being used in HESA’s business as usual Student Record collection.

HESA is equally as mindful of its requirements under Article 35 of the UK General Data Protection Regulation (‘GDPR’) and the supplementary guidance published by the Information Commissioner’s Office on the examples of processing that automatically require a Data Protection Impact Assessment (‘DPIA’) or processing that is ‘likely to result in a high risk’ and therefore triggers the same requirement.

Consequently, for each stage of the programme HESA has completed a DPIA, as well as undertaking information security risk assessments, penetration testing and following audit procedures, where appropriate.

Participant Obligations

It is important to note that, as an independent Controller, participant providers should determine how they will comply with their own requirements with respect to their data protection obligations. If you are unsure, we recommend that you speak to your Data Protection Officer or, if you do not have one, take specialist advice.

The Beta guidance note provides data protection information about the uses and intended processing of real personal data during the Beta phase of the Data Futures Pilot.

Beta guidance note

Data protection by design

HESA is approaching the completion of the Beta DPIA as an ongoing process which is embedded in the development of the programme and therefore subject to regular review. Consequently, given that this is an agile programme with three releases for Beta, HESA’s DPIA will continue to evolve alongside the progression of the programme. This means that any risks are identified in a timely manner and are added as a requirement to HESA’s Gateway Framework. If there are any perceived high risks, this Gateway Framework ensures that these are mitigated before the programme can move forward.

The Data Protection team has also worked (and continues to) at an ‘Epic level’ with the programme to ensure privacy by design from the outset of the development for Beta. This means that every part of the design and build of the programme (an Epic) is subject to an information security and data protection review at the outset, during and at the conclusion of the Epic.  This approach has supported the programme in identifying and embedding controls that will ensure the design and build of the HDP is done in a way that protects data subjects’ rights and freedoms, not least against the following core principles:

  • Lawfulness, fairness and transparency
  • Purpose limitation
  • Data minimisation
  • Accuracy
  • Storage limitation
  • Integrity and confidentiality (security)
  • Accountability

As a Controller, as defined by Data Protection legislation, and a custodian for personal data in the higher education sector, each phase for Beta in the Data Futures programme would not be approved for go-ahead by the HESA compliance function if the use of personal data was considered to present a risk to the rights and freedoms of data subjects, as defined by Recital 75 of GDPR.

HESA’s Data Protection Officer has provided sign off for Beta release 1, following completion of the DPIA for the phase, and is satisfied that the processing of personal data as part of the Data Futures pilot does not present a risk to the rights and freedoms of data subjects.

Robust risk management

As with business as usual, HESA continues to follow a risk management methodology aligned with ISO27001 and ISO27701. The Legal and Compliance function has implemented ‘Gateway’ phases to ensure that the necessary data protection and information security risk assessments have been undertaken ahead of Programme milestones as well as on an ad-hoc basis, where appropriate. This is in addition and supplementary to undertaking mandatory DPIAs.

Penetration Testing

HESA maintains a programme-specific Penetration Testing Roadmap. This roadmap ensures that development work is tested at defined stages before release into Beta or Production. Testing is undertaken by HESA’s external independent partner Bridewell Consulting. After each test, all critical and high vulnerabilities require remediation before the Legal and Compliance function give approval for the code to be deployed.

Our commitment to participant organisations

HESA is committed to ensuring that it designs and builds a platform that is technically compliant and robust. This commitment is reflected in clause 10.6 of the Data Futures Pilot Participation Agreement, in which HESA warrants that it will take appropriate technical and organisational measures as may be reasonably necessary to ensure confidentiality, availability, integrity and resilience of its systems.

Security Principles

The following cyber/information security arrangements are in place to protect the availability, confidentiality, and integrity of personal data within all HESA Data Futures Beta and Production environments:

  1. Role based access control, using HESA’s Identity System (IDS)
  2. Multi-factor Authentication due to be rolled out in March 2022.
  3. Full logging and monitoring within HESA Security Information Event Monitoring (SIEM) system
  4. Regular Vulnerability Testing and software updates
  5. Encryption at rest and in transit across the entire HDP
  6. Business Continuity and Disaster Recovery protocols
  7. Geographic filtering controls

 

HDP reference period

The HDP will be set up with a single reference period that starts from 1 August 2021 until 31 July 2022, with an extended sign-off period from 1 August 2022 until November 2022. We encourage providers to submit data during the reference period, and then go through the full quality assurance processes during the extended sign-off period, although these processes will still run throughout the reference period. 

It is not mandatory for providers to submit real student data to the Beta environment. The system can handle synthetic and/or anonymised data that meets the schema, but this may cause more quality issues that need to be resolved. This will also mean that engaging with any data migration work for year-on-year comparisons or populations will not be possible. 

File sizes

There are no limits on file sizes that can be submitted to the HDP. Files can be small – with only a handful of students - a file can contain tens of thousands of students. The files submitted do not need to be complete, i.e. full populations of students and course information, as long as the file meets the relevant schema and all the mandatory fields are populated (this can be with dummy codes or unknown values). For example, if you have 5,000 students, you could choose to submit a subsection of these and issue a file containing 1,000 students, or you could choose a specific scenario; for example, all postgraduate students or students studying History. 

During the early stages of Beta, we anticipate providers will submit smaller files as part of familiarisation and learning, but that once we get towards the latter stages the file sizes will begin to increase. 

Statutory Customers

To support the Statutory Customer transition to the new Data Futures model and changes required to their business processes, HESA will be delivering the data submitted to the HDP during Beta. This is only to support their planning and processes and will not be used for any statutory or regulatory purposes.  

February 2022: Introduction to Beta session 2 presentation 

February 2022: Introduction to Beta session 1 Question & Answer session

 

February 2022: Introduction to Beta session 2 Question & Answer session

February Beta launch: presentation (PDF)

HESA data platform

Login to the HESA Data Platform (HDP)

Please note you will not be able to access the HDP unless:

  • Your head of provider has signed the Beta Phase Participation Agreement.
  • You have completed the HDP IDS onboarding process: access to the HDP is managed through the HESA Identity System (IDS). Users within the IDS have roles assigned to them which govern their access to the HDP.

Further details regarding each role, including terms and conditions will be provided in a guide to HESA IDS roles for the HDP: we will communicate when this is available.

If further roles are required, information will be added and communicated accordingly.

Beta HESA Data Platform demonstration

Data Futures e-learning

Date of release Name of release Summary

 

16/02/2022

 

Beta phase 1: system opens

  • Beta 1.1.0 coding manual
  • Collection Activity log
  • Additional collection reports: frequency counts
21/02/2022 Bug fixes Applied a fix to allow providers to view Data Futures Student issues in the Issue Management System correctly and changes made to the reference data store.
28/02/2022 Bug fixes  Applied a fix to trigger emails to Liaison when providers adds a comment.
2/03/2022 Bug fixes Applied a fix to reopen Issues in IMS following a new file submission to HDP.
9/03/2022 Bug fixes Applied fix to redirect users to correct URLs when they select options from System dropdown.
3/05/2022 Coding manual release Beta coding manual version 1.2.0 released
23/05/2022 Bug fixes IMS bug resolved impacting tolerance override requests
30/05/2022 Beta phase 2
  • Improvements to the user interface
  • Additional collection reports: PGR Transfers In/Out
  • Online Validation Toolkit (OVT)
9/06/2022 Bug fixes

Applied a fix to ensure xml error feedback on the UI in OVT in Beta

16/06/2022 Bug fixes Applied a fix to incorrectly transformed files: When a submission is signed off (20056 collection), the original file is transformed to a JSON format, and there were a couple of cases that were some values were transformed incorrectly
23/06/2022 Coding manual release Beta coding manual version 1.3.0 released
11/07/2022 Bug fixes Applied a fix to remedy limit in the web report of schema errors. Previously, this was only displaying the first 101 errors found.
10/08/2022 Bug fixes Applied a fix to address IMS performance issues.
23/08/2022 Beta phase 3
  • Sign-off: 1 August 2022 – 30 November 2022
  • Ability to raise credibility issues in Issue Management System (IMS)
  • Additional collection reports
20/10/2022 Beta phase 4
  • IMS credibility updates
  • HDP user interface improvements
  • Additional collection reports
  • Data retention policy implementation
  • Bug fixes
1/11/2022 Bug fixes

Updates and enhancements to our Ref Data and Personal Data Store systems behind the scenes.

 

Data Entry Tool

The Data Entry Tool has been updated to be compatible with the new Data Futures schema. You will first need to load the relevant schema you wish to submit data to, in this instance the Beta 1.1.0 schema. This will then allow you to create data files that can then be uploaded to the HESA Data Platform. This has been a task that Alpha participants and HESA staff have found useful in helping them fully understand the new data model, and we would encourage all providers to create small files as part of a learning task.

We recommend you run it on a machine with the minimum specification of a 64-bit processor. The kit was developed using Microsoft's .Net Framework 4.8, so your computer should meet the minimum specifications published by Microsoft.

Download the Data Entry tool

Updated on 19/10/2022

Issue

Status

Date raised

Resolution

Quality rules

The following Quality Rules require amendment due to bugs identified. Please request a tolerance override for these if you are trying to sign off a submission. 

STUDENT.StudentFinancialSupport.APPSPEND.001 

STUDENT.Course.CLSDCRS.003 

STUDENT.Course.TTCID.002 

STUDENT.EntryProfile.CARELEAVER.008.V01 

STUDENT.EntryProfile.PERMADDPOSTCODE.008 

STUDENT.EntryQualificationSubject.SUBJECTID.003 

STUDENT.Course.FUNDLEVEL.001-008 

STUDENT.StudentCourseSession.SESSIONYEARID.001 

STUDENT.EntryProfile.PERMADDPOSTCODE.005 

 

The following Quality Rules have been temporarily switched off in Beta, these will not trigger. 

STUDENT.Course.FULLYFLEX.001.V01 

STUDENT.Disability.DISABILITY.005.V01 

STUDENT.Engagement.ENGSTARTDATE.006.V01 

STUDENT.EntryProfile.ESTRANGED.004.V01 

STUDENT.FundingBody.FUNDINGBODY.018.V01 

STUDENT.FundingBody.FUNDINGBODY.032.V01 

STUDENT.Module.CRDTPTS.005.V01 

STUDENT.Module.CRDTSCM.004.V01 

STUDENT.Student.BIRTHDTE.003.V01 

STUDENT.Course.QUALID.009.V01 

STUDENT.NationalIdentity.NATIOND.002.V01 

STUDENT.NationalIdentity.NATIOND.004.V01 

STUDENT.NationalIdentity.NATIOND.005.V01 

STUDENT.Student.NationalIdentity.001.V01 

STUDENT.Student.NationalIdentity.002.V01 

STUDENT.Student.NationalIdentity.003.V01 

STUDENT.Student.SURNAME.005.V01 

STUDENT.OffVenueActivity.ACTSTARTDATE.003.V01 

STUDENT.OffVenueActivity.ACTSTARTDATE.003.V01 

Open 19/10/2022  

Comparing a quality report for a submission with failures against a submission where there were none causes an error

When a previous submission contained no quality rules failures, and the user attempts to use the ‘compare’ function in the Quality report for a submission with failures, an error will occur on the HESA data platform.

Should this happen, you will still be able to navigate back to your submission using the side navigation bar, as per usual.

Open 18/10/2022  

When a user attempts to ‘Submit for approval’, they will be unable to do so if they have PGR students transferring out to other Providers as part of collaborative provision arrangements, but have not given consent to share that data with the ‘receiving’ Providers.

The HESA data platform doesn’t currently display any information about this, so it can look like all criteria have been met while this is still outstanding.

Open 18/10/2022  

Submit for approval is reached but there are rule failures outside of tolerance

Sometimes the HESA data platform can proceed to Submit for approval before all rule failures have been resolved. If this happens, please refresh the screen to resolve the issue.

Open 18/10/2022  

Credibility report drilldowns do not show for decimal values less than 1

When the user attempts to view the drilldown information for a credibility report cell which contains a decimal value which is less than 1, the drilldown does not show. There is no workaround for this issue.

Open 18/10/2022  

Compare report toggle cannot be turned off

When a user turns on the comparison toggle, they cannot turn it off again.

Instead, the user should refresh the page to reset the Quality Report to it’s original state, with the comparison feature turned off.

Open 18/10/2022  

Comparing a quality report for a submission with failures against the first submission made causes an error

When the user attempts to compare the quality report for the current submission with the quality report for the first submission made in the collection, an error will occur on the HESA data platform.

Should a user wish to compare their first submission, the only option would be to download and resubmit that file, and then download and resubmit the file which they wish to compare. Only then will they be able to perform the comparison, by selecting the later submission of the original file from the compare dropdown.

Open 18/10/2022  

AGEF1 and AGEM1 drilldown tables are empty

During testing we identified an issue where the 'drilldown' tables for the AGEF1 and AGEM1 credibility reports are displaying with no information.

Open 17/8/2022  

Comparing a quality report for a submission with failures against a submission where there were none causes an error

When a previous submission contained no quality rules failures, and the user attempts to use the ‘compare’ function in the Quality report for a submission with failures, an error will occur on the HESA data platform.

Should this happen, you will still be able to navigate back to your submission using the side navigation bar, as per usual.

Closed 17/8/2022 Issue now resolved

Unable to view rule guidance for STUDENT.EntryProfile.PARED.005.V01 and STUDENT.Engagement.RCSTDNT.003.V01

In the Quality report in the HESA data platform, the links to guidance for the rules STUDENT.EntryProfile.PARED.005.V01 and STUDENT.Engagement.RCSTDNT.003.V01 are incorrectly formatted.

The correct links are:
https://codingmanual.hesa.ac.uk/21056/EntryProfile/field/PARED

https://codingmanual.hesa.ac.uk/21056/Engagement/field/RCSTDNT

 

Open 17/8/2022  

The 'stepper' navigation sometimes takes time to update

After the submission has completed ‘Processing’ sometimes it takes time to update the status for the next step of the ‘stepper’ navigation (to a red cross or green tick depending on the status of quality assurance, i.e. whether the submission has triggered any quality rules or credibility failures).

Open 17/8/2022  

Files with filenames which includes underscores will fail the schema check

If the filename of your uploaded file includes one or more underscores ( _ ), that file will fail the schema check but the HESA data platform will give no information as to the cause of this failure.

You will know this has happened if the underscores have been replaced by underscores in square brackets, i.e. [_] .

If you encounter this issue, please rename your file to remove the underscores or replace them with another character, e.g. hyphens ( - ) .

Closed 17/8/2022 Issue now resolved

Derived field requires update

Z_ACT

Z_AGEAUG, Z_AGEJUL, Z_AGESTART

Z_DISABILITY

Z_EXPECT

Z_INACTFROMSCS, Z_INACTWUFROMSCS

Z_LEVEL

Z_NOTEND

Z_PERMADDGRP2, Z_PERMADDGRP4, Z_PERMADDGRP5

Z_POPQUAL, ZPOPSES

Z_PRINONUK

Z_PROVIDERGRP3, 4

Z_QCLASS

Z_QLEVEL

Z_STATUS

Z_STULOAD

Z_TARIFF

Need to be updated to reflect revised specification.

Closed 14/7/2022 Issue now resolved

The 'Processing' status list can get stuck just before 'Quality Processing Complete' is shown

When a submission is being processed, the list of statuses on the ‘Processing’ step should update automatically. There is an intermittent bug currently where this can get stuck just before reaching ‘Quality Processing Complete’. If you see a ‘tick’ icon next to ‘Generating additional collection reports’ and the ‘Quality Processing Complete’ doesn’t have a tick icon shortly afterwards, please refresh the page manually and this should update.

Closed 27/7/2022 Issue now resolved

Schema errors report cannot be downloaded if there is a very large number of errors

The schema error report can normally be downloaded using the icon alongside the schema check processing status on the Processing step for a submission. Where the number of schema errors for a provider is very large (several thousand), we have seen an issue which means that the download will not start. We can manually extract the report and share this with a provider securely, so please contact Liaison if you encounter this issue.

Closed 25/7/2022 Issue now resolved

Uploading an XML file with the extension in uppercase causes infinite Uploading state

The file checker we use to confirm that the file requested for upload is in a valid format (XML) relies on the extension of the uploaded file being in lowercase, i.e. ‘.xml’, and not uppercase, i.e. ‘.XML’, or a mixed case.

Before attempting upload of an XML file, the user should check that the extension of the said file is formatted in lowercase, i.e. ‘.xml’.

Closed 7/7/2022 Issue now resolved

Schema check limits the number of returned errors

We are still working to understand the root cause of this issue, and have not been able to replicate it, but it has been reported that the report generated following a failed schema check does not return all errors for a large file, and seems to limit the number returned.

Closed 21/06/2022 Issue now resolved

Tolerances over 100% aren't archiving issues in IMS

If a tolerance over 100% is approved for a rule using IMS, when a new file is submitted to HDP, even if the new tolerance brings the rule failure within tolerance, the issue created in IMS is not archived as expected and remains open. 

Closed 21/06/2022 Issue now resolved

Unable to view rule details for STUDENT.Engagement.Leaver.002_V01

It is not possible to view the rule details for the quality rule STUDENT.Engagement.Leaver.002_V01. Clicking on the rule failure count will result in an error.

Closed 21/06/2022 Issue now resolved

Credibility report downloads

An error in the OVA1 credibility report prevents a user from being able to download this report. This affects any download which would include the OVA1 report, including the chapter level download for 'Off Venue Activity' and the 'Download Reports' feature for Credibility reports.

There is an issue where many of the downloaded reports do not contain any data. So far we have observed this with: ITT2, ITT4, TAR1, TAR1P, all QUAL reports, PADD1, PADD3, PADD2, SEN1P, SEN2, AGEM1 and AGEF1.

Open 21/06/2022  

Closing a credibility drilldown report results in a download error if done before download starts

When the detailed 'drilldown' view is shown and the user downloads the report, they need to leave the report open until the download starts in their browser. Closing the report before the download starts results in an error.

Open 21/06/2022  

Cell shading not active in SEN1 credibility report for Accelerated Full Time tab and HND/HNC column on all tabs

There are some issues with shading not showing on the Accelerated Full Time tab or in the HND/HNC column on any tab. The rules can be triggered still, but this will not be clear on the report as the shading will not show in these areas.

Open 21/06/2022  

AGEF1 credibility report not counting all <SEXID> in Total

During testing we identified an issue where the AGEF1 credibility report is not counting the total number of female students correctly. In the file we tested, 70 students were identified as female (SEXID = 10), but the total only shows 69.

Open 21/06/2022  

Various cosmetic issues with user interface

We have seen some strange cosmetic issues with the HDP user interface. Below are a summary of those we have identified.

It is possible to scroll the submissions screens sideways and doing this results in some strange visual behaviour, including overlapping of elements over the left hand navigation bar.

In the Online Validation Tool, on the Upload screen, padding around the filename and download file link is reduced following upload.

On the Quality Report, the column header behaviour is inconsistent with regard to visual treatment for sorting. All columns are sortable apart from 'Regulator' and 'Issue status'. 'Rule' is sortable by cannot be highlighted by hover-over, 'Issue status' does highlight with hover-over but is not sortable. All sortable columns have an arrow icon appear if you hover slightly to the left of the heading, so this is the most reliable indicator for sorting.

Navigating to the Manage Submissions screen after scrolling down on another screen does not reset the screen and makes it appear as if not content has loaded. Scrolling up will show the missing content, as will refreshing the page.

Sometimes the credibility ‘drilldown’ view, i.e. the table showing row level data for a cell in a credibility report can show the contents of the previously viewed drilldown briefly before refreshing to show the correct data.

Open 21/06/2022  

Agreed tolerance overrides do not show in the Quality Rules report in HDP

The Quality Report in the HESA data platform (HDP) shows the tolerance which applies to each rule. This number is the 'default' tolerance value applied to all Providers for each Statutory Customer (SC).

When a tolerance override is agreed (by the provider, HESA or the SC) it is used by the quality rules processing when a new file is submitted, but the Quality Rules Report will still display the 'default' tolerance, and not the agreed overridden tolerance.

The agreed overridden tolerance will be shown correctly in the Issue Management System (IMS).

Closed

10/2/2022

Issue now resolved

Quality, Credibility and Additional Collection Reports section show a continuous loading icon where there is no data to load

When a file has not progressed far enough during processing to generate reports for the submission, it is still possible for a user to attempt to view those reports. There is no information provided to indicate that there is nothing to load, instead showing the loading icon (spinner).

This will be corrected by ongoing work to update the generic design with the new design, which will add in additional features to help a user follow the intended journey, and reduce the chances of these types of error.

Closed

10/2/2022

Issue now resolved

Compare dropdown menus on Quality and Credibility Reports can move with the view when scrolled

When a user scrolls the browser window after opening the 'Compare' dropdown menu for either report, the menu can scroll with the view, rather than staying as a static object on the page.

Closed

10/2/2022

Issue now resolved

Uploading an Enriched file causes infinite Uploading state

While there would be no reason to do this, if a user attempts to upload the Enriched file generated by their submission as a new submission, the processor will get stuck in an infinite Uploading state. It is still possible to start a new submission and upload a new valid file when this happens.

Open

10/2/2022

 

Credibility report SEN1 show percentage changes from 0 as 0%

In the HESA data platform, changes where the rule has to divide by 0 (an impossibly operation) show as 0% change. This is different from the current collection system which shows these changes as 100%.

Open

 

10/2/2022

 

Credibility report SEN1 comparison feature does not function correctly

In the HESA data platform, we offer a feature which allows a user to compare the credibility report from one submission with that of another submission within the same collection. Part of this feature is shading rules to highlight where a rule did not fail in the selected submission, but is now failing in the current submission.

Unfortunately, a bug has been identified in this feature which means the shading rule is not functioning correctly. While the difference indicator (an up arrow or down arrow) will indicate the change between values correctly, the shading is not accurate and should be disregarded.

Closed

10/2/2022

Issue now resolved

Schema errors because of badly formed XML do not generate a corresponding schema check failures report

Usually, a schema error will be accompanied by a detailed schema error report designed to assist Providers in identifying the issues with their XML file structure. Where the XML file is badly formed (i.e. not a valid Data Futures XML file) it is not possible to generate this report and it does not show.

To reduce the risk of this occurring, we suggest you run our files through the Beta Data Entry Tool prior to submitting them to the HDP.

Closed

10/2/2022

Issue now resolved

‘View All Issues’ button is navigating to the wrong IMS page

The ‘View All Issues’ button on the Quality Report links the user to the Issue Management System (IMS). Currently, a bug in the way the link is formed causes the button to link to the wrong page in IMS and show no issues. Clicking the ‘Search’ button in IMS will resolve this problem and issues will display correctly once that button has been clicked.

Closed

10/2/2022

Issue now resolved

Multi-users can encounter a variety of strange user interface behaviour

We have seen a variety of strange behaviours when IDS accounts contain roles associated with more than one Provider. The HESA data platform is designed to accommodate this scenario (one user working on submissions for more than one Provider organisation), but we are still working on resolving some bugs associated with this feature.

When logging into IDS and accessing the HDP, it is possible for the displayed Provider (in the top bar dropdown menu) and the displayed information to be mismatched. This is because the dropdown by default displays the first option in the list, whereas the displayed Provider information is determined by the order of Provider organisations in IDS.

For these reasons, we advise multi-users to select the Provider they which to view information for when they access the HDP, even if the selection drop-down is already displaying the Provider they want. We also advise that multi-users navigate back to the HESA data platform’s dashboard (currently a blank landing page for the application) before switching to another Provider.

Open

10/2/2022

 

Support materials

Further information on the Data Futures programme can be found in the Data Futures area and Data Futures resources pages

All e-learning releases are announced to our operational contacts in the HESA weekly update: if you are a non-operational contact, you can also subscribe to receive the latest news. 

Our e-learning platform is hosted by Easygenerator and you will need to subscribe to access the content. We will only use this information to administer and monitor use of the system.

Data Futures e-learning 

How to report issues during Beta

It is important that you report any bugs or defects in the systems that you may encounter during the Beta testing phase, as it will enable our teams to pinpoint the specific issue and identify what needs to be fixed. 

We have created an online form to be used each time you wish to report a bug. A good bug report should contain information about just one issue, be as concise as possible, and include sufficient information to allow the development team to recreate the user’s steps. The most important information that must be logged is the ‘how’ and ‘where’, in order to communicate exactly how the test was performed and where the defect occurred.  

Please use this online form to report any bugs that you identify during Beta: https://forms.office.com/r/jM9gsGG0BA 

Please try to be as detailed as you can with your feedback. The nature of your feedback might include: 

  1. An issue or defect - where the system does not behave as intended 
  2. An idea for improvement of existing functionality  
  3. A new feature request 
  4. Any other feedback, questions or comments about the systems 

 Top tips for documenting issues and defects: 

  • Check it is not an existing issue that we are aware of in the HESA Data Platform Known Issues section 
  • Confirm what account you were signed on as 
  • Confirm what system you were using when you encountered the issue I.e., HDP or IMS as this will help us get the right specialists looking at the problem. 
  • Please confirm the steps you were taking when you found the issue. 
  • If you can, confirm the actual result (i.e. incorrect output or error message) and the result you expected to see. 
  • If you can, provide any relevant screenshots, please email to: [email protected] and include the 'Title' (Q.5) as your subject line. 

Your personal data will be recorded against this form once submitted.  All open bugs will be published on the Beta support guide under ‘HESA Data Platform Known Issues’. 

 

The Identity System

Beta Identity System roles 

As per the Pilot Participation Agreements, we have committed to publishing the role Terms and Conditions as operational documentation. Please find below the role descriptions and Terms and Conditions for all provider roles for the pilot exercise. As the project is being delivered on an agile basis, it may be that from time to time we make changes to the terms and conditions for these roles. Where changes are made, these will be updated in the Identity System and you will be asked to reaccept these terms, if you do not accept the terms, do not accept the role.

Role descriptions

Provider IDS Record Contact - Student:

The Record Contact is the first point of communication during the Beta pilot. The Record Contact role does not provide access to the HDP.

The Record Contact is responsible for the delegation of Beta IDS roles to relevant colleagues and can approve roles that are requested by others within the Identity System. Record Contacts may either administer all other roles themselves or delegate this activity to those with the Provider IDS Admin – Student role.

Record Contacts are responsible for ensuring that people who either no longer act for your organisation or no longer have an active role in the Beta pilot have their roles revoked.

The Provider IDS Record Contact – Student role is administered by HESA. If you hold this role and need to delegate this to someone else in your organisation during the Beta pilot please contact HESA Liaison team.

Provider IDS Admin - Student:

The Admin role allows the administration of Provider HDP roles on behalf of the provider's Record Contact and to approve role requests within the Identity System.

The role does not provide access to the HDP.

Provider HDP Guest – Student:

The Guest role has read-only access to the HDP. Holders of this role can view reports displayed in the HDP but will not be able to upload, alter or delete data.

This role is useful for colleagues who are not involved in the submission of the data but who need to review submitted data and create and view reports.

Provider HDP Submitter – Student:

The Submitter role grants a user access to the HDP in order to upload and manage data submissions. This includes the ability to make submissions, view and interact with reports and create and view issues in the HDP.

Provider HDP Sign Off – Student:

The Sign off role grants a user access to submit a file for approval and to request sign off in the HDP.

Provider Issue Management System User – Student

The Issue Management System User role grants the required access to view and respond to data quality issues for the Beta pilot in the Issue Management System.

 

Role Terms and Conditions

Provider HDP Guest - Student

Provider HDP Sign off - Student

Provider HDP Submitter - Student

Provider IDS Admin - Student

Provider IDS Record Contact - Student

Provider Issue Management System - Student

Submission stages on the HDP

From the ‘Manage Submissions’ page you can either upload a file or display the previous files that have been uploaded. Once ‘upload file’ has been selected it will take you to the ‘Ingestion’ stage. 

Image of the Manage Submissions page of the HDP

The ‘Ingestion’ stage is where a file can be uploaded to the HESA Data Platform (HDP). Here you will have the option to either use the ‘drag and drop’ or select a file to upload.  

Image of where you upload a file on HDP

‘Uploaded’ will be displayed when this is complete. If a file fails to be ingested it will not be logged as an attempted upload in the HDP activity log. Failure may be caused due to a loss of connection to the HDP. If a file fails to complete ingestion then the same file can be uploaded again. 

Tips: 

  • Files can have any name 
  • Files must be in XML and conform to the relevant XML Schema Definition (XSD) file 
  • Only a single file can be held on the system 

Once uploaded, the file will begin processing and will go through the following stages: 

Image of the stages of processing a file goes through

  1. Uploaded – the file has been uploaded to the HDP 
  2. Provisioned – this is where the system gets space ready for processing 
  3. Schema check passed – your file meets the schema specification. If you file does not pass schema then the following will display Schema check failed will displayand you will need to reupload your file with a corrected schema. 
  4. Enrichment performed– this is when the derived fields are created and a ‘download’ button is available to download these. The file will include the submitted data as well as the derived fields. 
  5. Quality processing complete –the quality and credibility reports have been created 

An overview of your file will also be displayed here. 

Within the ‘Quality assurance’ section the quality and credibility reports will be displayed, along with any additional reports.  

Image of the quality reports available in the HDP

The Quality report will contain the details of any rules triggered by the submission. To pass validation all quality rules need to be within tolerance or be resolved through amending the file. When a new file needs to be uploaded please navigate to the ‘Manage submissions’ menu. Requests for changes to tolerances and thresholds are to be raised as an issue and are managed in the Issue Management System. 

Further details on the quality rules which apply to this collection can be found in the coding manual

The Credibility reports section provides an overview of the data submitted. The tables are broken down into chapters such as ‘Student instance profile’ and will look at year-on-year differences between your data.  

Additional reports will include IRIS, PGR Transfers In/Out. These will be available in later phases of Beta.  

The quality report will display the quality rule that is triggering, along with the population this rule is applicable to and the number of records (percentage) that are failing the rule.  

Image of how a quality rule is displayed

Where a quality rule is triggering and the data returned is correct, a tolerance amendment will need to be requested and an explanation provided. The three dots under ‘Issue status’ needs to be selected and then ‘Create issue’. An issue will then automatically be created in the Issue Management System (IMS). 

The Issue Management System can be accessed here: https://betaissuemanagement.hesa.ac.uk/ 

Image of how to create an issue for IMS

Where a quality rule is failing due to correct data you will need to request a tolerance override. In IMS you will need to select the relevant issue, then ‘request override’ and submit the tolerance requested along with a supporting comment. Once this has been submitted either HESA or your statutory customer will review and the request will be approved or you will be asked for further comment to clarify the request.  

Image of how to request an override in IMS

All quality issues will need to be resolved, either by an amendment in tolerance or correction to the data to progress to the final ‘Approval & sign-off’ stage 

Once all quality issues have been resolved and the reference period has ended, the data can be marked as ‘approved’.  

HESA and the relevant statutory customer will then approve your data.  

Once the approval stage is completed your data will need to be signed off by your Accountable Officer/Head of Provider.