HE-BCI UUK workshop
On 28 September 2022 Hannah Browne, Lead Policy and Research Analyst at HESA, joined colleagues from Universities UK (UUK) in hosting an online members workshop. UUK aims to foster a collaborative relationship between the 140 world-leading institutions it represents and develop trust between them, policymakers and the public. The interactive workshop was an opportunity for HESA to consult directly with colleagues from a diversity of institutions and discuss the six areas of the HE-BCI data collection that are developed as part of the review project.
Around 32 attendees were introduced to each of the six areas and split into three smaller groups. Colleagues were presented with a number of questions to prompt discussions and share feedback on the proposed measures and HESA’s findings to date. Topics included what types of knowledge exchange (KE) are valuable to them as an institution, whether data relating to each of the areas being captured and if so how, and whether the proposed burden is proportionate to the benefits changes to collection may present.
Group 1: ‘Geographic granularity’ and ‘Equality and diversity’
Group 3: ‘Social and community interactions’ and ‘Students and staff as agents of KE’
Group 2: In-kind contributions’
Group 2 firstly discussed the value of in-kind contributions and ways in which they are internally recorded and reported to HESA. In-kind data reported in the HE-BCI collection has been highlighted by centralised funding council as a mistrusted and unreliable data point. The reasons for the poor reliance in the data is due to concerns that ambiguous guidance, a lack of methodology, and misinterpretations of the level of accountability required has resulted in the reporting of inconsistent data. Consequently, the data is believed to be lacking the validity required to include in-kind data in funding allocation algorithms.
Data collection and reporting
The process of centralising data relating to in-kind contributions was highlighted as a challenging task. Attendees of the group confirmed that ‘in-kind’ contributions were often a significant form of exchange from partner organisations and stated that they were frequent features of collaborative partnerships. Despite their frequency they collectively suggested that a lack of a centralised recording mechanism within their institutions meant that data relating to these contributions was dispersed across their respective letters of support and contracts. Therefore, they detailed the manual and arduous process of reviewing issued letters of support to collate in-kind data for the purpose of the HE-BCI collection. Representatives from larger institutions stated this could often entail reviewing 100s of documents to extract in-kind data.
HESA are not currently proposing the introduction of additional in-kind data or alternative metrics to measure the value of these contributions. Instead, HESA aims to enhance the support available to providers in ensuring they are best equipped to consistently report in-kind data between collections and in-line with their peers. By sharing the realities of the process by which data is sourced and submitted to the HE-BCI data collection delegates developed our understanding of the implications of potential changes to the ways in which this data is to be quality assured and returned. It was noted that without a centralised recording mechanism available to all submitting providers the burden associated to the collection and recording of this data is dependent in the individualised processes used by institutions. Whilst they discussed a lack of shared practice of consolidating their data, they sought agreement that the reporting of this data in its current form is challenging and requires significant staff and time resources.
One representative highlighted the opportunity to link the HE-BCI data collection to UKRIs Gateway to Research (GtR) database . They stated that the main purpose of reporting in-kind data was felt to be to evidence contributions given to collaborative research and design (R&D) projects other than funded provided by UK Research and Innovation (UKRI) or a respective funding council. The GtR provides a publicly available and open source of data relating to these projects. The database includes details of the partners involved, costs and grants associated to collaborative R&D projects. It was highlighted that data on the economic value of contributions to projects outside of formal support could be obtained from this database. This would reduce the burden to providers in collating and maintain a similar database themselves. The use of this dataset is an area yet to be explored by HESA.
Multi-year projects
HESA is currently working to enhance the support to providers when returning contributions received as part of multi-year projects. Previous engagement with sector representatives has highlighted that there is diversity in reporting practices. Some have indicated that all contributions stated in letters of support are returned in the HE-Bci data respective of the year in which the project starts. Others have suggested contributions are apportioned across the years in which a project spans, with each portion submitted to their respective collections. These approaches are problematic for two reasons; 1) the differences in reporting methods creates highlights inconsistencies in the data and limits its comparability, and 2) some institutions may be reporting ‘lumpy’ data that does not depict a reliable reflection of contributions received.
Attendees of the UUK session discussed their differences in reporting. Some suggested that in stating contributions outlined at the start of project they may be overestimating or underreporting the true contributions received throughout its lifecycle. However, they recognised an alternative proposal may be to monitor contributions throughout the project. Whilst they recognised this may reflect the fluidity of the receipt of contributions to projects they suggested that implementing a change to process to accommodate for this method of recording would be unfeasible. They stated that monitoring contributions to projects over a period of time would require additional resources such as additional staff and time; both of which are not available.
The group indicated that there needs to be a more flexible and ‘open’ approach to the reporting of in-kind contributions. This included removing the link between HE-BCI data and finance data, for example by removing reference to financial audit when discussing institutional accountability. They acknowledged that in-kind contributions vary greatly in type and value and that the guidance and collection should respect and reflect this.
Overall, representatives suggested that without incentive to amend reporting processes they would be limited in being able to do so. They highlighted collaborative research and development (collaborative R&D) data does not influence the allocation of the higher education innovation fund (HEIF) in England and therefore additional efforts would not be financially rewarded. It was suggested that when facing limited resources focus is given to driving areas of knowledge exchange (KE) that are financially supported and centrally funded, for example consultancy and continuous professional development (CPD).
‘Commercialisation’
Attendees of the session highlighted that the commercial outputs included in the HE-BCI data collection are not reflective of the diversity of the disciplines and expertise of all providers. They stated the collection lacks output types commonly related to the arts and humanities, for example films, royalties and expositions. Representatives stated that if they were to be included, and their value monitored in the record, we could create a better understanding of the value of KE undertaken in these sectors. Therefore, they suggested that the breadth of commercialisation covered in the record should be expanded and be more inclusive to the diversity of institutions across the UK.
The use of institutions investment in its spin-outs was discussed and highlighted as a potentially useful way to monitor their perceived value. Attendees recognised that the true value of spin-outs is influenced by factors beyond that of the institution and therefore welcomed the use of a metric that suggests the future value of these organisations once they reach full maturity. It was noted that data on investment should be averaged across either two or three years by using data submitted in previous collections. This would avoid ‘lumpy’ data and give a general view of the investment an institution gives. However, one attendee highlighted that the lack of investment by an institution into its spin-outs may not be a reflection of their lack of wanting to do so, but instead a reflection of their ability to do so. They stated that investment is only possible when there are the financial resources available, and some providers do not have the capability to invest in equity stakes. Therefore, they questioned what this data could show us at a sector level and suggested it may present a demonstration of institutional capabilities rather than strategic aims or perceived worth of spin-outs.
Additionally, attendees discussed the challenges faced when obtaining data from start-ups. One particular institution stated it is typical for their staff members to be founders of start-up companies prior to becoming employees of the institution. They said that these employees were often reluctant to provide information about their organisations. Their hesitation to do so was exacerbated by the lack of incentive to do so – either to them as individuals or to the institution as a whole. Similarly, attendees stated that obtaining company data from graduate founders was often a difficult task and again was made more challenging without a clear incentive to either their own organisations or the institution. They often rely on the use of publicly available datasets such as those held by Companies House. Although felt this is often outdated and inaccurate it gave them an additional source of data for submission in the HE-BCI data collection. Brief discussion was had about the potential to provide founder names and registration dates to aid the linking of Companies House data to HE-BCI.