Guidance on assessing performance and value of Public Sector Research Establishments – GOV.UK

We use some essential cookies to make this website work.
We’d like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.
We also use cookies set by other sites to help us deliver content from their services.
You can change your cookie settings at any time.
Find information and services
Search for a department and find out what the government is doing
Departments, agencies and public bodies
News stories, speeches, letters and notices
Detailed guidance, regulations and rules
Reports, analysis and official statistics
Consultations and strategy
Data, Freedom of Information releases and corporate reports
Published 25 January 2022

© Crown copyright 2022
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected].
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/public-sector-research-establishment-value-framework/guidance-on-assessing-performance-and-value-of-public-sector-research-establishments
Public Sector Research Establishments (PSREs) play an important role in the UK’s science, research, development, and innovation (RDI) landscape. Principally, they support government by providing science advice to policymakers, by acting as a strategic capability in policy delivery and by delivering critical science services for government, business, and society. Given the wide-ranging policy and operational needs of government departments, PSREs themselves vary greatly in function and mission. Some are highly research intensive (for example, The Defence Science and Technology Laboratory), seeking to develop new technologies, whilst others play critical operational and regulatory roles necessary for running the country (for example, the Environment Agency).
The Science Capability Review (2019) recognised that PSREs represent a significant public asset that is currently under-utilised and not well understood across government (see Science Capability Review Recommendations 3 and 4). The purpose of this document is to provide a broad framework and a set of common principles to support departments in assessing the value and performance of the PSREs they sponsor, and to think critically about how they could be better utilised. This framework should be used during the Cabinet Office’s Arms-Length Body (ALB) review process to ensure a consistent standard of reviews of PSREs across departments, but it is also recommended that future reviews of PSREs conducted separately to the ALB reviews, should make use of this framework. In addition, departments that sponsor PSREs that are not considered to be ALBs should also make use of this framework.
This document is not intended as a single, complete treatment for planning a review of any individual PSRE. Development of specific PSRE performance measures – and the conduct of reviews more broadly – should be co-developed in consultation with key stakeholders. Critically, the PSRE’s Chief Scientist (or equivalent) and the sponsoring department’s Chief Scientific Adviser (CSA) should be involved throughout the review process. The performance measures and methodology identified throughout the course of the initial PSRE review should continue to be monitored and used for future reviews, to ensure consistency of methodology over time and tracking of progress.
It must also be recognised that whilst this document focuses on the guidance of reviews of individual PSREs, reviews should acknowledge the wider departmental context. Specifically, there should be alignment between reviews of different PSREs conducted simultaneously by the department. The department’s CSA has an important role to play in ensuring reviews of the department’s sponsored PSREs are conducted with coherence and a focus on ensuring the portfolio of PSREs are delivering the capability needed by the department and wider government within their respective RDI areas.
The guidance contained within this document is applicable to a range of ALBs: while this guidance will predominantly be applied to PSREs, it can be applied to other bodies that fit within the guidance’s scope. At present, the following definition of PSREs from the Royal Society[footnote 1] is being used. PSREs that fall under this definition are listed in the accompanying Royal Society document.[footnote 2]
PSREs are a diverse collection of public bodies carrying out research. This research supports a wide range of government objectives, including informing policy making, statutory and regulatory functions and providing a national strategic resource in key areas of scientific research. They can also provide emergency response services. They interact with businesses around a wide array of innovation-related functions.
Within this, we use the definition of research outlined in the Frascati Manual, namely that this covers work undertaken on a systematic basis to increase the stock of knowledge, and the use of this stock of knowledge to devise new applications. As such, this includes all activities along the spectrum of discovery research, translational research, mission-driven and applied research and the development, demonstration and diffusion of new approaches, methods, and technologies.
It is strongly recommended that departments use this Value Framework for any ALBs matching the below three key criteria:
the ALB is owned, partly owned, or sponsored by a central government department, and receives a substantial part of its total funding from public sources
the organisation has a clear remit to conduct research itself (as opposed to contracting external organisations to conduct research)
the ALB plays a key role in the UK science and research community, and is already, or has the potential to be, well aligned with strategic governmental priorities
The Lead Reviewer. The Cabinet Office’s “Guidance for Undertaking Reviews of Public Bodies” recommends that departments appoint an independent lead reviewer responsible for ensuring that a proportionate and effective review is conducted. The Lead Reviewer is ultimately accountable for the recommendations of the reviews they lead. In addition to requirements set out by the Cabinet Office, this person should have a relevant scientific background for the PSRE that is being reviewed. The lead reviewer must also be sufficiently independent to enable an objective review. The Cabinet Office’s “Guidance for Undertaking Reviews of Public Bodies” provides further details of the roles of the Lead Reviewer.
The Review Team are responsible for supporting the lead reviewer and will be made up of individuals from the department undertaking the review. It is the responsibility of departments to adequately resource their reviews depending on the nature of the PSRE under review. The departmental Review Team should be independent of the PSRE, and its sponsoring team. Other responsibilities of the review team include shortlisting Lead Reviewers, building a trusting and effective relationship with the PSRE, ensuring appropriate engagement between the Lead Reviewer and the PSRE and ensuring appropriate engagement with the department’s CSA. The Cabinet Office’s “Guidance for Undertaking Reviews of Public Bodies” provides further details of the roles of the Review Team.
The Department’s Chief Scientific Adviser (CSA) should take an active and supporting role in the review process to ensure the reviews are being conducted to an appropriate standard. If the department sponsors multiple PSREs, the CSA must also ensure coherence across the various reviews being undertaken by the department. This will include ensuring Lead Reviewers and Review Teams work together during the review process and ensuring that a portfolio view is taken so that the relevant sponsored PSREs are able to deliver the capabilities needed by the department and wider government. They are also responsible for disseminating the results of the review to key stakeholders within the department.
This Value Framework aims to help departments develop performance measures, assess performance and value and identify areas of further opportunity through four assessment steps.
Evaluate Mission: To identify the PSRE’s roles within the RDI landscape and assess its relevance to the department, to broader government priorities and to the wider RDI landscape.
Assess Impact: To develop appropriate PSRE performance measures relevant for the PSRE’s role and assess the performance of the PSRE with respect to its mission.
Assess additional value: To identify the wider value created by the PSRE beyond its core functions (for example, skill development, resilience and infrastructure, operational standards).
Assess further opportunity: To think critically about how the PSRE can be better utilised by the department and across government.
By gathering evidence to inform these steps, the review team should develop an evidence report and, if necessary, convene an expert panel to support an assessment of the evidence and any recommendations developed. The Lead Reviewer and Review Team should also consider the option of consulting with the PSRE’s Scientific Advisory Committee (if one exists) on the findings and recommendations of the review. Evidence from previous reviews can, and should, be leveraged where possible if the original evidence remains relevant and applicable. The evidence report can then be used for a variety of outcomes:
the report will be submitted to the Cabinet Office as part of the ALB review process and can be disseminated further (including to Permanent Secretaries), to enhance visibility of PSREs across government
if necessary, the report can inform an independent expert panel assessment of the impact of the PSRE and its scope for further improvements
the report should include, or be used to develop, recommendations to stakeholders within the host department, including Chief Scientific Advisers or sponsor committees, around enhancing the impact and opportunities of a PSRE
The outcome of this review process will play a crucial role in boosting the impact that PSREs have, and in enhancing awareness of the role that PSREs play in the RDI landscape. This will be especially important in the context of the wider governmental push to increase spending on R&D, with an aim to spend £22 billion annually by 2026 or 2027 and hit an overall R&D spending target of 2.4 percent of GDP by 2027.
This section outlines how to gather evidence and carry out an evaluation of a PSRE. It is separated into the four assessment strands. Departments should collect the evidence outlined in this section into the form of an evidence report.
Having a clearly defined mission is vital to set the overarching direction and scope of a PSRE. Mission statements should, in a paragraph or less, accurately define why the PSRE exists and what it hopes to achieve over the mid- to long-term. With a clearly outlined mission statement, an evaluation of the impact of a PSRE and its potential scope for improvement can then be carried out in relation to the mission.
Given that there are a very wide range of PSREs with varied scopes and providing different services, three broad categories are outlined within which mission statements can fall. Any individual PSRE may have multiple objectives that cover one or more categories, and the categories are defined as follows:
Development and exploitation of new technologies and knowledge assets: The PSRE plays a role in research and development, enhancing existing or developing new technologies, products, services, and other knowledge assets (models, methods, procedures).
Policy-making and regulatory support to government: The PSRE provides crucial research, information, monitoring/surveillance capabilities, data, and scientific advice to inform government policy.
Operational science services to government, business, and society: The PSRE provides operational science services (for example, forecasting, measurement, collection custodians, horizon scanning) to government, society, and businesses.
As an example, the Defence, Science and Technology Laboratory (Dstl) delivers new technology that enhances military capability across land, sea, air, cyber and space (technology development), and provides technical advice to support counterterrorism (policy-making support). In these cases, it is important to be clear on the different mission types as the assessment tools will likely differ (see below).
Table 1
*Examples are illustrative and not exhaustive of PSRE objectives.
While PSREs’ missions often align with the policy and operational needs of its sponsor department, a PSRE’s mission may be more cross-cutting to align with wider government objectives. In turn, appropriate strategic leadership is required to ensure a PSRE’s mission maximises value to its sponsor department or wider government priorities. To understand whether a PSRE is carrying out the best and most relevant research, development and innovation, judgement is required from a range of stakeholders. A non-exhaustive list of specific review questions is provided below, to help inform this judgement.
Recommendation: The first step of the PSRE Value Framework is to identify the PSRE mission statements and understand within which mission categories these fall, before reviewing this mission in the context of the PSRE’s sponsor and wider RDI landscape.
The second evaluation section is to understand a PSRE’s progress towards its mission. To carry out a comprehensive review that reflects all the objectives within a PSRE’s mission, a variety of quantitative and qualitative tools can be used – as outlined in the table below. Each evaluation measure is evidence that can be used to enable a full, context-dependent evaluation of a PSRE.
If assessments of the impact of a PSRE have already been carried out, the Review Team should refer to these and include their findings in the evidence reports that they compile, if the original assessment remains relevant and applicable. Examples of such reports could include looking at societal impacts, assessing returns on investment, or commissioned economic cost-benefit analyses.
For the impact assessment, there are two sets of measures that offer different advantages:
Broad, mission-relevant narrative: an overarching narrative, whilst difficult to quantify, is a useful measure of the overall health of the PSRE and demonstrates the degree to which the PSRE is progressing towards its mission statements.
Narrow, mission-relevant metrics: these indicators can be used to provide factual evidence of specific aspects of PSRE activity – they are useful in understanding the exact role and impact of a PSRE, within its RDI context.
In deciding which evaluation tools to use, both categories should be made use of fully.
Broad, mission-specific measures, such as PSRE self-reporting on their progress, provide a good qualitative narrative as to what an individual PSRE views as their achievements and provides apt framing for more specific indicators of progress later in the review process.
Narrow metrics are able to shine light on specific PSRE strengths and weaknesses, and provide evidence in support or disagreement of the overall PSRE narrative. These indicators are specific for the different mission categories, and must not be interpreted in the wrong context. (For example, it would be inappropriate to evaluate the commercialisation activity of a PSRE that has a purely regulatory role.) Care must therefore be taken when selecting narrow indicators to ensure they measure the right aspects of PSRE activity relevant to a PSRE’s objectives.
A wide, non-exhaustive range of suggested indicators across these two categories can be viewed in the table below. PSREs may already have specific indicators relevant to their functions and these indicators should be considered in the ‘narrow indicators’ assessment. In addition, review teams should consult the PSRE on the most relevant narrow indicators and these should be agreed in advance of compiling the final report. Finally, the results from these indicators should be considered in the context of the PSRE and comparisons between other similar PSREs or ALBs should be used as a starting point only in understanding how a PSRE might be improved.
Table 2: Broad and Narrow indicators, oriented around a PSRE’s mission, to guide evaluation of a PSRE’s impact
Recommendation: Agree between Lead Reviewer, Review Team, PSRE Chief Scientist (or equivalent) and departmental CSA the most appropriate performance measures for the PSRE in the context of its mission and carry out a comprehensive assessment of impact based on those performance measures.
As well as the value generated from directly delivering on their missions, PSREs add value to society in several more indirect ways. These can include developing skills of employees, enhancing local economic productivity and supporting the UK’s RDI landscape.
An assessment of these impacts, which are mission-agnostic, can be carried out by selecting additional value indicators. These enable review of aspects of PSRE value outside of direct delivery of mission-oriented objectives:
Table 3: Indirect value drivers, used to assess wider, non-mission specific impacts of a PSRE.
Recommendation: Agree between Lead Reviewer, Review Team, PSRE Chief Scientist (or equivalent) and departmental CSA the most appropriate indirect value drivers to measure and use this to achieve an understanding of the wider impact and added value of the PSRE.
The fourth key evaluation section is to assist departments in understanding the areas of potentially improved use of a PSRE. Identifying areas for PSRE improvement can be separated into three main sections. Key review questions for each section can be seen in the table below.
First, critical to PSRE improvement are effective internal mechanisms for monitoring and evaluating organisational progress and creating feedback loops to enable learning from activities. The performance measures identified through this Value Framework should be revisited at appropriate time frames and reported to the department’s CSA (the time frames, formats, and routes for reporting should be agreed between the PSRE and the CSA in accordance with the governance framework agreement in place). This will provide a consistent approach to monitoring and evaluation and, with the right data collection infrastructure in place, prevent excessive future reviews and review fatigue.
Second, identifying what current barriers and enablers exist that hinder or drive PSRE performance can shape decisions around to better utilise it. Key barriers that might hinder PSREs include governance structure, funding gaps and low visibility across government. Key enablers may include investment in existing or new capabilities expected to drive future performance and productivity.
Third, review questions can be used to identify specific areas for further PSRE alignment with government priorities, either through current pipeline projects or PSRE capabilities.
Table 4: Areas to assess for improved use of PSRE.
Recommendation: Evaluate the scope for improvement in performance of the PSRE, potential further opportunities, and scope for enhancing its impact and visibility across government.
The above evaluation tools and evidence questions provide a strong foundation to develop an evidence report. The evidence report must cover the four evaluation sections, with an emphasis on understanding the impact of the PSRE (Question 2) and its scope for further opportunity (Question 4).
As a minimum requirement, the final report should be reviewed by the ‘independent reviewer’ in charge of the ALB process within a department, and then submitted to the Cabinet Office, as part of the departmental submission to the wider ALB review process.
In situations where a PSRE is found to either be having a minimal impact relative to its mission(s), or where there are substantial opportunities for improving the performance of the PSRE, this evidence report should then be submitted for review by an expert panel. The panel can provide judgement on the four assessment questions, and should be composed of experts in the field of the PSRE, or with significant relevant background:
Diverse stakeholders should be considered, including experts in the field alongside the independent ALB reviewer, departmental Chief Scientific Advisor, and senior departmental officials.
The Lead Reviewer and Review Teams should also consider the option of consulting the PSRE’s Science Advisory Committee (if one exists) on the findings and recommendations of the review.
Finally, based on the evidence report, the PSRE together with the lead independent reviewer (or the expert panel) can make recommendations on potential routes for increasing the impact of the PSRE, removing barriers to its success, and enhancing its visibility and role. These recommendations should then be passed on to relevant departmental and cross-government stakeholders, including GO-Science where necessary, to ensure they are implemented.
Recommendation: Collect the evidence required for the review in the form of an evidence report and submit to Cabinet Office. Decide whether a further review by an expert panel is required and pass on to relevant stakeholders any recommendations stemming from the evidence report.
Information – research or acquiring data.
Experiments – conducting experiments or tests.
Prototypes – Building out elements of a solution to gather information such as user feedback or to confirm feasibility.
Algorithms – developing and implementing steps that solve a problem.
Infrastructure – foundational elements that support things like facilities, applications, systems, and business processes.
Tools – for example, technology tools such as a business application are viewed as the objectives of a business.
Automation – Systems and equipment that automate work.
Quality – improving the quality of technology services, e.g., improving the availability of an application.
Measurement and calculations – developing measurements, calculations, metrics, reports, and benchmarks.
Explainer, https://royalsociety.org/topics-policy/publications/2020/uk-research-organisations/ 
List of public and non-profit research organisations, https://royalsociety.org/topics-policy/publications/2020/uk-research-organisations/ 
Don’t include personal or financial information like your National Insurance number or credit card details.
To help us improve GOV.UK, we’d like to know more about your visit today. We’ll send you a link to a feedback form. It will take only 2 minutes to fill in. Don’t worry we won’t send you spam or share your email address with anyone.

source