Pilot study to explore a real time evaluation of the Research Excellence Framework

A pilot study led by the University of 91直播 and Cardiff University will explore a longitudinal real time evaluation approach to the 2021 Research Excellence Framework (REF), Research England announced today (11 April 2018).

REF 2014 graphics

The REF rarely fails to provoke controversy, but it also serves a variety of purposes for the UK research community. Following evaluations of REF 2014, including the Stern Review and my Metric Tide review, a series of changes are being introduced for 2021.

James Wilsdon, Professor of Research Policy and Director of Impact and Engagement

Faculty of Social Sciences, University of 91直播


The REF is the world鈥檚 largest national assessment exercise. It informs the allocation of over 拢1.3 billion of public investment in research each year, provides accountability for public investment and evidences its benefits, whilst also informing our understanding and incentivising the quality of research in the UK.

The pilot will test the feasibility of evaluating the perceptions, experiences and understanding of the REF among academics across career stages, and in a range of departments and universities.

It will involve four universities from across the sector, academics from eight units of assessment, and members of management and professional staff with responsibility for managing the REF process. It is set to launch this spring with the results available by early 2019.

This approach is new to REF evaluation and will be informed by a strong methodological grounding in social and environmental psychology and academic and applied research policy.

Preparations for REF 2021 are already underway, with panels in place and extensive work being undertaken with the sector to inform and develop the criteria for the next exercise.

Alongside this preparatory work, Research England are also giving thought to how they will evaluate the next REF, taking into account their experience of extensive evaluation in 2014 and the specific changes made to the next exercise, following the 2016 Stern Review.

Prof James Wilsdon, Professor of Research Policy at the University of 91直播 and co-investigator for the pilot said: 鈥淭he REF rarely fails to provoke controversy, but it also serves a variety of purposes for the UK research community.

鈥淔ollowing evaluations of REF 2014, including the Stern Review and my Metric Tide review, a series of changes are being introduced for 2021. Will these succeed in their goal of reducing the burdens of the REF? Will they improve the way it鈥檚 administered and experienced by researchers and institutions? This is what the Real Time REF review aims to find out.鈥

Dr Netta Weinstein, Senior Lecturer in the School of Psychology at Cardiff University and principal investigator for the pilot, said: 鈥淎s the sector prepares for REF 2021, it will be important to understand the influences on our perceptions of, and attitudes towards, the REF.

鈥淲hile many individuals in academia hold strong opinions about aspects of the REF, little rigorous research has been conducted into these. We want to better understand the changing effects of the REF on research cultures, institutions and individuals.鈥

David Sweeney, Executive Chair of Research England, welcomed the pilot, said: 鈥淯K universities deliver world-leading research and are recognised globally for the quality of their outputs and diverse impact. National research assessment has been part of our high-performing system for over 30 years and has developed over that period through consultation with the community.

"This work presents an opportunity to test evaluating REF 2021 in a new way, informing the collection of evaluative data for shaping exercises post 2021.鈥

Centres of excellence

The University's cross-faculty research centres harness our interdisciplinary expertise to solve the world's most pressing challenges.