Our commitment to Responsible Research Assessment

We also place a high value on research with far-reaching impact on our academic disciplines, and across disciplinary boundaries, transforming our paradigms and theories, our understanding, our methodologies, creating new fields of research and generating significant new knowledge. We expect our staff to publish and promote their research outputs in ways that ensure the new knowledge generated is widely disseminated, and that the quality and significance of the research outcomes are recognised, maximising the potential for impact. We are strongly committed to open research, where that is possible, as means to ensure the global reach of our research both within and beyond academia.

The University of Birmingham is a signatory of the San Francisco Declaration on Research Assessment (DORA), The Declaration recognises the need to improve the ways in which research is assessed, and since its development in 2012, it has become a worldwide initiative covering all scholarly disciplines and all key stakeholders including funders, publishers, professional societies, academic institutions, and researchers.

We are committed to assessing the quality of our research fairly and transparently and in line with the DORA principles, through:

• Assessing research outputs and other research contributions based on their intrinsic merit, highlighting that the scientific/intellectual content, and the impact (or potential impact) of that content, is what matters.
  
  
• Avoiding the use of the title or impact factor of the journal in which the work is published (or venue/route of publication for other types of output) as a proxy measure of quality; and never relying exclusively on metrics in research-output assessment, but supplementing these with expert peer opinion.
  
  
• Being explicit about the criteria used to evaluate the outputs and outcomes of the research contribution of our staff, and the evidence we will seek to include in the evaluation.
  
  
• Recognising the value of all relevant research outputs (such as articles, books, artistic outputs, patents, data sets, and software etc), as well as other types of contribution to the research environment/ecosystem, such as training early career researchers, leading open research initiatives, delivering high-quality research impact and policy influence, or delivering high-quality public engagement with research.

In addition, where we are aware of circumstances that have impacted a researcher’s output, we will take these circumstances into account in any assessment of contribution, recognising that the range of impacts may be broad. Our commitment to DORA reflects the importance we place on inclusive and team-based research and team science, aligned to our wish to support and enhance the career opportunities of all our staff, at all stages in their careers. 

Our commitments

As a University, as individual researchers, and as decision-makers:

• We will not use journal-based metrics (e.g. journal impact factors) as surrogate measures of the quality of individual research articles (or venue/route of publication more generally in relation to other types of output) when assessing an individual researcher’s contributions, including in recruitment, promotion or internal funding decisions.
  
  
• We will be explicit about the criteria we use to reach recruitment, probation, and promotion decisions, highlighting that the scientific/intellectual content of a paper is more important than publication metrics alone or the reputation of the journal or publisher/location of publication.
  
  
• We will consider the value and impact of all research outputs (including patents, datasets and software, and, in the arts and humanities, items such as compositions, exhibitions and practice research outputs), recognising that quality in different disciplines is exemplified in different ways.
  
  
• We will appropriately recognise knowledge generation and attribution through approaches that take specific disciplinary practices and authorship guidelines into account.
  
  
• We will use a diverse range of responsible and fair metrics and indicators on personal and supporting statements as evidence of the impact of our individual published articles and other research outputs. Metrics and indicators will be used to support – not supplant – decisions made through peer review, and wherever possible we will assess the content and merits of the work rather than relying on indicators alone.
  
  
• Where we use metrics in the assessment of research quality, these will be selected to be appropriate to the discipline, output type or research activity in question; a single metric will never be used in isolation; and wherever possible such metrics will be normalised if comparisons beyond sub-discipline are being made.
  
  
• Given our duty to consider the impact on equalities in our decision making, we will avoid using metrics that reflect or introduce bias, or ensure that such bias is mitigated.
  
  
• We will challenge research assessment practices that rely on journal impact factors or other location-based indicators of quality, and promote best practice that focuses on the value and influence of a diversity of research outputs and outcomes.
  
  
• We will be attentive to a broad range of impact measures, including both quantitative and qualitative indicators of research impact, such as commercialisation data, or influence on policy and practice, as well as other contributions to the research environment such as training early career researchers, good open research practices, or delivering high quality public engagement with research.
  
  
• We will ensure that personal circumstances impacting on an individual’s ability to deliver research outcomes are taken into account in our assessment processes, where these circumstances are known to us and have affected research outputs.
  
  
• We will work to develop a culture in which a responsible approach to the assessment of research is the norm.

Open Research Board

In May 2020, the University established a Responsible and Fair Approaches to Research Assessment Task and Finish Group to explore how we could best integrate responsible and fair metrics principles and practices into our research environment. This Group oversaw the first couple of years of our work on responsible metrics. Leadership and oversight of activity has now been subsumed within our Open Research Board (ORB).

The ORB is chaired by the DPVC for Research Integrity and Governance. He is supported on the Group by academic colleagues drawn from all Colleges and professional services staff drawn from Library Services, Research Strategy and Services Division, Legal Services and University of Birmingham Enterprise.

We have designated Elizabeth Westlake as the University’s Responsible Research Assessment Officer, to support the delivery of the RRA action plan and facilitate the use of responsible research metrics at the University.

Key progress

We have:

• Ensured we have a clear statement of commitment to responsible and fair approaches to research assessment via this website.
  
  
• Reviewed and updated HR documentation prior as part of the Birmingham Academic Career Framework Programme.
  
  
• Prepared an action plan outlining tasks to raise awareness and promote compliance with our strategy to be undertaken over the next year and assigned clear responsibilities.
  
  
• Developed an approach to piloting the monitoring of progress towards delivering our key goals and action plan.
  
  
• Updated our recruitment, promotion and career-progression policies and practices as required to reflect our principles as articulated above.
  
  
• Developed appropriate guidance for staff involved in the above processes to ensure that they understand our principles and abide by them.
  
  
• Developed appropriate guidance for applicants/candidates that explains our principles and encourages them to provide as full a range as possible of types of research outputs and contributions to the research environment/ecosystems in addition to conventional publications.
  
  
• Ensured that our REF Output Review Protocol reflects our responsible research assessment commitments.

Our work programme, to be carried out during the 2025/26 academic year, will cover:

• The delivery of our communications and advocacy plan to support the ongoing culture change needed to embed these principles in our working practices at UoB.
  
  
• Maintaining a watching brief on the Coalition for Advancing Research Assessment (CoARA), and initiatives associated with this.
  
  
• Ongoing evaluation and monitoring of the current understanding of Responsible Research Assessment by our research community.

We will do this by:

• Informing and discussing our responsible research assessment principles and plans with academic colleagues through a variety of routes such as established College mechanisms (College and School Research Committees) and our early career networks and the like, and taking other opportunities as they arise to ensure our policy in this area is understood.
  
  
• Working collaboratively with colleagues across the sector to learn from best practice elsewhere.

We will continue to review progress on a regular basis and refine our plans and approaches as appropriate. This statement will therefore evolve as these plans and approaches evolve and as our discussions as a University community develop.