Research analytics

The University of Leeds position clearly states that expert judgement is the primary way to assess the contribution to knowledge that a piece of research provides. As a member of a recruitment or promotion panel you may be thinking about using research metrics to analyse the research outputs of a candidate.

Consider inherent bias

It is important to be mindful of biases within the scholarly publishing industry and to remember that different researchers will have had different publishing opportunities based on their own circumstances.

Some of the considerations you should take into account if you are thinking about using research metrics within recruitment or promotion panels include gender and racial inequalities and the effect of geography.

Gender inequality

Evidence shows that gender inequality (Nature, PDF) still exists in research, for example:

• men publish more papers on average than women
• women are less likely to participate in collaborations that lead to publications
• women are much less likely to be listed as first or last author.

Regarding research grants, female academics also report less access to academic and funding resources when compared to their male counterparts.

Lack of diversity and racial inequalities

There is a lack of diversity in the scholarly publishing industry, there are racial biases in peer-reviewing and the publishing process and there are racial inequalities in research funding.

For example, research has shown that researchers of colour receive less scientific and health funding than white researchers, both in the UK (New Scientist, 2020) and the US (Science, 2019).

Geographical influence

Geographical backgrounds can also influence research productivity and impact, for example, not all countries have the same resources and investment for research and publication.

Recommendations for using metrics for recruitment and promotion

The following are recommendations that you might want to consider if you are thinking about using research metrics during recruitment processes:

• evaluate and consider all types of research outputs (not just articles) and evaluate all the researcher’s contributions, not only those included in the large commercial databases
• metrics can depend on the “academic age” of a researcher as some author metrics grow as citations accumulate
• avoid using journal metrics, such as the journal impact factor, to evaluate candidates or research articles
• be transparent about the measures you are using to evaluate research
• focus on quality over quantity
• find out about the contributions and roles researchers had in their research, for example looking at the CRediT role (if their publications use this taxonomy)
• ensure that any quantitative evaluation supports qualitative, expert assessment
• do not compare researchers against each other – they will have had different publishing opportunities
• be cautious when making comparisons between disciplines because publication patterns vary from one discipline to another.