Leeds University Library
Our website is changing

The Library website is changing. Our new website will go live at the start of semester 2.

Limitations of bibliometrics

Quality

  • A large number of citations does not automatically mean that a work is of high quality. A work may be heavily cited because other authors are refuting its research
  • Bibliometrics does not measure quality. It is important to put the data in context using a combination of metrics and other qualitative information where appropriate, such as funding received, number of patents, awards granted and qualitative measures such as peer review when evaluating quality of work.

Discipline Variation

  • Citations patterns differ greatly between disciplines so direct comparisons cannot be made
  • Bibliometrics predominantly focuses on journal article citations, but some disciplines such as the arts, humanities and social sciences publish research in different types of publication
  • Different fields of research publish at different rates. For example, in biomedicine, there is generally a much stronger culture of publishing in journals and citing the work of peers than in engineering which makes more use of conference papers

Database Variation

  • The bibliometric databases do not cover all research areas and do not index all publications. For example, conference proceedings or reports are often poorly covered. Results will vary depending on the database you use, so don't rely on just one

Bias and Discrepancies

  • Citation bias. People may inappropriately cite their own work, their colleagues, or work from the journals in which they publish. A number of bibliometric tools allow you to exclude self-citations
  • Experienced researchers have an advantage over early career researchers as they will have produced more outputs over a period of time and so will have more citations
  • There is a bias towards English language material
  • Time is needed before a meaningful citation analysis can be made, so new journals tend to fare badly
  • Bibliographic tools cannot always reliably differentiate between researchers who share the same surname and initials, meaning that citation counts may be inflated. Researchers can use unique researcher IDs to reduce the risk of this. Find out more about using author identification systems.