Self-citations in around a dozen countries are unusually high

Academics in 12 countries are excessively citing their own scholarly papers, according to an analysis of 24 years of data.

The authors of the study, published in PLoS ONE on 29 December¹, say that policies in these countries that incentivize or encourage such citation behaviours are probably behind the trend.

For the study, they evaluated the citation patterns of the 50 countries with at least 100,000 publications from 1996 to 2019 indexed in the Scopus research database.

They found that in 38 of the countries, the number of ‘country self-citations’ — in which researchers in a particular country cite their own papers — decreased over the 24-year period.

But in Colombia, Egypt, Indonesia, Iran, Italy, Malaysia, Pakistan, Romania, Russia, Saudi Arabia, Thailand and Ukraine, researchers increasingly cited their own papers over the same time period.

The same authors had already published a study² in 2019 suggesting that researchers in Italy were excessively citing their own work and that of other scientists in the country. They linked this to the 2010 roll-out of a controversial policy, which is still in place, that required academics to pass productivity thresholds — based in part on publication volume and citations — to be considered for promotion.

Citations show gender bias — and the reasons are surprising

For the 2019 study, the researchers measured the phenomenon using a metric they dubbed ‘inwardness’. This broader definition of country self-citation is computed using citations from any author based in the same countries as the authors of a given paper. It also had no limits on how long ago papers could have been cited.

For the new study, they created a new metric, SR^II. This type of country self-citation is defined more narrowly: in an SR^II citation, the authors of a work cite their own previous work within a certain timeframe. The self-citation window was introduced to correct for the fact that older papers have more time to garner citations, and to make it easier and fairer to compare self-citation rates between countries, says Alberto Baccini, an economist at the University of Siena in Italy, who co-authored both studies.

Legislative reasons

As part of the new study, Baccini and his colleagues analysed the relevant policies in countries with unusual citation patterns. They concluded that all 12 nations in which self-citation had risen have policies that directly or indirectly incentivize high citation counts. Baccini acknowledges, however, that his study didn’t examine policies in the remaining 38 countries, because it’s difficult to retrace all legislation in every nation.

“We were surprised to have such strong results with such a narrow definition of country self-citations,” Baccini says, referring to SR^II. He adds that countries should abandon policies that reward scientists for their performance in metrics such as citation rates. “This is a major problem.”

Jevin West, an information scientist at the University of Washington in Seattle who has studied self-citations, says that although it will be arduous, analysing the policies of the other 38 countries is crucial. “That is the most important missing piece in this paper,” he says, because that would allow researchers to assess the influence of policy on all self-citation patterns. Other limitations include not studying non-English-language literature or research outputs in the rest of the world, West notes.

Surge in number of ‘extremely productive’ authors concerns scientists

For most countries, both metrics used by Baccini and his co-authors — inwardness and SR^II — are highly correlated and show a similar trend. One notable exception is China, where the inwardness metric has been rising over time but the SR^II metric has decreased. Baccini attributes this trend to increased international collaborations involving authors in China, because inwardness counts citations by researchers based in other countries involved in the cited paper.

Marco Seeber, a sociologist and public-policy scholar at the University of Agder in Kristiansand, Norway, says the new study is more “methodologically sound” than the 2019 one, noting that the authors fixed some pitfalls in the measures used previously. “It’s pretty convincing of the fact that you have a relationship between certain policies and certain behaviours,” he says.

Seeber co-authored a 2019 study³ that compared the rates of self-citation by researchers in Italy before and after the controversial 2010 policy was introduced and found a significant spike after the legislation was rolled out.

West says he would be surprised if there was no link between self-citations and extraneous factors, such as research policies or researchers’ backgrounds. He co-authored a 2017 study⁴ examining the citation patterns of more than 1.5 million papers, and found that men cite their own papers, on average, 56% more than women do.

New evaluations are needed

West says there are alternative approaches to solely using metrics to evaluate research performance — and these can avoid incentivizing the wrong behaviours. “I think the alternatives to this are costly, but they’re much better evaluations,” he says, pointing to approaches that mix peer review with metrics, for instance.

Baccini notes that his findings don’t mean that only these dozen countries have bad incentives for researchers. An environment in which scientists feel a pressure to produce articles to justify their employment — the ‘publish or perish’ culture — is “everywhere”, he says. “But these anomalous countries have reinforced the bad system of incentives of publish-or-perish by administrative law.”

For West, analyses that look at the degree of self-citation and within-country citation should help to detect when policies don’t “necessarily align with our grander goals of science, which would be to encourage scientists to do good science rather than good manipulation of citations”.