Niels Buus Lassen, PhD Fellow at the Department of Digitalization, reports that the same issue for PhD dissertations.

“I’ve checked the citation rates for PhD dissertations, and they are, unfortunately, not that high. And that’s a shame. If we had more transparence in downloads of PhD dissertations it could lead to a better debate about dissemination,” he says.

Niels Bjørn-Andersen and Niels Buus Lassen are both very critical about how their work, which also includes teaching, fundraising and dissemination, is being evaluated and measured. Because, although a researcher’s work is multi-faceted, Niels Bjørn-Andersen argues that what matters for researchers’ careers and their pay checks depends almost exclusively on the number of scientific articles they manage to publish, which is summarized in a researcher’s h-index.

“I was part of running a research project with a DKK 41 million budget, and if it was a success, we could establish a lot of jobs in Denmark. However, my co-director of the project from another university was exclusively interested in how many scientific articles they could get published,” says Niels Bjørn-Andersen, who has published many scientific papers and has an h-index is 30.

Both of the researchers think it’s time to turn away from the rather one-sided way of measuring and evaluating research, and they have an idea on how it can be done. But we’ll get back to that.

Are you on the Twi-Li Index?

The discussion of how researchers and their research should be measured is ongoing, but has recently been reignited by the launch of a new index. The Twi-Li Index.

The Twi-Li Index measures researchers’ power of penetration on the Twitter and LinkedIn social media platforms, based on their number of followers. According to Mike Young, editor, historian of ideas and creator of the index, the two platforms are ideal for researchers wishing to disseminate their research and network with other researchers.

The new index lists the top 50 researchers who are doing especially well on social media in the Greater Copenhagen / Øresund region. Niels Buus Lassen is ranked 50 on the list and he has the company of Professor MSO, Mikkel Flyverbom (12), Professor Susana Borras (27) and PhD Stefan Kirkegaard Sløk-Madsen (34) all from CBS.

“The Twi-Li Index shouldn’t stand alone, but it can be used as a good indicator of how well a researcher is disseminating her or his research through other channels than the usual,” says Niels Buus Lassen, continuing:

“And I think that every researcher has an obligation to be good at this. If you have important knowledge, you should disseminate it and in an interesting way so that it’s picked up and used. Even if your research is a little boring, it’s your job to present it in an interesting way.”

Niels Bjørn-Andersen likes the Twi-Li Index and agrees with Niels Buus Lassen that it is a good measure of the extent to which researchers actively disseminate to a non-academic audience. But it shouldn’t stand alone.

“Actually, we should measure everything. Teaching, dissemination, fundraising and research. We have a model for it already at CBS, the REEAD model, but in my experience, it is not being used extensively. In the situation I have knowledge about, CBS would rather pick a candidate with a high h-index irrespective of relevance or impact of the research,” says Niels Bjørn-Andersen and continues:

“We need to ask researchers what society can gain from them. Who can use the research, for what, and for the benefit of whom? Will the research contribute to better health, economic growth, or more sustainability? Or we could ask, how will the research for instance contribute to the UN’s Sustainable Development Goals. We don’t ask those questions, but we should. We should make sure that the research we conduct is contributing societal relevance. If research isn’t doing that, why should taxpayers fund us?”

What’s your V-index?

Niels Bjørn-Andersen cares so much about this issue that two years ago, he created a counterpart to the h-index. The V-index.

“We need to supplement the almost meaningless counting of articles and calculation of h-index with a comprehensive measure of all dimensions in the REEAD model with a measurement of what I call the V-index, where the V stands for value,” says Niels Bjørn-Andersen and points out that the V-index is made as a self-assessment exercise, where the individual researcher is evaluating him/herself and later calibrating the assessment with his/her manager, for example, head of department as part of the annual development talks.

The V-index is described in a paper, and measures value based on two categories – relevance and effect. According to Niels Bjørn-Andersen’s paper, the relevance of academic research is measured on the extent to which the research contributes to one or more of the following five areas:

• Economic, growth, employment, commercial benefit
• Health and welfare
• Public policy, law or service
• Culture, art, entertainment
• Quality of life and work

The proposal is that the researcher should categorize her/his research within one or more of the five areas mentioned above with a one-liner. For example: “This researcher primarily contributes to more effective use of IT in private organizations for commercial benefit” or “This researcher primarily contributes to solving malnutrition challenges in the Sahel region of Africa”.

The second dimension measures the societal or organizational effect of the research. This is measured on the five points below:

• The researcher is actively disseminating his/her research through public presentations, media work, exhibitions, etc. to relevant outside stakeholders
• The research results of the researcher are being picked up (utilized) by non-academic relevant stakeholders
• The researcher is playing an active role in offering research-based advice as a member of professional networks, public committees, public meetings, panels and user groups
• The researcher is working in partnership (advice, action research, collaboration and consulting) for non-academics (societal groups at local, country, region, global level and/or individual organizations, including industry, government agencies, non-governmental organizations, interest groups, etc.)
• The researcher is obtaining research funds from industry, government agencies or other societal stakeholders

“When Niels presented the V-index, I just thought ‘Wow, this is interesting’,” says Niels Buus Lassen.

When asked if Niels Bjørn-Andersen lives by his own index, he answers:

“I have never worked with a research problem that had no practical relevance. Furthermore, I try to disseminate the results as much as possible, and I thoroughly enjoy collaborating with industry in all sorts of ways. So yes, I try to excel on all five dimensions as much as possible,” he says and continues:

“However, we live in a publish or perish culture, and many of my publications have been crafted to satisfy the counting towards the h-index. This happens even though I know, the reviewers know, and the editor knows that the article doesn’t have any – as in any at all – relevance to anyone. Unfortunately. It takes time to break away from this given the current reward culture.”

Paper factories vs. research that matters

But has it always been like this? Have universities and researchers always been measured on their ability to publish scientific articles in top-tier journals rather than the usefulness of their research?

No, explains Niels Bjørn-Andersen.

“Back in the 1950s, the teachers at business schools were practitioners, often CEOs and directors of companies, who taught ‘best practices’ or ‘war stories’ about the best way to do supply chains, marketing, accounting and so on. This was not satisfactory, and big studies funded by Ford and Rockefeller foundations called for business schools to become more academic and more evidence based like Economics. Today, it’s all about being evidence based for the sake of evidence, and we have forgotten the content,” he says.

This tendency isn’t only evident at business schools world-wide, it’s also a trend that has been picked up by universities where the focus on high publication rates has risen, explains Niels Bjørn-Andersen.

“For many years, top UK and US universities like Stanford, Cambridge and Oxford have been at the top of the rankings. Now, we see universities from China jumping up the rankings. These top Chinese universities don’t see themselves as teaching institutions. They have become research factories. They hire hundreds of postdocs and use artificial intelligence to produce and publish thousands of scientific articles. Articles that almost nobody benefits from, but that will get the university climbing the ranks. This is what happens if you only focus on the number of published articles.”

However, just as the economic cycle fluctuates, Niels Bjørn-Andersen believes that the focus on the actual benefits and relevance of the research will increase and take over. Eventually.

“I’m sure the winds will change. Taxpayers will insist that we spend their money on research that actually makes a difference. There’s so much junk research out there. Of course, there’s also a lot of really good and important research. We just need more of it, and it’s about time that we reward researchers for doing and publishing research that is valuable to society instead of wasting their time” he says.

Research