“In the Netherlands, we do not have a quality assessment of scientific research coordinated by the government. Instead, the government made an agreement with universities and other research institutes that they take care of this quality assessment themselves through a protocol created by all the above-mentioned parties together: the Strategy Evaluation Protocol (SEP), previously known as the Standard Evaluation Protocol,” Koier explains. As described in the SEP, research institutes have to do a six-yearly self-evaluation of the quality, societal relevance and vitality of their research.
In the UMCG this self-evaluation is done per research institute. The assessment is executed by a committee consisting of researchers with the right expertise, who write an evaluation report and presents that to the Executive Board of the University of Groningen (UG).
The new approach of acknowledging scientific research caused changes in the SEP, of which Koier is well aware. She explains, “The SEP is adapted every six years; how it changes depends on the vision of the parties who determine the SEP: the VSNU, KNAW (Royal Dutch Academy of Sciences) and NWO.”
The science system is changing. The VSNU (Association of Universities in the Netherlands), NWO (Netherlands Organisation for Scientific Research), NFU (Netherlands Federation of University Medical Centres) and ZonMw (Netherlands Organisation for Health Research and Development) are developing a new approach to recognising and rewarding academics (source: NWO).
In the past years there has been a big discussion on how scientific research should be evaluated and what that evaluation means. That is expressed in the San Francisco Declaration of Research Assessment (SFDORA), which recognises the need to improve the ways in which researchers and the outputs of scholarly research are evaluated. “Methods that were once developed to get an overview of developments within scientific research, are now being used to acknowledge individual scientists, for which these methods are not suitable,” Koier adds.
One thing that changed in the SEP is the acknowledgment of scientists through the journals they publish in. In the previous SEP, scientists were acknowledged for publishing in highly appreciated journals (high impact factor) such as Nature and Science. Koier does not agree with this method: “A journal can have a great editorial office, but in the end the journal does not reflect the quality of one article. Now this is prohibited by the SEP, which is quite unique, because it was accepted six years ago.”
Another change is the focus on societal impact, although it is not completely new. Koier states that societal impact was already mentioned two SEPs ago. The emphasis on not just thinking about, but really generating societal impact, on the other hand, is a new development. In the new SEP, attempts are being made to provide options for researchers to show what societal impact their work has.
Besides the focus on societal impact, another change is the focus on the context in which science is conducted. That context has four different aspects, called specific aspects. Koier elaborates, “The first one, PhD Policy and Training, has always been part of the SEP. The new ones are: Academic Culture, in which integrity is included, Human Resources Policy and Open Science. This way, society can use the results of scientific research, but knowledge from society can also be used in scientific research.”
The change in acknowledging scientists and scientific researchers can be difficult for researchers. Koier explains, “For years they have been acknowledged by statistics of their publications; at first on productivity and later more on the H-index, an author-level metric that measures both the productivity and citation impact of the publications of a scientist.”
Now, they are asked to work on the societal impact of their research. Koier notices that it takes time before the developments of the SEP are incorporated in the daily life of researchers. “There are many researchers that are doing good things in terms of impact, for example, inviting non-academic partners to their research projects.
Koier believes that it is important to make scientists more aware of the opportunities that they have in terms of societal impact. “It is important to help scientists deal with non-academic partners who have a different view. In collaborations with non-academic stakeholders, you have to serve different interests, but these collaborations also offer many possibilities.”
