Research Intelligence conducts benchmark study for Cancer Center Amsterdam

Data analysis and comparison with 20 other national and international research institutes.

03/09/2020 | 2:39 PM

In order to understand how the Amsterdam UMC Cancer Center Amsterdam relates to other research institutes around the world, the University Library was commissioned by them to conduct a benchmarking exercise.

‘We collected data from 21 institutes in two weeks' time. That in itself was quite a job,’ says René Otten, member of the University Library's Research Intelligence team. We asked Gitta Kuipers, programme manager at the Cancer Center Amsterdam, how the assignment went. ‘In 2016 I was asked to merge the cancer research institutes of the AMC and VUmc, the two parties that became Amsterdam UMC. Together, we employ nearly 1500 researchers who are conducting cancer research in all sorts of areas, from fundamental laboratory research to clinical hospital research. Across the board, these scientists study very many different types of tumours.’ GittaKuipersKlein

Cancer Center Amsterdam (CCA) is part of Amsterdam UMC, which started in June 2018. The CCA was curious to know its status in relation to other research institutes. What are its strengths and in what types of research is it lagging behind other institutes? ‘We wanted an analysis, a comparison. We wanted the data of national and international parties in the field of cancer research to be compared for the benchmark, the ones we consider to be important. This is the question we laid before the University Library.’

Data from the CCA and 20 other institutes
Creating the benchmark involved eight Dutch academic centres, various research centres in Europe and cancer institutes across from the rest of the world. The Research Intelligence (RI) team set out to find answers to the CCA's questions. Together with a number of RI colleagues, René Otten began work on this massive assignment. ‘We collected data from the CCA and 20 other institutes in two weeks' time. How was the CCA performing compared to these organisations?

13 types of cancer
And we also wanted to know the scientific impact of the research institutes with regard to 13 types of cancer. How is the CCA doing in comparison to these other institutes for each type of cancer?’ Kuipers adds: ‘René and I sat down together and for every type of cancer we determined which category it belonged to. For example, we categorised kidney cancer, bladder cancer and prostate cancer as urological cancer. Or take gastrointestinal cancer, which is a very broad category. Many types of cancer can be classified as this. We looked at what the primary types of tumours were for us with regard to the research we conduct and included these in our benchmark.’

Once René Otten had started work, based on the category division, he and Gitta Kuipers got together again to fine-tune the categorisation. Continuing to categorise, determining what belonged to each other, looking things up, reaching agreement. Kuipers: ‘We drew up as complete a list of search terms as possible, using the types of tumours.’ This laid substantive groundwork for the benchmark. ‘Throughout the process, we made adjustments to a few things, for example when we realised that sometimes the search results yielded publications that had nothing to do with cancer. That means you have to make adjustments.’

‘We can clearly understand our status now.’
Kuipers continues: ‘In the lead-up process, the Research Intelligence team asked us questions like: Which time period will you currently be including in the comparison with other institutes? Will you only be looking at a journal's impact factor, or will you be looking at citations throughout the years? And which years do you need to include in order to form a good impression? That's very nice, of course, since they have the most knowledge about these matters.’

‘The things we looked at concerned the scientific articles published by the CCA,’ Otten goes on. ‘How often were the researchers cited? What were the journals they published their articles in? How often were the scientific articles published in the top 1% of the most cited journals?' Among other things, the comparison made by the RI team showed that over 10% of the CCA publications related to brain tumours made the list of the top 1% of most cited articles, versus 5% for the institutes the CCA was being compared to, and versus 1% for the world average.

Overview and insight
Kuipers: ‘Thanks to the research, we can clearly understand our status now. For each domain, we now have an overview of our status compared to other Dutch institutes and to ones abroad. All of the categories we defined are covered in the report: the number of publications, a journal's average impact factor. We publish a lot on gastrointestinal tumours, which was something we were already aware of. However, the average journal impact factor is somewhat lower than average. Although, this isn't necessarily a problem, you can use this information to determine whether you want to address this. Within your own institute, you can clearly see which areas we are doing better in than others. Otherwise, you can also look at the analysis of all institutes taken together, or you can look at the individual institutes. It's really wonderful: you can look at each institute's profile or combine these into a single representation. You can really get a lot of information out of these analyses.’ Kuipers is clearly enthusiastic about the way the report has been presented by the RI team.

Graphic depiction of the results
In addition to a document on paper including a graphic depiction of the results, Jordy Gevers, an RI team member, also created an online version of the analysis. This can be used for a ‘self-service’ visualisation. For such a visualisation, you can select the institutes and types of cancer you want depicted in a diagram or chart yourself. The CCA itself provided the factual interpretation of the benchmark. ‘The University Library explained how you should read the report, and interpret the figures and overviews. The conclusions we make about these, are ones we came to ourselves.’

The added value of the CCA
As programme manager, Kuipers works on bringing together CCA researchers on a daily basis. She organises meetings in order to encourage collaboration between the AMC and VUmc locations. ‘As a result, people are getting to know each other better and now really seek each other out. They work together more often, which really creates added value. We are no longer doing the same thing at two locations, without being aware of the other's activities. It is also becoming increasingly irrelevant at which location someone is employed. And this has also become less noticeable.’

Do you want to know what Research Intelligence can mean for your work? Please contact the Research Intelligence team of the University Library.

Would you like to know more? Read What the University Library's Research Intelligence service can do for researchers.