Researchers at Utrecht University Develop an Open-Source Machine Learning (ML) Framework Called ASReview to Help Researchers Carry Out Systematic Reviews

Scientists often start their research on a topic by reviewing previous study findings. Therefore, conducting systematic literature reviews or meta-analyses can be very demanding and time-consuming. There may be an extensive amount of research available focusing on different topics, which may not always be relevant to a researcher’s work.

Researchers at Utrecht University have developed a machine learning framework that could significantly accelerate this process by automatically running through numerous past studies and compiling high-quality literature reviews. This framework is called ASReview, which could prove particularly useful for scientists in researching the COVID-19 pandemic.

Jonathan de Bruin, a lead engineer in the study, states that going through all the available new literature in their domain is a very time-consuming task, mainly when they want to do this systematically. Professor Rens van de Schoot, one of the team’s researchers, set out to develop a technique that would significantly speed up conducting systematic reviews and meta-analyses. He is developing the method in collaboration with various machine learning experts, engineers, and information managers at Utrecht University.

The framework is optimized to find a metaphorical ‘needle’ or multiple ‘needles’ in a haystack. Automatically recognizing the most relevant studies about a topic can be highly valuable to scientists. The researchers trained their machine learning model using an interactive approach called active learning. According to de Bruin, the challenge for their machine learning framework is to minimize the number of irrelevant articles shown to the researcher.


Most machine learning systems are trained to classify images, texts, or other data accurately. In comparison, the system created by de Bruin and his colleagues is trained to analyze various documents available and determine which ones are relevant to a given research topic and which ones are not.

The ongoing pandemic required medical guidelines and searches for new treatments to be developed in record time. De Bruin worked together with the Allen Institute for AI, which published the most extensive database with the coronavirus’s academic literature. He and his colleagues made their automatic system for conducting systematic reviews public during the first few weeks of the COVID-19 pandemic. They believed that it could significantly speed up research about the SARS-CoV2 virus.

ASReview, a user-friendly version of the system, has since been used by numerous researchers to review past studies about the new coronavirus and contribute to more effective medical guidelines. In the future, ASReview can be used to conduct several systematic reviews and meta-analyses, which could ultimately accelerate research in various fields.

At the same time, De Bruin believes that it is crucial to ensure that interactive machine learning approaches are fully transparent and explainable. In the upcoming period, he wishes to portray the possibility to apply interactive machine learning responsibly in applications like legal documents and court verdicts.