We use data science to find predictors for court decisions

Why does the EVICT project use a data-driven methodology to study legal data?

It would be impossible to manually analyse the thousands of court judgments. Therefore, the EVICT project adopts a data-driven approach that is unique in the legal discipline.

The project aims to contribute to the development of legal research methodology by complementing traditional legal research techniques with data science techniques.

Using citation network analysis, we aim to conceptualise the right to housing as a network of connected international housing rights and to conduct the first systematic empirical exploration of the impact of this right in datasets of case law from both national supreme courts as lower level courts.

With the help of machine learning techniques, EVICT will identify predictors for court decisions and will explain how these predictors (may) mirror the international right to housing. Combining traditional legal methods with data science techniques could – if successful – be used in future research projects to analyse data in other areas of the law.

This website functions as the EVICT project’s datahub and available datasets will be published under Findings & Data. The publications related to the project can be found here.

No, we are not. The EVICT project team thinks automatic systems should not be used instead of human judges. Please read our article The Danger of Reverse-Engineering of
Automated Judicial Decision-Making Systems (PDF) for more information on our thoughts on that.

Does the EVICT project use traditional legal research techniques?

Yes, it does. The EVICT project combines doctrinal and normative research techniques with methods from data science. The key characteristics of doctrinal legal analysis are that the legal texts are manually collected, read, summarised, commented upon, and placed in the context of the overall legal system.

Moreover, we will conduct an empirical-normative analysis and study the various arguments in case law on, for example, the role of the right to housing, used by the parties and courts in case law. These arguments are related to, for example, theories on the public/private law divide, the hegemony of property rights, and the relationship between national sovereignty and international law.

How does the EVICT project use citation network analysis to study legal data?

EVICT will apply citation network analysis techniques to conceptualise the international right to housing and to determine the impact of this right on national case law.

Legal texts, such as case law, can be conceptualised as a network because of their referential character.

The EVICT project identifies whether and how

1) international courts and committees refer to other judgments/decisions; and

2) national courts refer to judgments of international and national courts, and to committees.

Legal texts such as judgments are seen as the nodes in the network, and references between the cases as the edges. The research team collects citation data and calculates relevant network statistics.

EVICT combines these data with the information obtained by doctrinal analysis, empirical-normative analysis and a Systematic Content Analysis to make sure that the network also contains contextual information on the judgements.

How does the EVICT project use machine learning to study legal data?

The EVICT project will use Machine Learning (ML) techniques to analyse case law of lower level courts. Building on earlier research of our team, we will build a system that predicts whether or not the court will decide that the persons should leave their home.

If the model predicts the results adequately, we will subsequently analyse which words, sentences or facts make the most impact, and thus identify what are predictors for the court’s decision.

For this task, EVICT will use supervised ML, and use ML algorithms such as Support Vector Machine and Neural Network approaches. Such an approach requires a substantial amount of data. Adopting this approach in EVICT is possible, because the EVICT project has developed a unique, manually annotated dataset consisting of over 1100 eviction judgments of Dutch district courts and courts of appeal.

The EVICT project also aims to explain, for example, the predictors identified by ML. Therefore, the data-driven approach is combined with an empirical-normative analysis. The data will provide insights into the use of normative arguments in case law, which are related to the theories discussed above. This method is comparative in nature: the analysis shows how solutions adopted elsewhere function and the weight attached to normative arguments.

Watch this video presentation of our team member Masha Medvedeva to learn more about using machine learning to analyse case law.

Explore the pillars of our research:

Evictions & right to housing

Having a place to call home is a fundamental human right, but every year hundred thousands of people are evicted from their home. They lose their home involuntary.

Read More

International law

The right to housing is laid down in international and European law. National courts are at the centre of the complex interaction between national and international law.

Read More

Using data science in law

EVICT project adopts a unique data-driven approach. We combine traditional legal methods with data science techniques, such as network analysis and machine learning, to find and explain predictors for court decisions.

Read More

Publications on legal research methods and data science

  • Schepers, I., Medvedeva, M., Bruijn, M., Wieling, M., & Vols, M. (2023). Predicting citations in Dutch case law with natural language processing. Artificial Intelligence and Law. Advance online publication. https://doi.org/10.1007/s10506-023-09368-5
  • Medvedeva, M. (2022). Identification, Categorisation and Forecasting of Court Decisions. University of Groningen. Link: https://research.rug.nl/files/236807645/Complete_thesis.pdf
  • Medvedeva, M., Wieling, M., & Vols, M. (2023). Rethinking the field of automatic prediction of court decisions. Artificial Intelligence and Law, 31(1), 195-212. https://doi.org/10.1007/s10506-021-09306-3
  • Vols, M. (2021). Legal Research: One hundred questions and answers. Eleven International Publishing. https://www.boomdenhaag.nl/webshop/legal-research.
  • Vols, M. (2021). Methodologische kansen en beperkingen van gebruik van statistiek en machine learning bij de bestudering van rechtspraak. In P. W. J. Verbruggen (Ed.), Methoden van systematische rechtspraakanalyse: Tussen juridische dogmatiek en data science (pp. 125-141). Boom Juridisch. https://research.rug.nl/en/publications/methodological-opportunities-and-limitations-of-using-statistics-.
  • Medvedeva, M., Üstün, A., Xu, X., Vols, M., & Wieling, M. (2021). Automatic judgement forecasting for pending applications of the European Court of Human Rights. In K. D. Ashley, K. Atkinson, L. K. Branting, E. Francesconi, M. Grabmair, V. R. Walker, B. Waltl, & A. Zachary Wyner (Eds.), Proceedings of the Fifth Workshop on Automatec Semantic Analysis of Information in Legal Text (ASAIL 2021) (pp. 12-23). CEUR Workshop Proceedings. https://pure.rug.nl/ws/portalfiles/portal/189865814/Medvedeva_ASAIL2021_1_.pdf.
  • Medvedeva, M., Dam, T., Wieling, M., & Vols, M. (2021). Automatically identifying eviction cases and outcomes within case law of Dutch Courts of First Instance. In E. Schweighofer (Ed.), Legal Knowledge and Information Systems (pp. 13-22). (Frontiers in Artificial Intelligence and Applications; Vol. 346). IOS Press. https://ebooks.iospress.nl/doi/10.3233/FAIA210312.
  • Medvedeva, M., Xu, X., Wieling, M., & Vols, M. (2020). JURI SAYS: An Automatic Judgement Prediction System for the European Court of Human Rights. In S. Villata, J. Harašta, & P. Křemen (Eds.), Legal Knowledge and Information Systems: JURIX 2020: The Thirty-third Annual Conference, Brno, Czech Republic, December 9–11, 2020 (pp. 277-280). IOS Press.
  • Medvedeva, M., Vols, M., & Wieling, M. (2020). Using machine learning to predict decisions of the European Court of Human Rights. Artificial Intelligence and Law, 28(2), 237-266. https://doi.org/10.1007/s10506-019-09255-y
  • Vols, M. (2020), Juridisch onderzoek, The Hague: Boom Juridisch 2020.
  • Bruijn, L. M., & Vols, M. (2020). Upperdogs versus Underdogs: Judicial Review of Administrative Drug-Related Closure Cases in the Netherlands. Recht der Werkelijkheid, 41(1), 25-49. https://www.bjutijdschriften.nl/tijdschrift/rechtderwerkelijkheid/2020/1/RdW_1380-6424_2020_041_001_004
  • Bruijn, L. M., Vols, M., & Brouwer, J. G. (2018). Home closure as a weapon in the Dutch war on drugs: Does judicial review function as a safety net? International Journal of Drug Policy, 51, 137-147. https://doi.org/10.1016/j.drugpo.2017.08.003
  • Vols, M. & Jacobs, J.P.A.M (2017), Juristen als rekenmeesters: over de kwantitatieve analyse van jurisprudentie, in P.A.J. van den Berg & G. Molier (Eds.), In dienst van het recht, Den Haag: Boom Juridische uitgevers.
  • Vols, M., Tassenaar, P.G. & Jacobs, J.P.A.M. (2015), Dutch Courts and Housing Related Anti-social Behaviour. A first statistical analysis of legal protection against eviction, International Journal of Law in the Built Environment, 7, pp. 148-161.