Predicting Citations in Dutch Case Law with Natural Language Processing

Iris Schepers, Masha Medvedeva, Michelle Bruijn, Martijn Wieling, and Michel Vols published their new work titled ‘Predicting Citations in Dutch Case Law with Natural Language Processing’.

With the ever-growing accessibility of case law online, it has become challenging to manually identify case law relevant to one’s legal issue. In the Netherlands, the planned increase in the online publication of case law is expected to exacerbate this challenge.

In this paper, we tried to predict whether court decisions are cited by other courts or not after being published. Thus, in a way, we can distinguish between more and less authoritative cases. This type of system may be used to process large amounts of available data by filtering out large quantities of non-authoritative decisions.  Implementations may help legal practitioners and scholars to find relevant decisions more easily. In turn, this will drastically reduce the time spent on preparation and analysis. For the Dutch Supreme Court, the match between our prediction and the actual data was relatively strong (Matthews Correlation Coefficient of 0.60).

Our results were less successful for the Council of State and the district courts (MCC of 0.26 and 0.17, relatively). We also attempted to identify the most informative characteristics of a decision. We found that a completely explainable model, consisting only of handcrafted metadata features, performs almost as well as a less well-explainable system based on all text of the decision.

The paper was published open access, and the work can be found in Artificial Intelligence and Law.