An Approach Based on Cross-Attention Mechanism and Label-Enhancement Algorithm for Legal Judgment Prediction

Abstract

Legal Judgment Prediction aims to automatically predict judgment outcomes based on descriptions of legal cases and established law articles, and has received increasing attention. In the preliminary work, several problems still have not been adequately solved. One is how to utilize limited but valuable label information. Existing methods mostly ignore the gap between the description of established articles and cases, but directly integrate them. Second, most studies ignore the mutual constraint among the subtasks, such as logically or semantically, each charge is only related to some specific articles. To address these issues, we first construct a crime similarity graph and then perform a distillation operation to collect discriminate keywords for each charge. Furthermore, we fuse these discriminative keywords instead of established article descriptions into case embedding with a cross-attention mechanism to obtain deep semantic representations of cases incorporating label information. Finally, under a constraint among subtasks, we optimize the one-hot representation of ground-truth labels to guarantee consistent results across the subtasks based on the label-enhancement algorithm. To verify the effectiveness and robustness of our framework, we conduct extensive experiments on two public datasets. The experimental results show that the proposed method outperforms the state-of-art models by 3.89%/7.92% and 1.23%/2.50% in the average MF1-score of the subtasks on CAIL-Small/Big, respectively.

1. Introduction

Legal Judgment Prediction (LJP) is a crucial task in legal artificial intelligence [1], especially in the civil law system, and its goal is to automatically predict judgment results according to factual descriptions of legal cases and established law articles. The civil law system, also known as the statutory law system, is a legal system that has been adopted by many countries including China, France, and Germany. In many jurisdictions, the excessive workload of a court can result in severe delays. Suitable LJP models can assist legal professionals by improving work efficiency. In addition, they can provide legal consulting to those who lack legal expertise. As a result, it has become a research hotspot. At present, LJP is generally regarded as a text classification task that comprises three subtasks, i.e., applicable article prediction, charge prediction, and the term-of-penalty prediction, as illustrated in Figure 1. Some studies attempted to solve several subtasks simultaneously [2], whereas others focused on solving only one [3]. They all proposed excellent LJP models. However, due to the complexity of the semantics of legal case descriptions and the characteristics of multiple subtasks, the existing methods still have several shortcomings.

How to utilize limited but valuable label information. Labels carry valuable information that can aid in text classification [4], and it has become common to incorporate labels’ semantic information rather than solely treating them as indexes. Accordingly, some existing methods integrate established articles’ contents when encoding cases’ factual descriptions [5]. However, they all take the approach of directly encoding the entire established articles for fusion. In Figure 1 and

Table A1. Examples of established articles and corresponding keywords.

Content of Established Articles	Crime-Keywords
Law Article 338—Crime of Polluting the Environment: Any person who, in violation of state regulations, discharges, dumps, or disposes of radioactive waste, waste containing infectious disease pathogens, toxic substances, or other harmful substances that seriously pollute the environment shall be sentenced to a fixed-term sentence of not more than three years imprisonment, criminal detention, or a fine. If the circumstances are serious, the sentence shall be fixed-term imprisonment of not less than three years but not more than seven years, and a fine shall be imposed. Under any of the following circumstances, the sentence shall be fixed-term imprisonment of not less than seven years and a fine.	Discharge, Wastewater, Chromium, Monitoring, Zinc, PH, Concentration, Environment, Pollutant, Environmental Protection Agency, Waste, Total Chromium, Department of Environment Protection, Standard, Exceeding Standard, Content, Monitoring Station
Law Article 266—Crime of Fraud: For defrauding public or private property, if the amount is relatively large, the sentence shall be fixed-term imprisonment of not more than three years, criminal detention, or public surveillance, and a fine. If the amount involved is large or there are other serious circumstances, the sentence shall be three years imprisonment. If the amount involved is substantially large or other extremely serious circumstances exist, the sentence shall be fixed-term imprisonment of not less than ten years or life imprisonment and a fine or confiscation of property. Where there are other provisions in this law, the provisions shall prevail.	Trust, Fake, Borrow Money, Bank Card, Cheat, Clothing Fee, Stolen Money, Tipping for Desk Fees, Hide, Fabricate, For the Reason, Squander, Take Possession of, Amount of Money, Real Situation, Lie, Mortgage, Bank, Defraud, Property
Law Article 193—Crime of Loan Fraud: Under any of the following circumstances, for the purpose of illegal possession or defrauding loans from banks or other financial institutions, if the amount is relatively large, the sentence shall be fixed-term imprisonment of not more than five years or criminal detention and a fine of not less than CNY 20,000 but not less than CNY 200,000. If the amount is substantially large or there are other extremely serious circumstances, the sentence shall be fixed-term imprisonment of not less than five years and not more than ten years and a fine of not less than CNY 50,000 but not more than CNY 500,000, or life imprisonment and a fine of not less than CNY 50,000 but not more than CNY 500,000.	Finance, Credit Union, Principal and Interest, Student, Borrow Money, Repayment, Loan, Credit, Take Possession of, Associated Agency, Fraud, Fake, Sub-branch, Bank, Mortgage, Cheat, Repay, Principal Money, Interest, Installments
Law Article 175—Crime of Fraudulent Loan, Bill Acceptance, Financial Documents: Any person who obtains loans, bill acceptances, letters of credit, letters of guarantee, etc., from banks or other financial institutions through fraudulent means and causes heavy losses to banks or other financial institutions shall be sentenced to fixed-term imprisonment of not more than three years or criminal detention and shall also be fined. Any person who causes substantially heavy losses to banks or other financial institutions or there are other extremely serious circumstances shall be sentenced to fixed-term imprisonment of not less than three years but not more than seven years and shall also be fined.	Finance, Credit Union, Company, Borrow Money, Loan, Acceptance Bill, Credit, Cheat, Fake, Associated Agency, Sub-branch, Bank, Maturity, Guarantee, Fraud, Principal Money, Interest, Exchange Bill, Repay
Law Article 192—Crime of Fundraising Fraud: Any person who illegally raises funds through fraudulent means for the purpose of illegal possession, if the amount is relatively large, shall be sentenced to fixed-term imprisonment of not less than three years but not more than seven years and shall also be fined. If the amount is substantially large or there are other extremely serious circumstances, the sentence shall be fixed-term imprisonment of not less than seven years or life imprisonment, a fine, or confiscation of property. Units that commit crimes in the preceding paragraph shall be fined, and the persons directly in charge and other directly responsible personnel shall be punished in accordance with the provisions of the preceding paragraph.	Borrow Money, Fundraising, Funds, High Amount of Money, Absorption, High Interest, Take Possession of, Amount, Monthly Interest, Fraud, Illegal, Cheat, Public, Investment, Repay, Principal Money, Interest, Bait, Raise Funds
Law Article 176—Crime of Illegally Absorbing Public Deposits: Any person who illegally absorbs public deposits or absorbs public deposits in a disguised form, thereby disturbing the financial order, shall be sentenced to fixed-term imprisonment of not more than three years or criminal detention and shall also be sentenced to a fine. If the amount is large or there are other serious circumstances, the sentence shall be fixed-term imprisonment of not less than three years but not more than ten years and a fine. If the amount is substantially large or there are other extremely serious circumstances, the sentence shall be fixed-term imprisonment of not less than ten years and a fine.	Finance, Member, Borrow Money, Disruption, Fundraising, Investors, Biao Hui, Funds, High Amount of Money, Absorption, Membership, Monthly Interest, Public, Repay, Investment, Bank Savings, Principal Money, Interest, Bait, Meeting Day

, it can be seen that the text in a factual description is distinctly different from the established articles. Specifically, most words in a factual description describe the event process. In contrast, an established law article concisely describes the event behavior using more digital-related words such as the amount of money, percentage, multiple, term of penalty, etc. The method of directly encoding established articles for fusion ignores the gap between the two. Zhong et al. [3] proposed that judging whether the law articles and cases are relevant mainly depends on whether the key elements are consistent with the law articles. They manually divided each established article into seven elements to bridge the gap. Inspired by their work, we automatically extract the keywords related to each charge from cases’ factual descriptions to form a keyword dictionary, which is used as the label information rather than the established articles. There are no gaps in the integration process since the keywords come from the factual descriptions of cases. Table A1 shows the keywords associated with the crime of polluting the environment that are closely related to the factual description in Figure 1. By comparison, the keywords have a more concentrated feature distribution and have no digital-related information that is insensitive to the model and difficult to learn. Furthermore, due to the complexity of the factual description, there is a lot of redundant information. Similarly, the crime-keywords dictionary constructed with an unsupervised method also has noisy data. This motivated us to fuse them using a cross-attention mechanism [6] to capture valuable information.

Subtask consistency constraint of labels in LJP. Most existing studies only consider the topological dependencies among the LJP subtasks [2] while ignoring the semantic or logical label relationships. Unlike topological dependencies, the label relationship exists naturally, and, as shown in Figure A1, each charge is only logically or semantically related to some fixed articles, and vice versa. LJP methods can predict unreliable results when ignoring this constraint. Recently, Dong and Niu [7] adopted relational learning to enforce a consistency constraint among the labels. However, there are confusing labels in LJP, and using a one-hot representation may cause the model to become over-confident. We use the label-enhancement (LE) [8] algorithm to modify the original one-hot label distribution based on the consistency constraints of a graph. By exploiting the network backpropagation algorithm, we could iteratively optimize the model parameters to gradually integrate the consistency constraint into the model.

In this paper, we propose an approach based on Cross-Attention Mechanism and Label-Enhancement Algorithm (CMLEA) for the task of LJP. Specifically, we first extract the keywords corresponding to each charge from the factual descriptions of cases to form a crime-keywords dictionary and then construct a charge similarity graph. Second, we use an improved cross-attention algorithm to help locate the key elements in the factual descriptions and discover the charge-related words from the graph. Finally, we add a subtask consistency constraint by modifying the original one-hot label distribution based on the LE algorithm. The contributions of this paper can be summarized as follows:

1.

We propose a new LJP method that avoids the gap in information fusion by using keywords extracted from cases as label-enhanced information instead of established articles.

2.

We propose a novel cross-attention fusion distillation mechanism to fuse keywords, which not only identifies the distinctive keywords of each case but also optimizes the representation of keywords into a distinguishable representation.

3.

We use the LE algorithm to add a subtask consistency constraint to the one-hot distribution of labels, which not only improves the rationality and consistency of the prediction results but also alleviates the issue of the over-confidence of the model caused by confusing labels.

4.

We conduct experiments on two real datasets and achieve excellent results, outperforming all baseline models.

The rest of this paper is organized as follows. Section 2 reviews the related research works. Section 3 shows some mathematical notations and formalizes the LJP task. Section 4 presents our framework and introduces each module step by step. Section 5 conducts experiments to evaluate the proposed model and analyzes the experimental results. Section 6 provides an ethical discussion of LJP. Finally, Section 7 concludes this paper.

2. Related Works

This section reviews the existing literature relevant to our research and points out the differences between previous works and our work.

2.1. Legal Judgment Prediction

LJP has been studied for decades. It is an essential and representative task in countries with civil law systems such as France, Germany, Japan, and China. The early works [9] adopted mathematical or statistical methods to analyze legal cases in specific circumstances. However, these methods were limited by the small amount of labeled data and required manually designed features. With the development of neural networks, many researchers began to explore LJP using deep learning methods. These works can be divided into two primary directions as described below.

The first is the adoption of a novel architecture to improve task performance. For example, Chen et al. [10] proposed a deep gating network using a gating mechanism to improve the performance of predicting the term of penalty. Pan et al. [11] proposed a multi-scale attention model to handle cases with multiple defendants. In addition, Wang et al. [12] proposed a unified dynamic pairwise attention model to predict multiple applicable articles. Li et al. [13] proposed an element-driven attentive neural network model to improve LJP.

The second is to leverage external knowledge or the inherent properties of LJP to improve performance. Hu et al. [14] incorporated ten discriminative attributes of charges to help predict low-frequency charges. Similarly, Guo et al. [15] constructed nine case auxiliary sentences to help predict low-frequency charges. Zhong et al. [16] integrated a real judgment process into the model by iteratively asking questions to detect elements in the factual description and predict the judgment results. Gan et al. [17] proposed integrating declarative legal knowledge as a set of first-order logic rules into a coattention network-based model. These methods have been proven to be effective for LJP. However, integrating external knowledge requires manual construction and annotation, which limits its transferability. Some researchers utilized the relationships among the subtasks. For example, Luo et al. [18] utilized the attention mechanism relationship between facts and law articles to help predict charges. Similarly, Bao et al. [19] used this feature to design a legal attention mechanism. Zhong et al. [2] formalized the dependencies among subtasks as a Directed Acyclic Graph (DAG) and proposed a topological multi-task learning framework. Based on the topology structure among subtasks, Yang et al. [20] proposed a multi-perspective bi-feedback network with a word collocation attention mechanism and Xu et al. [21] designed a novel attention mechanism to distinguish between confusing charges. Moreover, some researchers have explored how to incorporate established articles’ information into the model. Chen et al. [5] proposed an article-wise attention mechanism to enable the content of established articles to guide the prediction of multi-label legal judgment. Yue et al. [22] utilized the results of intermediate subtasks to separate the factual description into different circumstances, which they exploited to make predictions of other subtasks. Dong and Niu [7] formalized LJP as a graph-node classification problem. They utilized a masked transformer network to obtain case representations and achieved local consistency of each node’s label distribution through relational learning. However, these methods ignore the differences between the description of the established articles and the cases. To address this problem, Zhong et al. [3] manually divided each article into seven elements and labeled the various elements. In contrast, we use the keywords extracted from the factual descriptions as the label-enhanced information. Since the two have the same distribution, there are no gaps when they are fused. Furthermore, we add a consistency constraint to the subtasks to improve the rationality and consistency of the prediction results.

2.2. Attention Mechanism

Attention is a technique that focuses on different parts of an input vector to capture long-term dependencies. Mnih et al. [23] developed an attention-based RNN model for image classification, which popularized the attention mechanism. Subsequently, Bahdanau et al. [24] applied the attention mechanism to natural language processing for the first time. They added an attention mechanism to the encoder–decoder method for machine translation, which achieved good results. Yang et al. [25] proposed a hierarchical attention network for document classification, which has been used in LJP [18]. Xiong et al. [6] proposed a coattention mechanism that simultaneously focuses on both questions and documents for the task of question answering. Furthermore, Vaswani et al. [26] proposed a transformer based entirely on the attention mechanism and achieved excellent performance in machine translation tasks. Wu et al. [27] proposed a dual attention matching module to model the relationship between the global features of one modality and the local features of another modality. Similar to these methods, we propose a cross-attention mechanism that is suitable for LJP, which not only obtains the keywords related to each case but also optimizes the hidden layer representation of these keywords to make them distinguishable.

2.3. Label Enhancement

Representing truth labels with one-hot vectors is a common practice for text classification models. However, inadequate one-hot representations tend to train over-confident models, which can lead to arbitrary prediction and model overfitting, especially for confusing datasets (with very similar labels) or noisy datasets (with labeling errors). To address this problem, label-enhancement methods have been proposed.

Szegedy et al. [28] employed the label-smoothing (LS) mechanism, which calculates loss, not with a hard truth label (with one-hot representation) but with a weighted mixture of these targets by adding a noise distribution. Geng [29] proposed a new learning paradigm called label distribution learning (LDL), where the label distribution covers a certain number of labels, representing the degree each label describes the instance. Moreover, Xu et al. [8] and Gao et al. [30] proposed label enhancement (LE), which is a technique for recovering label distributions from a training set by leveraging the topological information in the feature space and the correlations among labels. However, a truth-label distribution is difficult to obtain for many existing classification tasks and we only have a unique label for each sample. Guo et al. [31] proposed a novel label confusion model as an enhancement component for existing popular text classification models, which inspired us to improve the label propagation-based LE algorithm by dynamically adding a consistency constraint for labels among subtasks to the one-hot representation of truth labels.

3. Problem Formalization

The corresponding notations are shown in

Table 1. Main mathematical notations.

Notation	Description
$X^d=\{x_1^d,\dots ,x_l^d\}$	a word sequence of the factual description of the case
$Y^a=\{a_1,\dots ,a_m\}$	the set of applicable article labels
$Y^c=\{c_1,\dots ,c_n\}$	the set of charge labels
$Y^t=\{t_1,\dots ,t_p\}$	the set of term-of-penalty labels
$D={\left\{d_i\right\}}_{i=1}^n,d_i=\{k{w}_1,\dots ,k{w}_k\}$	the dictionary of crime keywords

. Similar to existing studies, we formalize LJP as a document classification problem. The training legal case set can be denoted as $\Gamma ={(X_i^d,Y_i)}_{i=1}^N$, where N is the number of legal cases. In this paper, LJP aims to predict the applicable article, charge, and term of penalty for each case, so $Y_i=\{y_i^a,y_i^c,y_i^t\}$, where $y_i^a\in {\{0,1\}}^m$, $y_i^c\in {\{0,1\}}^n$, and $y_i^t\in {\{0,1\}}^p$, and m is the number of article labels. Correspondingly, n and p are the numbers of charge labels and term-of-penalty labels, respectively. Our goal is to learn a classifier f from $\Gamma$ that can predict the judgment results on data with unknown labels, i.e., $f\left(X_u^d\right)\Rightarrow \{\widehat{y_u^a},\widehat{y_u^c},\widehat{y_u^t}\}$, where $X_u^d\notin \Gamma$.

4. Methodology

This section introduces the framework of our proposed model and then presents each component.

4.1. Overview of the Proposed Framework

We propose an approach based on cross-attention mechanism and label-enhancement algorithm (CMLEA, see Figure 2) for LJP. Specifically, the CMLEA first builds a charge similarity graph as label-enhanced information based on the crime-keywords dictionary constructed from a set of cases. Then, the graph performs a distillation operation to extract a more discriminative node representation, $E^c$. Next, a cross-attention module is introduced to enable effective interaction between the factual description and label information. This module is followed by a fusion layer, which generates a case representation corresponding to each subtask. Finally, given the subtask consistency constraint, we use the labels of the subtasks as nodes to construct a consistency constraint graph and dynamically modify the one-hot representation of truth labels through label propagation to improve the rationality of the model’s prediction results.

4.2. Basic Encoder Module

We employ a basic encoder module to obtain the preliminary vector representations of the factual descriptions of the cases. Since Transformer [26] and its variants have achieved great success in many fields, e.g., Dong and Niu [7], we first encode word sequence $X^d$ using BERT [32] to obtain the hidden vector representations $H_d$ and $h^d$ of the factual description.

$$\begin{array}{cc}\hfill H^d& =BERT(x_1^d,x_2^d,\dots ,x_l^d)\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill h^d& =maxpooling\left(H^d\right)\hfill \end{array}$$

(2)

where $H^d=\{h_1^d,h_2^d,\dots ,h_l^d\}\in {\mathbb{R}}^{l\times d_h}$, l is the length of the factual description and $d_h$ is the dimension of the last hidden layer of BERT.

4.3. Crime-Keywords Fusion Module Based on Cross-Attention Mechanism

4.3.1. Crime-Keywords Dictionary Construction

We calculate the importance $\alpha$ of word $x\in X^d$ for each charge $c\in Y^c$ using YAKE [33]. Furthermore, in order to extract more representative words for each charge, we use the same method as Liu et al. [34], i.e., enhance YAKE with inverse document frequency [35].

$$\alpha =\frac{w}{log\left(\right(N+1)/(1+df\left)\right)}$$

(3)

where w is the importance score of word x calculated by YAKE, N is the number of documents, and $df$ is the number of documents that contain word x.

According to the score $\alpha$, we select the top k words of each charge to form a crime-keywords dictionary D with size $n\times k$. Unlike Section 4.2, since keywords are discrete and have no contextual semantic information, we obtain the corresponding initial embedding through Word2Vec [36].

$$E_{d_i}=Word2Vec(k{w}_1,k{w}_2,\dots ,k{w}_k)$$

(4)

where $E_D={\left\{E_{d_i}\right\}}_{i=1}^n\in {\mathbb{R}}^{n\times (k\times d_w)}$ is the vector representation of D; $d_w$ is the dimension of the word vector, as stated previously; and n is the number of charge labels.

4.3.2. Charge Similarity Graph Construction

As in Equation (4), we obtain a charge label’s initial representation with ${\left\{e_{c_i}\right\}}_{i=1}^n\in {\mathbb{R}}^{n\times d_w}$. Then, based on $E_D$, we improve the representation of each charge label, $e_{c_i}$, by incorporating the corresponding keyword representation $E_{d_i}={\left\{e_{k{w}_j}\right\}}_{j=1}^k$.

$$e_{c_i}^{*}=\frac{e_{c_i}+{\sum }_{j=1}^k{e}_{k{w}_j}}{k+1}$$

(5)

where $e_{k{w}_j}$ is the keyword representation corresponding to charge $c_i$.

Based on the representation, $e_{c_i}^{*}$, we employ the cosine similarity to calculate the weight of the edges between two charges. Furthermore, to focus on the most similar charges, we remove edges with weights less than a predefined threshold $\mu$.

$$\begin{array}{cc}\hfill s_{ij}& =sim(i,j)=\frac{e_{c_i}^{*}·e_{c_j}^{*}}{\left|e_{c_i}^{*}\right|·\left|e_{c_j}^{*}\right|}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill M_s(i,j)& =\left\{\begin{array}{cc}{s}_{ij}\hfill & i\ne j,s_{ij}\ge \mu \\ 0\hfill & otherwise\end{array}\right.\hfill \end{array}$$

(7)

For better convergence, we normalize the adjacency matrix through $M_s^{*}=P_s^{-\frac{1}{2}}{M}_s{P}_s^{-\frac{1}{2}}$. Here, $P_s$ is a diagonal matrix with the elements ${\widehat{M}}_s(i,i)={\sum }_{j=1}^n{M}_s(i,j)$. Finally, we obtain the charge similarity graph ${\mathcal{G}}_s=({\mathcal{V}}_s,{\mathcal{E}}_s)$, ${\mathcal{V}}_s$ as the set of nodes whose feature matrix is $E^{*}={\left\{e_{c_i}^{*}\right\}}_{i=1}^n\in {\mathbb{R}}^{n\times d_w}$. ${\mathcal{E}}_s$ is the representation of edges associated with the matrix $M_s^{*}\in {\mathbb{R}}^{n\times n}$.

The construction procedure is provided in Algorithm 1.

Algorithm 1 Construction of crime-keywords dictionary and charge similarity graph.

Input:   The legal case set $\Gamma$.   The set of charge labels $Y^c=\{c_1,\dots ,c_n\}$. Output:   Crime-keywords dictionary D.   Charge similarity graph ${\mathcal{G}}_s$.   1. Construct the crime-keywords dictionary D using Equation (3) based on $Y^c$ and $\Gamma$;   2. Obtain the vector representation $E_D$ corresponding to D using Equation (4);   3. According to the keywords of each crime in D, calculate the vector representation $e_{c_i}^{*}$ of each charge using Equation (5) and obtain $E^{*}={\left\{e_{c_i}^{*}\right\}}_{i=1}^n$;   4. Calculate the similarity $s_{ij}$ of each charge and obtain the similarity matrix $M_s$ using Equation (7). Then, normalize $M_s$ through $M_s^{*}=P_s^{-\frac{1}{2}}{M}_s{P}_s^{-\frac{1}{2}}$;   5. Obtain the charge similarity graph ${\mathcal{G}}_s=({\mathcal{V}}_s,{\mathcal{E}}_s)$ based on $E^{*}$ and $M_s^{*}$;

4.3.3. Distillation Operation

Due to the existence of confusing charges, we design a distillation operation to extract a more discriminative node representation, which was inspired by Xu et al. [21] and Yue et al. [22]. Unlike graph convolution [37] where aggregated node representations can become indistinguishable, especially in similarity graphs, we remove similar features between nodes in ${\mathcal{G}}_s$ through a distillation operation and then utilize the distinct ones to enrich the representation of the central node. The detailed calculation is as follows:

$$E_c^{(l+1)}=E_c^l-\frac{M_s^{*}{E}_c^{\left(l\right)}{W}^{\left(l\right)}}{\left|N_c\right|}$$

(8)

where $E_c^{\left(l\right)}\in {\mathbb{R}}^{n\times d_w}$ is the representation of the nodes at the $l_{th}$ layer and $E_c^{\left(0\right)}=E^{*}$, $W^l\in {\mathbb{R}}^{d_w\times d_w}$ is the trainable parameter. $N_c={\left\{\left|{\mathcal{E}}_{c_i}\right|\right\}}_{i=1}^n$ is the number of edges of each node $c_i$, that is, the number of non-zero values in each row of $M_s$.

4.3.4. Cross-Attention Module

The keywords corresponding to different cases can vary, and even similar cases can have subtle differences in their keywords. Therefore, we propose a cross-attention module that integrates L2F attention and F2L attention based on coattention [6]. Specifically, L2F attention is the interaction between the discriminative charge label representation $E_c$ and the document representation $H^d$ to extract the key elements in the case. F2L attention is the interaction between $h^d$ and the keyword representation $E_D$ to extract the keywords associated with each case from the dictionary D.

$$\begin{array}{cc}\hfill A& =softmax\left(\frac{\left(E_c{W}_q\right){\left(H^d{W}_k\right)}^{\mathrm{T}}}{\sqrt{d_h}}\right)\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill B& =softmax\left(\frac{\left(h^d{W}_q^{{}^{\prime }}\right){\left(E_D{W}_k^{{}^{\prime }}\right)}^{\mathrm{T}}}{\sqrt{d_h}}\right)\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill H^{lf}& =\lambda A\left(H_d{W}_v\right)+(1-\lambda )B{\left(E_D{W}_v^{{}^{\prime }}\right)}^{\mathrm{T}}\hfill \end{array}$$

(11)

where $W_q,W_k^{{}^{\prime }}$, and $W_v^{{}^{\prime }}\in {\mathbb{R}}^{d_w\times d_h}$ are the transformation matrices, $W_k,W_q^{{}^{\prime }}$, and $W_v\in {\mathbb{R}}^{d_h\times d_h}$, which are all parameters that can be learned during training. As described above, $A\in {\mathbb{R}}^{n\times l}$ and $B\in {\mathbb{R}}^{n\times k}$, where n is the number of charge labels, l is the length of the factual description, and k is the number of keywords for each charge. $softmax$ is performed across the column. $\lambda \in (0,1)$ is a balancing parameter that controls the fraction of the information inherited from the L2F attention and F2L attention mechanisms. $H^{lf}\in {\mathbb{R}}^{n\times d_h}$ is the output of the cross-attention module.

4.4. Fusion Layer

Given $H^{lf}$, we fuse the residual links of the factual description $h^d$ to obtain a case representation of the different subtasks, i.e., $H^a$, $H^c$, and $H^t$.

$$\begin{array}{cc}\hfill & H^c=W_c{h}^d+H^{lf}{W}_c^{{}^{\prime }}+b_c\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & H^a=W_a{h}^d+W_a^{★}{H}^{lf}{W}_a^{{}^{\prime }}+b_a\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & H^t=W_t{h}^d+W_t^{★}{H}^{lf}{W}_t^{{}^{\prime }}+b_t\hfill \end{array}$$

(14)

where $W_c\in {\mathbb{R}}^{n\times d_h}$, $W_c^{{}^{\prime }}\in {\mathbb{R}}^{d_h}$, $W_a\in {\mathbb{R}}^{m\times d_h}$, $W_a^{★}\in {\mathbb{R}}^{m\times n}$, $W_a^{{}^{\prime }}\in {\mathbb{R}}^{d_h}$, $W_t\in {\mathbb{R}}^{p\times d_h}$, $W_t^{★}\in {\mathbb{R}}^{p\times n}$, and $W_t^{{}^{\prime }}\in {\mathbb{R}}^{d_h}$ and $b_c\in {\mathbb{R}}^n$, $b_a\in {\mathbb{R}}^m$, and $b_t\in {\mathbb{R}}^p$, which are the trainable weight matrices and biases.

4.5. Subtask Consistency Constraint Module

Naturally, in a training dataset, a subtask consistency graph ${\mathcal{G}}_{co}$ consists of all possible labels $Y=\{Y^a,Y^c,Y^t\}$ as graph nodes and two labels from the subtask that co-occur in no less than ten cases as an edge. The matrix associated with its edges is constructed as follows:

$$M_{co}(i,j)=\left\{\begin{array}{cc}1\hfill & i<m<jorm<i<m+n<j<\left|L\right|\\ 0\hfill & otherwise\end{array}\right.$$

(15)

where $\left|L\right|=m+n+p$ is the number of all labels and $M_{co}\in {\mathbb{R}}^{\left|L\right|\times \left|L\right|}$. Next, similar to the procedure of constructing the charge similarity graph, we normalize $M_{co}$ to obtain $M_{co}^{*}=P_{co}^{-\frac{1}{2}}{M}_{co}{P}_{co}^{-\frac{1}{2}}$ based on its degree matrix $P_{co}$.

Since the truth label is represented by a one-hot vector, it cannot fully represent the corresponding text features. Especially in LJP, the label relationship of a subtask is complex (as shown in Appendix A.2) and there are noisy data. Inspired by Xu et al. [8], we convert hard labels represented by one-hot vectors to soft labels by adding a subtask consistency constraint. Unlike the graph-based LE algorithm [8], by adding case representation to $L^{\left(0\right)}$, we can achieve dynamic changes in the label distribution for different cases of the same label.

$$\begin{array}{cc}\hfill L^{(t+1)}& =\eta L^{\left(t\right)}{W}^{\left(t\right)}{M}_{co}^{*}+(1-\eta )L^{\left(0\right)}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill L^{\left(0\right)}& =Y+H{M}_{co}\hfill \end{array}$$

(17)

where $Y\in {\{0,1\}}^{\left|L\right|}$ is the hard truth label of the training dataset and $H=[H^a,H^c,H^t]\in {\mathbb{R}}^{\left|L\right|}$ is the case representation of the subtask. $\eta \in (0,1)$ is a balancing parameter that controls the fraction of the information inherited from the label propagation. $W^{\left(t\right)}\in {\mathbb{R}}^{\left|L\right|\times \left|L\right|}$ and $b^{\left(t\right)}\in {\mathbb{R}}^{\left|L\right|}$ are the trainable parameters and biases, respectively. Finally, $L^{(t+1)}$ will converge to L, which is then viewed as the soft target to supervise the model training.

4.6. Prediction and Training

Here, LJP has three subtasks. For ease of description, we use $i=1,2,3$ to represent them, respectively, e.g., $H^a,H^c,H^t$, which can be abbreviated to $H^i$. We input the case representation $H^i$ into the softmax classifier to obtain the prediction for each subtask.

$${\widehat{y}}^i=softmax\left(H^i\right)$$

(18)

Algorithm 2 Optimization Algorithm.

Require:   Training set $\Gamma ={(X_i,Y_i)}_{i=1}^N$.   Crime-keywords dictionary $E_D$.   Charge similarity graph ${\mathcal{G}}_s$.   Task consistency graph ${\mathcal{G}}_{co}$.   T = 10;   t = 0;   % pretrain;   while $t<T$ and not converged do     for each batch in $\Gamma$ do       Compute $\widehat{y}(x_i,\theta ,E_D,{\mathcal{G}}_s)$ using Equation (18);       $\theta$ = $argma{x}_{\theta }{\mathcal{L}}_p({\widehat{y}}_i,y_i)$, as in Equation (19);       $t++$;     end for   end while   % add subtask consistency constraint;   while $t<Epoch$ do     for each batch in $\Gamma$ do       Compute soft targets $L(x_i,\theta ,E_D,{\mathcal{G}}_s,{\mathcal{G}}_{co})$ using Equation (16) and Equation (17);       Compute ${\widehat{y}}_i(x_i,\theta ,E_D,{\mathcal{G}}_s)$ using Equation (18);       $\theta$ = $argma{x}_{\theta }{\mathcal{L}}_p({\widehat{y}}_i,l_i)$, as in Equation (21);       $t++$;     end for   end while

For training, the loss function is divided into two parts, where one is the prediction loss calculated on the one-hot representation Y and the other is the consistency loss calculated on the soft label L. Specifically, we calculate the cross-entropy loss for each subtask and take the sum of the losses for all subtasks as the corresponding loss.

$$\begin{array}{cc}\hfill {\mathcal{L}}_p& =-\sum _{i=1}^3\sum _{j=1}^{\left|Y^i\right|}{y}_j^{i}log\left(\widehat{y_j^i}\right)\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{con}& =-\tau \sum _{i=1}^3\sum _{j=1}^{\left|Y^i\right|}{l}_j^{i}log\left(\widehat{y_j^i}\right)\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill \mathcal{L}& ={\mathcal{L}}_p+{\mathcal{L}}_{con}\hfill \end{array}$$

(21)

where $|Y^i|$ denotes the number of labels for the various subtasks. $y_j^i\in {\{0,1\}}^{|Y^i|}$ is the one-hot representation of the truth label. $l_j^i\in {(0,1)}^{|Y^i|}$ is the soft label representation obtained in Section 4.5. $\tau$ is a predefined tunable parameter. The training process of the CMLEA is divided into two stages and the procedure is shown in Algorithm 2.

5. Experiments

In this section, we conduct experiments on two benchmark LJP datasets to show the effectiveness of our approach and present a detailed analysis.

5.1. Datasets

We conducted experiments on the publicly available dataset CAIL2018, which consists of CAIL-Small and CAIL-Big [38]. Various data preprocessing methods were adopted from existing studies, leading to varying results. To the best of our knowledge, Neurjudge [22] and R-former [7] are the latest LJP methods. To conduct the performance comparison, we followed the data preprocessing pipeline of Yue et al. [22] and ran the open source code of R-former on the same dataset. The corresponding data preprocessing process is as follows: (1) Exclude the case samples corresponding to law articles and charges with a frequency of no greater than 100. (2) Divide the terms of penalty into non-overlapping intervals. (3) Filter out complex samples with multiple applicable articles and charges. The detailed statistics of the datasets are shown in

Table 2. Statistics of datasets.

Dataset	CAIL-Small	CAIL-Big
#Training Set Cases	108,619	1,593,982
#Test Set Cases	26,120	185,721
#Validation Set Cases	13,738	-
#Law Articles	99	118
#Charges	115	129
#Term of Penalty	11	11

.

5.2. Baseline Methods

• SVM+word2vec: uses SVM [39] to classify the text represented by word2vec [36].
• FLA [18]: a neural network with attention mechanisms for capturing the interaction between the factual description and applicable articles.
• TopJudge [2]: a topological multi-task learning model that incorporates the DAG dependencies among multiple subtasks into LJP.
• Few-Shot [14]: an attribute-attentive charge prediction model that can predict few-shot charges and alleviate confusing charge issues.
• LADAN [21]: an attention-based model that employs a graph neural network to learn the distinction between confusing legal articles and further distinguishes between confusing charges.
• NeurJudge+ [22]: incorporates the semantics of established articles into facts to help divide the factual description into various subtasks for LJP.
• R-former [7]: utilizes a masked transformer network to obtain case-discriminative representations, and achieves local consistency of each node’s label distribution through relational learning.

5.3. Experimental Setup and Evaluation Metrics

For CNN- or RNN-based methods, we first used THULAC (https://github.com/thunlp/THULAC (accessed on 30 November 2021)) for word segmentation because the factual description is written in Chinese without spaces, and then utilized word2vec [36] to learn 200-dimensional vector representations of words. Meanwhile, we set the maximum document length to 350 and all hidden sizes to 150. For BERT-based methods, we used a pretrained Chinese model [32] and set the maximum document length to 512 tokens. For the proposed methods, when constructing the crime-keywords dictionary, we exploited the word embeddings (https://ai.tencent.com/ailab/nlp/en/download.html (accessed on 24 December 2021)) trained by Tencent AILab and the embedding size of each word was 200. We set the variable k to 20, threshold $\mu$ to 0.8, weight $\lambda$ and $\eta$ to 0.5, and $\tau$ to 0.01. During the training, we utilized the Adam optimizer [40] with a learning rate of ${10}^{-5}$. We utilized PyTorch [41] to implement the proposed model and train it on a server (the system environment is described in Appendix A.1). Moreover, we employed a mini-batch for training to reduce the memory overhead and trained each model with 16 epochs. For the R-former, we trained the pretrain $\theta$ with 10 epochs and E-step with 6 epochs.

Finally, we chose four metrics typically used in document classification tasks [42], i.e., Accuracy (ACC.), Macro Precision (MP), Macro Recall (MR), and Macro F1 (MF1), to evaluate the performance of the model. Since both CAIL-Small and CAIL-Big are imbalanced datasets, we mainly used MF1 for comparison with other methods.

5.4. Experimental Results

Comparison against baselines. We first compared the CMLEA with some baselines and the results are shown in

Table 3. Comparison of results on CAIL-Small. The best results for the main evaluation indicator are in bold.

Methods	Law Articles (%)				Charges (%)				Term of Penalty (%)
	ACC.	MP	MR	MF1	ACC.	MP	MR	MF1	ACC.	MP	MR	MF1
SVM+word2vec	84.17	80.74	75.96	77.09	83.37	80.78	77.30	78.25	33.00	25.56	25.11	22.50
FLA	85.63	83.46	73.83	74.92	84.72	83.71	73.75	75.04	35.04	33.91	27.14	24.79
TopJudge	87.28	85.81	76.25	78.24	86.48	84.23	78.39	80.15	38.43	35.67	32.15	31.31
Few-Shot	88.44	86.76	77.93	79.51	88.15	87.51	80.57	81.98	39.62	37.13	30.93	31.61
LADAN	88.78	85.15	79.45	80.97	88.28	86.36	80.54	82.11	38.13	34.04	31.22	30.20
NeurJudge+	90.37	87.22	85.82	86.13	89.92	87.76	86.75	86.96	41.65	40.44	37.20	37.27
R-former	92.55	89.99	88.18	88.62	92.87	91.07	90.92	90.88	42.94	41.15	38.97	38.82
CMLEA	93.19	89.96	89.37	89.58	93.40	91.59	91.99	91.71	43.45	42.23	40.11	40.73

and

Table 4. Comparison of results on CAIL-Big. The best results for the main evaluation indicator are in bold.

Methods	Law Articles (%)				Charges (%)				Term of Penalty (%)
	ACC.	MP	MR	MF1	ACC.	MP	MR	MF1	ACC.	MP	MR	MF1
SVM+word2vec	92.62	77.92	61.03	64.29	92.09	82.26	65.28	69.06	46.73	28.98	20.92	20.91
FLA	93.51	74.94	70.40	70.70	93.01	76.56	72.75	72.94	54.29	38.39	29.34	30.85
TopJudge	93.24	74.24	71.19	70.40	93.19	79.44	75.52	75.50	53.52	44.58	30.41	30.61
Few-Shot	93.74	78.51	73.79	74.18	93.24	80.59	76.62	76.89	54.54	39.09	33.36	33.48
LADAN	93.27	75.10	72.04	71.26	93.26	81.21	77.65	77.60	53.62	41.52	37.53	36.06
NeurJudge+	95.58	82.01	77.05	78.05	95.57	85.57	78.81	80.54	57.07	47.65	40.01	41.18
R-former	97.02	86.40	81.87	82.64	97.08	90.67	86.90	87.57	59.78	48.87	45.55	45.81
CMLEA	97.39	89.04	84.62	86.10	97.46	92.14	88.86	89.99	60.65	50.66	46.40	47.44

. We found that our method achieved the best results for all the metrics of the three subtasks. Specifically, we observed the following: (1) SVM+word2vec did not perform as well as the deep learning-based models. This may be due to its limitations when using static word vector features to represent text. (2) FLA and TopJudge both utilized the relationships among the subtasks. FLA achieved poor results because it is a pipeline model, which can easily lead to error propagation. Although TopJudge utilized the topological relationship among the subtasks, it performed better than FLA. However, because it did not utilize valuable label information, it was inferior to the other deep learning-based models. (3) The performance of Few-Shot and LADAN was roughly equivalent and enriched the label information from different perspectives. However, Few-Shot did not consider the constraints among the subtasks and LADAN treated the entire established article as label information, ignoring the gap between the established article and the factual description of the case. (4) The CMLEA outperformed the state-of-the-art methods NeurJudge+ and R-former. Specifically, on the average MF1-score of the three subtasks, the CMLEA improved by 3.89%/7.92% relative to NeurJudge+ and 1.23%/2.50% relative to R-former in CAIL-Small/Big, respectively. We believe that this is due to our incorporation of effective label information (i.e., crime keywords with a distillation operation) and the addition of a consistency constraint among the subtasks. (5) Furthermore, we made an interesting finding by comparing the results of the models on CAIL-Small and CAIL-Big. When predicting law articles and charges, the MF1-score of all methods on CAIL-Big was worse than on CAIL-Small, whereas the accuracy was better. This phenomenon contradicts the rule that the larger the data, the better the model performance. However, after comparing the label frequency of the two, we found that the labels of the law articles and charges in CAIL-Big were highly imbalanced relative to those in CAIL-Small (as shown in Figure A2 and Figure A3). In contrast, the frequency distribution of the term-of-penalty label was similar, and accordingly, the models performed better on CAIL-Big than on CAIL-Small.

Comparison against BERT-based methods. We compared the CMLEA with some methods based on the BERT encoder and its variants, as shown in

Table 5. Comparison of results on CAIL-Small (BERT-based methods). The best results for the main evaluation indicator are in bold.

Methods	Law Articles (%)		Charges (%)		Term of Penalty (%)
	ACC.	MF1	ACC.	MF1	ACC.	MF1
BERT	90.81	86.06	90.68	87.69	40.37	34.09
BERT-Crime	91.30	85.70	91.26	87.81	40.90	34.65
R-former	92.55	88.62	92.87	90.88	42.94	38.82
NeurJudge+	92.64	88.75	92.91	90.89	43.81	39.76
CMLEA	93.19	89.58	93.40	91.71	43.45	40.73

. We found that BERT and BERT-Crime [43] performed well on LJP tasks but performed worse than the CMLEA, which further validates the effectiveness of our proposed model. In addition, we found that the performance of NeurJudge+ with BERT as an encoder outperformed that with GRU as an encoder, which verifies that BERT can facilitate downstream tasks.

Ablation analysis of CMLEA. In addition, we analyzed the impact of the crime-keywords fusion module and consistency constraint module by comparing the CMLEA with its variants on CAIL-Small. First, we performed some degradation on the CMLEA to confirm the effectiveness of the individual modules (the performance is illustrated in Figure 3). Compared to the CMLEA, -kw removed the crime-keywords fusion module so the subsequent modules were performed directly on the factual description $h^d$. -con removed the subtask consistency constraint and loss ${\mathcal{L}}_{con}$ calculated on the soft label representations, which predicted the three subtasks based on $H^a,H^c,H^t$, respectively. We observed that both -kw and -con performed worse than the CMLEA, indicating that these two modules boosted the CMLEA to some extent. Furthermore, we replaced the subtask constraint of the CMLEA with the relational learning structure of R-former, denoted by -con+rl, which led to improvements compared to -con, but the performance was still slightly worse than the CMLEA.

5.5. Case Study

We performed a qualitative analysis of the CMLEA. The crime-keywords fusion module adopted by the CMLEA is divided into two parts, the distillation operation to optimize the keyword representations in the crime-keywords dictionary and the cross-attention module to ensure that the keywords interact with the factual description. In order to intuitively see the difference in the vectors after the above operation, we visualized these features in a two-dimensional space.

Specifically, we selected a number of crimes that are easy to confuse: “fraud”, “loan fraud”, “fraudulent loan, bill acceptance, financial documents”, “fundraising fraud”, “illegally absorbing public deposits”, etc. We used the heat map to show the changes in their correlations before and after the distillation operation, as shown in Figure 4. The image on the left is before the distillation operation and the one on the right is after the distillation operation. We found that after the operation, the correlation between each charge decreased and the distance became larger, which indicates that the procedure was effective.

Next, we visualized the factual description changes using t-SNE [44], as illustrated in Figure 5. The factual descriptions of the five crimes of “fraud”, “loan fraud”, “fraudulent loan, bill acceptance, financial documents”, “fundraising fraud”, and “illegally absorbing public deposits” are, respectively, denoted by different colors, i.e., red, dark blue, green, yellow, blue, etc. On the left-hand side is the word embedding of the factual description, which is easily confused. In the middle and on the right-hand side are the representations of these cases after CMLEA processing. We can see that dark blue (loan fraud) and yellow (fundraising fraud) can be distinguished, even though they are still close to other colors and the divisions are not clear. We can see from their corresponding keywords in the crime-keywords dictionary, as shown in Table A1, that many of their keywords are consistent. Furthermore, we know that both have a small number of training samples. We speculate that after we adopted the distillation operation, these factors led to the remaining keywords being insufficient to represent the charge label using the cross-attention mechanism.

In addition, we also analyzed the role of the consistency constraint module. Specifically, we chose two representative cases to compare the changes in the prediction results before and after adding this module, as shown in

Table 6. Comparison of prediction results before and after adding the consistency constraint. ✓indicates the correct label and ✗indicates the incorrect label.

Factual Description of the Case	Without Consistency Constraint	With Consistency Constraint
Defendant Chen invited more than 20 relatives and neighbors to place wreaths at the Boca Chemical Plant to obstruct production because of the death of her husband, an employee of the Chemical Plant, which seriously affected the normal production and operation of the Boca Chemical Plant.	290, ✓ Crime of sabotaging producti- on and operation, ✗ More than nine months in pri- son. ✓	290, ✓ Crime of gathering crowds to disturb social order, ✓ More than nine months in pri- son. ✓
Defendant Lin organized Fang to appeal to the State Letters and Calls Bureau, causing a large number of petitioners to gather for a long time on the sidewalk and bicycle lane opposite the reception desk of the State Letters and Calls Bureau, seriously disrupting social order.	290, ✓ Crime of picking quarrels and provoking trouble, ✗ More than one year in prison. ✗	290, ✓ Crime of gathering crowds to disturb social order, ✓ More than three years in pris- on. ✓

. In the absence of the consistency constraint, both examples in the table correctly predicted the relevant article but incorrectly predicted the corresponding crime name, which we expected, i.e., by adding the constraint module, the correctly predicted law articles will be associated with their corresponding crimes, thereby increasing the probability of predicting the proper charge and term of penalty, and vice versa.

6. Ethical Discussion

Although LJP has the potential to improve efficiency and fairness in criminal justice, it is also a sensitive area and some ethical issues inevitably arise [45].

The first dimension to consider is data ethics [46]. We used the public dataset CAIL2018 collected from an official government website (http://wenshu.court.gov.cn/ (accessed on 16 October 2018)). The private information (e.g., name, ID number, plate number, etc.) of all cases was anonymized before the case details were published on the website. In addition, the purpose of the government’s disclosure of judgment documents is to promote judicial justice and enhance judicial credibility. Correspondingly, models that can learn from existing cases can help to maintain fairness.

The second is the robustness of the model. Models inevitably have drawbacks. For example, models tend to be biased toward classes that have large amounts of data and are easy to learn due to imbalanced data distribution and varying levels of difficulty. As shown in Table 3 and Table 4, the ACC-score and MF1-score for the subtask of the term-of-penalty prediction were only 43.45%/60.65% and 40.73%/47.44% on CAIL-Small/Big, respectively. This indicates that the results achieved by these methods are not reliable, especially in the legal field, where a legal judgment mistake could have serious consequences, for instance, a person remaining in prison as a convicted criminal or being acquitted as a free man. Therefore, the predictions made by the models cannot be directly adopted.

In summary, we should note that the proposed method is not intended to replace offline criminal proceedings, nor is it intended to replace independent judgments of judicial officers. It is an aid for legal professionals, i.e., it helps judges to locate the applicable articles quickly and plays an important role in ensuring the principle of “treating like cases alike”. The predictions made by these models should be used as references, and the actual decision-making process still needs to be conducted by the professionals themselves. Therefore, utilizing non-human intelligence to determine the legal rights and obligations of human beings does not violate any ethical codes.

7. Conclusions

In this paper, we propose CMLEA, an approach for LJP based on cross-attention mechanism and label-enhancement algorithm. We construct a crime-keywords dictionary as the label information and design a novel cross-attention algorithm fused with a distillation operation on a charge similarity graph to help locate the key elements in factual descriptions and discover the case-related words in the dictionary. Furthermore, we use the label-enhancement algorithm to add a consistency constraint to subtasks to improve the prediction results. Our experimental results show that the CMLEA can achieve comparable performance with state-of-the-art LJP models. The average MF1-score of three subtasks exceeded two state-of-the-art methods by 3.89%/7.92% and 1.23%/2.50% on the CAIL-Small/Big dataset, respectively.

However, we observed that LJP performed poorly in the subtask of term-of-penalty prediction, with an MF1-score of only 40.73%/47.44%, whereas the MF1-scores of the other two subtasks were 89.58%/86.10% and 91.71%/89.99%, respectively. Meanwhile, we found that the CMLEA had some limitations in dealing with small sample sizes, that is, the cross-attention mechanism did not perform well in these cases and the frequency information of different labels was not taken into account. In addition, the interpretability of the prediction results was insufficient and needs improvement. In the future, we plan to explore (1) how to integrate digital-related information into established articles with factual descriptions to improve the performance of the term-of-penalty prediction; (2) techniques for optimizing the fusion of label information to address the issue of the poor performance of LJPs with small sample sizes; and (3) methods to improve the interpretability of LJP prediction results.