Using AI to Ensure Reliable Supply Chains: Legal Relation Extraction for Sustainable and Transparent Contract Automation

Abstract

Efficient contract management is essential for ensuring sustainable and reliable supply chains; yet, traditional methods remain manual, error-prone, and inefficient, leading to delays, financial risks, and compliance challenges. AI and blockchain technology offer a transformative alternative, enabling the establishment of automated, transparent, and self-executing smart contracts that enhance efficiency and sustainability. As part of AI-driven smart contract automation, we previously implemented contractual clause extraction using question answering (QA) and named entity recognition (NER). This paper presents the next step in the information extraction process, relation extraction (RE), which aims to identify relationships between key legal entities and convert them into structured business rules for smart contract execution. To address RE in legal contracts, we present a novel hierarchical transformer model that captures sentence- and document-level dependencies. It incorporates global and segment-based attention mechanisms to extract complex legal relationships spanning multiple sentences. Given the scarcity of publicly available contractual datasets, we also introduce the contractual relation extraction (ContRE) dataset, specifically curated to support relation extraction tasks in legal contracts, that we use to evaluate the proposed model. Together, these contributions enable the structured automation of legal rules from unstructured contract text, advancing the development of AI-powered smart contracts.

1. Introduction

Supply chains play a pivotal role in ensuring sustainability around the world. Supply chains are complex and dynamic ecosystems involving suppliers, manufacturers, distributors, retailers, logistics providers, and consumers. These entities engage in numerous transactions, each governed by contracts that outline terms, conditions, obligations, and expectations. From procurement agreements and service-level agreements (SLAs) to distribution and payment contracts, these legal documents play a vital role in ensuring smooth operations.

However, traditional contract management remains inefficient and error-prone. The manual handling of contracts, combined with verification delays and reliance on intermediaries, often results in disputes, financial losses, and operational inefficiencies, thus negatively impacting supply chain sustainability. Paper-based and traditional contracts further lack transparency and auditability, making them vulnerable to fraud and misinterpretations [1]. Figure 1 shows a simple representation of actors in the supply chain and the contracts regulating the relations between them.

The major challenges of contract management in supply chains are as follows:

• The high volume and complexity of contracts create administrative burdens, leading to delays, financial losses, and potential legal disputes.
• The centralized nature of traditional contract execution exposes organizations to risks of fraud, non-compliance, and inefficiencies.

These challenges necessitate an urgent transition toward automated and secure contract management solutions. The motivation behind this research is to address these inefficiencies by applying AI-driven automation to build more secure, scalable, and sustainable supply chains.

AI-Driven Automation in Contract Management

With the exponential growth of digital data and advancements in artificial intelligence (AI), there has been growing emphasis on automating contract analysis. Natural language processing (NLP), a subset of AI, has emerged as a powerful tool for extracting and analyzing structured information from unstructured legal texts.

This study contributes to our broader effort to extract information from paper-based contracts and use it to automatically generate corresponding smart contracts for the supply chain. These smart contracts can then be deployed in a blockchain environment to ensure efficient execution. We have already succeeded in extracting meaningful legal named entities, such as parties, effective date, governing law, etc., from contracts using an NLP task called named entity recognition (NER) [2]. This paper focuses on the next step in the information extraction process, relation extraction (RE), which identifies and retrieves meaningful relationships from these named entities. RE outputs structured data in the form of triplets, comprising two entities and their corresponding relation. This structured representation is essential for advanced NLP applications such as knowledge graph completion [3], question answering (QA) [4,5], and text summarization [6,7].

Over the years, extensive research has been conducted on entity and relation extraction, both in generic contexts [8,9] and in domain-specific scenarios [10,11,12]. Most of this research has focused on extracting relationships between two entities presented within individual sentences [13,14]. However, in legal contract texts, relation extraction is more complex, as entities and their relationships are often distributed across multiple sentences or clauses, requiring advanced contextual understanding and document-level reasoning [15,16]. Furthermore, a single sentence may contain more than two entities and multiple relationships, making sentence-based approaches inefficient.

Unlike other domains, such as the biomedical and chemical ones, where a lot of research has been conducted and there are many publicly available datasets for various NLP tasks, there is comparatively less research in the legal domain, especially in the case of contracts. The main reason for this is the confidential nature of this domain, which prohibits many researchers from publishing the datasets they have created [17]. Additionally, companies and law firms prefer not to share their clients’ contracts for research work, as these documents are often confidential.

These limitations highlight two critical research gaps: (1) the lack of contract-specific relation extraction models capable of capturing cross-clause dependencies and legal semantics; (2) the scarcity of publicly available, annotated datasets for contractual relation extraction. Addressing these gaps is essential for enabling AI-driven automation in legal contract analysis, particularly for applications such as smart contract generation and compliance monitoring.

To address this problem, we present ContRE, a dataset specifically designed for entity and relation extraction from legal contracts. For this task, we collected relevant contracts from the U.S. Securities and Exchange Commission (SEC)’s Electronic Data Gathering, Analysis, and Retrieval (EDGAR) system (https://www.sec.gov/edgar accessed on 6 May 2025) [18]. This system serves as a repository for publicly filed contracts that U.S. companies are required to submit and disclose.

In a previous study, we applied a QA-based method to extract relevant contractual clauses from lengthy documents, significantly reducing document size while preserving critical content [19]. These clauses were subsequently annotated with legal entities and relationships, forming the foundation for the NER [2] and RE phases in our contract automation pipeline. This marked the starting point for our current work, which focuses on designing a robust architecture capable of accurately modeling both intra- and inter-clause legal relations.

To effectively handle relation extraction in legal contracts, our proposed methodology employs a hierarchical transformer model designed to extract both global and segment-based contexts from legal texts. This architecture uses a two-layer hierarchical transformer that applies self-attention mechanisms to capture global document-level context while also utilizing local embeddings to focus on specific entities or segments of interest. By combining these contexts, the hierarchical transformer model integrates comprehensive document-level information with detailed local features, significantly enhancing the extraction of meaningful relationships and attributes from complex contractual texts.

Although attention-based models such as ATLOP [20] have been proposed for document-level relation extraction in general domains, they rely on flat transformer architectures and do not address the unique structural characteristics of legal contracts, such as clause-based organization and long-range inter-clause dependencies. This highlights the need for specialized models capable of handling the hierarchical and semantically rich nature of contractual texts. While hierarchical transformer architectures have been explored in tasks like document-level NER [21,22], their application to relation extraction (RE) remains limited.

To the best of our knowledge, no existing work has explicitly applied a clause-aware hierarchical transformer to the task of relation extraction in legal contracts. Our model addresses this gap by introducing a hierarchical attention mechanism that aligns with the structural organization and complexity of contractual documents.

With key entities and their relationships successfully extracted using our AI model, the next step is to use this information for secure and transparent contract management. Our ultimate goal is to enhance security, transparency, and efficiency in contract execution by transforming extracted contractual data into smart contracts within a blockchain-enabled supply chain. To achieve this, we utilize a rule-based approach to automatically generate executable business rules from the extracted contractual information. These business rules serve as the foundation for smart contract generation, enabling a self-executing, decentralized, and tamper-proof alternative to traditional paper-based agreements. The integration of smart contracts in supply chains ensures the following:

• Supply chain efficiency and sustainability;
• Real-time, automated contract enforcement;
• Enhanced transparency and security through blockchain immutability;
• The elimination of intermediary dependencies.

The overall idea and main focus of this paper are shown in Figure 2.

The remaining sections of this paper are organized as follows: Section 2 provides an in-depth analysis of existing studies and datasets related to relation extraction, offering valuable insights into the current state of research in the field; Section 3 provides a detailed description of our dataset, outlining its structure and characteristics; following this, our model architecture, the hyperparameter settings, and a comparative analysis of various models using the dataset are presented in Section 4; Section 5 analyzes the results obtained; Section 6 discusses the challenges encountered in NLP within the contract domain; and finally, Section 7 concludes the paper, providing insights for future research.

2. Related Studies in RE

The traditional RE methods were rule-based, with the extraction of relations from texts based on specific rules [6,23]. In RE, a set of patterns is defined for extracting predefined relations from unstructured text. Once the pattern is matched, the corresponding relations are extracted from the text. The disadvantage of this method is that it needs comprehensive knowledge of domains and linguistics for its implementation. Rules or patterns that work on one dataset may not perform well on another dataset. Therefore, to effectively perform RE in a new domain, we need to create a tailored set of rules and a dataset for the domain in question.

To overcome the disadvantages of a rule-based system, another approach called open information extraction (OpenIE) was introduced in [24]. It focuses on extracting entities and relations from unstructured text without relying on predefined relation schemas. OpenIE systems aim to discover a wide range of relations and entities, making them suitable for handling diverse domains. They achieve this goal through a combination of linguistic analysis, pattern-based extraction, and leveraging of syntactic and semantic structures in sentences. Since the establishment of OpenIE, many OpenIE-based RE systems have been developed, such as the Stanford OpenIE system [25], OpenIE4 [26], ArgOE [27], OpenIE5, and OpenIE6 [28].

Recent advancements in deep neural networks (DNNs) [15] and pre-trained language models (PLMs) [29] have revolutionized RE, elevating its performance to unmatched levels. Traditionally, RE methods fell into two categories: pipeline-based and joint approaches. Pipeline methods, though effective, suffer from error propagation due to separate entity recognition and relation extraction stages [30]. In contrast, joint approaches, aiming to capture interdependencies between entities and relations, perform both tasks simultaneously, though they are more complex [31]. Despite the advancements in RE, challenges persist, such as the scarcity of labeled data for long-tail relations. Distant supervision [15,32,33] and few-shot relation extraction techniques [34,35] have the potential to address these challenges, leveraging knowledge graphs and limited labeled samples, respectively.

While most RE research concentrates on extracting facts from single sentences, the demand for systems capable of analyzing longer documents with multiple sentences has spurred interest in cross-sentence RE. Recent studies have explored document-level RE [20,29,36], which aims to capture and analyze relational information across a document, representing a significant advancement over sentence-level approaches. The potential of document-level RE is especially relevant in scenarios where important relational data are dispersed throughout a document, necessitating a comprehensive analysis to fully and accurately capture and interpret these relationships. As such, document-level RE is pivotal for applications requiring a deep understanding of content within large texts, such as in legal document analysis, comprehensive news article processing, and in-depth academic research reviews.

Popular Datasets in RE

This section discusses the various RE datasets available in different domains. Many RE datasets belong to general domains created using corpora such as Wikipedia and Wikidata, while some belong to specific domains such as biomedicine [37,38]. Most datasets are created using the distant supervision method with the help of a vast knowledge base (KB).

Hendrickx et al. [30] introduced a widely used dataset for sentence-level relation extraction tasks. This dataset comprises various relation types annotated on sentences from newswire texts. It has been instrumental in advancing relation extraction research by providing a benchmark for evaluating different algorithms and models.

The ACE dataset introduced by Doddington et al. [39] is a seminal resource in the field of relation extraction. It focuses on the automatic content extraction (ACE) program and includes annotated relations across multiple domains such as newswire, broadcast news, and web data. Data sources encompass not only textual information but also audio and image data, spanning languages such as Arabic, Chinese, and English.

The New York Times (NYT) dataset, by Riedel et al. [40], was created through distant supervision, which involves leveraging existing knowledge bases to automatically label text from the NYT news using Freebase as the knowledge base. The process involves aligning entities mentioned in the text with entities in the KB and using these alignments to label sentences with relation mentions. This alignment process is based on heuristics, such as entity linking and relation extraction rules. It provides a valuable resource for relation extraction tasks, especially in scenarios where labeled text is scarce or unavailable.

The WebNLG dataset, introduced by Gardent et al. [41], stands as a cornerstone resource in natural language generation (NLG) research. It provides researchers with a carefully curated, structured dataset translated into human-readable natural language descriptions. By systematically converting RDF triples into coherent and informative text, WebNLG plays a vital role in advancing NLG. Zeng et al. [42] adopted this dataset for the RE task.

WIKI-TIME, by Yan et al. [43], emphasizes relation extraction with temporal reasoning and includes 57 relations across the Wikidata and Wikipedia corpora. It involves gathering data from a KB with temporal annotations of relations between entities, aligning entities with existing knowledge bases, annotating temporal information specifying the valid period of relations, and ensuring dataset quality through rigorous verification measures. Han et al. [34] present the FewRel dataset, using distant supervision with Wikipedia and Wikidata as a KB followed by human annotation to clean and filter the dataset by removing incomplete or falsely linked instances.

Hewlett et al., in 2016, introduced the WikiReading dataset [44], which offers 884 domains for large-scale language understanding tasks across Wikipedia articles. It serves as a comprehensive resource for training models to comprehend and extract information from encyclopedic text. DocRED by Yao et al. [15] is a large-scale human-annotated dataset for document-level relation extraction, created from sources like Wikipedia and Wikidata. Comprising 96 relations, this dataset focuses on extracting relations from entire documents, providing a more holistic view of the information present in textual sources.

ChemProt, by Peng et al. [45], comprises 14 relations and assesses transfer learning in biomedical natural language processing. This dataset focuses on the extraction of relations between chemical entities and proteins, crucial for various biomedical applications. DDI, introduced by Herrero-Zazo et al. [46], is dedicated to pharmacological substances and drug–drug interactions, containing five annotated relations. It plays a vital role in understanding the complexities of drug interactions and their implications in healthcare.

Rui Xing et al. [47] created the biomedical relation extraction dataset (BioRel) using a distant supervision process. The dataset draws upon the unified medical language system (UMLS) as a foundational KB and utilizes Medline, a vast repository of biomedical literature, as its primary corpus.

Jain et al. [48] introduced SciREX, which contains 21 relations and serves as a challenge dataset for document-level information extraction. It provides researchers with a platform to explore advanced techniques for extracting structured information from scientific documents. FOBIE, introduced by Kruiper et al. in 2020 [49], focuses on semi-open relation extraction from scientific texts and includes three relations. This dataset presents challenges in extracting relations from scientific literature, where information is often implicit or scattered across documents.

CUAD, by Hendrycks et al. [4], is a question answering dataset that contains 42 questions and answers and serves as an expert-annotated NLP dataset for legal contract review. It facilitates the development of NLP models tailored to legal domain applications, such as analyzing and understanding contracts.

The MNRF dataset by Zeng et al. [50] introduces an innovative approach to conducting RE from social media posts, particularly those lacking textual context, by integrating relevant images. Spanning over 10,000 sentences sourced from Twitter and encompassing 31 distinct relations, MNRF is annotated by crowdsourced workers. Each sentence is paired with a related image, providing a rich multimodal context for relation extraction.

Finally, FinRED, introduced by Sharma et al. [51], focuses on relation extraction tasks within the financial domain, containing 29 relations. This dataset, created using distant supervision, is tailored to applications in finance, such as extracting relations between companies, financial events, and market trends, aiding in financial analysis and decision-making processes.

Table 1. The classification of RE datasets.

Dataset	Relations	Category
SemEval 2010 task 8 [30]	9	General
ACE dataset [39]	24	General
NYT dataset [40]	24	General
WebNLG dataset [41]	246	General
WIKI-TIME [43]	52	General
FewRel [34]	100	General
WikiReading [44]	867	General
DocRED [15]	96	General
ChemProt [45]	5	Biomedical
DDI [46]	4	Biomedical
BioRel [47]	125	Biomedical
SciREX [48]	21	Scientific
FOBIE [49]	3	Scientific
CUAD [4]	41	Legal QA
MNRE [50]	31	Multimodal
FinRED [51]	29	Financial

summarizes the datasets discussed above.

However, these datasets are not suitable for our task, as our focus is exclusively on the legal contract domain, where contractual language is highly structured and domain-specific. Training a model on datasets from entirely different domains, such as social media and finance, would likely lead to reduced performance due to significant linguistic, structural, and contextual differences between general texts and legal contracts.

3. ContRE Dataset Preparation

Creating a dataset requires comprehensive study and an analysis of the task it will be used for, as well as a clear understanding of the desired final results. Thus, the primary aim of this study was to develop a dataset for document-level relation extraction from contractual texts. This dataset had to encompass crucial details, including mentions of named entities and the relationships among all entity pairs present in the documents.

The ContRE dataset was designed to capture relationships essential for automating legal processes, such as linking payment terms to due dates and associating notification periods with the terms of the contract. These extracted relationships are critical for automating the generation of smart contracts, which can streamline operations in blockchain-enabled supply chain systems.

We collected 500 contracts from the U.S. SEC’s EDGAR dataset as part of this study, building on our previous work [2], where we created NER datasets with 250 contracts. The dataset preparation process involved four key steps, namely, clause selection, named entity selection, the selection of possible relations among the entities, and annotation. In the following subsections, we explain each of these steps.

3.1. Clause Selection and Named Entity Annotation

Contracts vary widely in their complexity and length, depending on the specific requirements of the agreement they govern. Generally, a contract begins with an introductory segment called the preamble section that outlines the title, effective date, parties involved, and identities and addresses of the parties.

Following the preamble, the contract typically unfolds into several sections containing specific clauses. The first section is often called ‘Definitions’, ensuring that all terms used throughout the document are clearly and explicitly defined to avoid ambiguity. The subsequent sections are titled in a manner that reflects their content, facilitating the navigation through the document. For example, provisions concerning the duration and cessation of the contract can be found under ‘Term and Termination’, while clauses about the legal framework governing the contract are listed under ‘Governing Law’ or ‘Applicable Law’.

Given that contracts can span numerous pages, analyzing an entire document for RE demands substantial computational power. In addition, processing such extensive documents in their entirety can compromise the model’s efficiency [17]. When it comes to annotating a lengthy document for tasks like NER and RE, the dataset often becomes highly imbalanced. This imbalance arises because the number of named entity tokens is much lower compared to the number of other tokens in the document. As a result, the model’s performance can be adversely affected, leading to bias towards predicting tokens that are not part of any named entity, as they form the majority.

Our named entities and relation labels were present mainly in the preamble section of the contracts, ‘Term and Termination’ clauses, and ‘Governing Law’ clauses. Therefore, to deal with the imbalance in the dataset and to reduce the model complexity caused by considering the entire documents, we reduced the number of other tokens by only extracting the clauses that contained named entities and relationships. Some clauses, such as the preamble, were extracted manually using rules, and other clauses were extracted utilizing the Atticus CUAD QA [4] model.

The dataset was annotated with thirteen named entities, each vital for contract analysis. The details of the named entity labels are given in

Table 2. Selected named entity labels.

Named Entities	Meaning
Title	The title of the document or agreement
Party	The parties involved in the agreement
Alias	An alternative name or abbreviation for a party
Address	The physical address of a party
EffectiveDate	The date the agreement takes effect
Term	The duration or time period of the agreement
Renew	Terms for renewing the agreement
Notification	Deadlines or timeframes for giving notice
Govlaw	The governing law or jurisdiction of the agreement
Liqdamage	Liquidated damages for breach of contract
Payment	Payment terms or amounts specified in the agreement
Penalty	Penalty rates or conditions for non-compliance
Pay_Due	The timeframe for payment to be made

.

3.2. RE Label Selection

Relation labels serve to depict and characterize the relationship between two entities. By annotating the relationships among entities, we can extract contextual information from a text. In other words, understanding how different entities relate to each other can provide valuable insights into the meaning and structure of a text, facilitating more effective information extraction and analysis. Based on our requirement for the information to be extracted, we identified the following eight necessary relation labels for our task. The details of the selected relation labels are given in

Table 3. Selected relation labels.

Relation	Meaning
Contractor–Counterparty	Relationship between two parties in a contract
Party–Alias	Connection between a party and its alias
Party–Address	Links a party to its physical address
Effective–Term	Connects the effective date with its duration
Notification–Term	Relates notification deadlines to agreement terms
Payment–Pay_Due	Links a payment amount to its due date
Payment–Penalty	Associates a payment amount with penalties
Penalty–Pay_Due	Connects a penalty to the corresponding due date

, and Figure 3 shows an example of clause entities and relationships from the ContRE dataset.

The annotation tool used for annotating the named entity and relation labels was UBIAI (https://app.ubiai.tools, accessed on 6 May 2025), a platform designed to facilitate the annotation of datasets for NLP tasks. Figure 4 shows an example of annotation using UBIAI.

While ContRE represents a valuable resource for relation extraction in legal contracts, it is important to note that publicly available datasets in this domain remain extremely limited. Legal contracts are typically confidential, and organizations are often unable or unwilling to share them due to privacy, regulatory, and commercial constraints. This creates a significant barrier to building large, diverse, and openly available datasets for benchmarking and evaluating generalization performance. ContRE helps address this gap by offering a specialized dataset constructed from publicly accessible contracts filed with the U.S. SEC. In future work, we plan to explore transfer learning and domain adaptation approaches to assess how well models trained on ContRE generalize to other contract types and related legal documents, such as those included in the CUAD dataset.

4. Methodology

In this work, we introduce an improved document-level RE model that leverages a hierarchical transformer to identify relationships in complex contractual documents. The extracted relationships are then transformed into structured business rules, which serve as the foundation for automating smart contract creation. Unlike previous approaches that combine NER and RE into a single step, we treat them as separate tasks. This distinction is crucial, because contracts contain many entities, some of which may not participate in explicit relationships but are still essential for defining contractual terms. If NER and RE were performed jointly, entities without direct relationships might be overlooked, leading to incomplete information. Since every entity and its associated relationships contribute to contract automation, separating these tasks ensures that all relevant entities are retained, improving the completeness and accuracy of the extracted data.

Our process began with a set of preprocessed and annotated contractual clauses, where named entities and their mentions had already been identified, and our aim was to efficiently model the relationships between these entities. The input to our model consisted of these annotated clauses, ensuring that our dataset remained structured while preserving its semantic richness. The overall process flow is shown in Figure 5, illustrating the key stages of relation extraction and demonstrating the transformation from raw contractual text to structured relationship extraction using a hierarchical transformer model.

To enable accurate RE in contractual documents, we employed a hierarchical transformer model that integrates both local and global contextual information. Inspired by prior approaches such as ATLOP [20], which pools sentence-level context for document-level RE, we extended this idea by customizing hierarchical attention specifically to the clause-based structure of legal contracts. This design choice was motivated by the nature of contractual texts, where critical relationships often span multiple clauses or sections. Unlike standard flat transformers, our model separates intra-clause and inter-clause attention, which makes it particularly effective for capturing long-range dependencies in legal documents.

4.1. Entity Embedding Generation

As a preliminary step in RE, we leveraged pre-identified named entities from annotated contractual clauses. The model assumes that NER and clause segmentation have already been performed, ensuring that entity spans are correctly labeled.

Each entity is encoded using a BERT-based entity encoder, which extracts a rich set of linguistic features, transforming each entity mentioned into a dense vector representation. The entity embeddings encapsulate the following key aspects:

• Semantic meaning: Contracts contain a wide range of legal terminologies, obligations, and conditions, making it critical to capture the meaning of an entity within its specific context. For example, a date mentioned in a contract may be designated as the ‘EffectiveDate’ in one section, representing the start of contractual obligations, while in another section, it may be referenced as the ‘Termination Date,’ indicating the conclusion of the agreement. The BERT-based encoder ensures that a word’s semantic meaning is derived from its surrounding textual environment.
• Syntactic relevance: In legal texts, the grammatical role of an entity plays a crucial role in defining contractual relationships. The model identifies whether an entity functions as a subject, object, or modifier within a clause. For instance, in the phrase ‘Party A shall indemnify Party B’, the syntactic structure establishes that Party A is the obligor, while Party B is the beneficiary of the indemnification.
• Positional information: In long contractual documents, the position of an entity relative to other entities significantly affects relationship extraction. Entities appearing in the same clause often have a stronger direct relationship, while entities separated by multiple sections may be linked through cross-clause dependencies. The BERT-based encoder preserves positional embeddings, allowing the model to understand how proximity or distance impacts the strength of a relationship.

These entity embeddings serve as the input features for the hierarchical transformer model, which is responsible for relationship modeling across different contractual clauses.

4.2. Hierarchical Transformer Architecture

To effectively capture relationships at both the clause and the document levels, we propose a BERT-based hierarchical transformer model consisting of two layers:

• Local context layer—Processes relationships at the clause level, ensuring fine-grained entity interaction modeling.
• Global context layer—Expands contextual understanding to the document level, capturing cross-clause relationships.

4.2.1. Local Context Layer

The local context layer is designed to capture intra-clause relationships, ensuring that interactions within the same clause are accurately modeled. Since contractual obligations and legal relationships are often self-contained within clauses, this layer enables the precise extraction of close-proximity dependencies, which is crucial for improving relation extraction accuracy. Although contractual clauses often define relationships clearly, their legal language can be complex, and to ensure accurate extraction, the model must differentiate between key obligations and supporting details.

To achieve this, a self-attention mechanism is applied to tokens within each clause, allowing the model to dynamically assign importance to different words surrounding the entities. This mechanism ensures that the model prioritizes legally relevant terms while filtering out auxiliary words that do not contribute meaningfully to the relationship. By implementing self-attention, the model can effectively distinguish between key legal phrases and generic contractual language, thereby enhancing the precision of relation extraction. The ability to correctly weigh words within a clause is particularly important in contractual texts, where a single word can alter the legal interpretation of an obligation.

In addition to identifying key terms, the local context layer must also differentiate subtle variations in contractual relationships, as minor changes in phrasing can significantly impact legal interpretations. For instance, the relationship between ‘Payment’ and ‘Pay_Due’ specifies a due date for a payment obligation, whereas the relationship between ‘Penalty’ and ‘Pay_Due’ indicates a financial consequence for non-compliance. The local context layer ensures that these semantic nuances are captured accurately, preventing the misclassification of legal obligations and ensuring that the extracted relationships reflect the true intent of the contract. Without precise modeling at this stage, contractual terms may be misinterpreted, leading to incorrect relationship extractions in later processing stages.

Once the intra-clause dependencies are accurately captured, the localized entity representations are refined to maintain the integrity of the identified relationships. These refined representations are then passed to the global context layer, where broader inter-clause dependencies are modeled. Since contractual obligations often extend beyond a single clause, this next stage enables the model to integrate information across multiple sections of the document, ensuring that long-range legal relationships are properly recognized.

4.2.2. Global Context Layer

While intra-clause relationships are essential, many contractual relationships extend beyond a single clause and span multiple sections of a contract. The global context layer ensures that these cross-clause dependencies are captured accurately, allowing the model to recognize relationships between entities even when they appear in different sections of the document. Without this broader context, key contractual connections—such as payment terms, penalties, or notification requirements—might be misinterpreted or missed.

Unlike traditional transformer models that process text sequentially, our hierarchical approach applies self-attention across the entire document, enabling the model to link related entities across distant sections. This is particularly useful in contracts where key terms, such as the ‘Effective Term’, may be introduced in one section but further explained or modified in another part of the document. Similarly, a payment due date (‘Pay_Due’) may be stated separately from the penalty clause (‘Payment–Penalty’), making it essential to capture these connections across sections. By expanding attention beyond individual clauses, the model ensures that important relationships are not treated in isolation.

Contracts often specify conditions in one section that affect obligations elsewhere. For instance, a contract may define a ‘Notification Term’ stating how far in advance a party must inform the other about changes or termination. This information might be referenced later in a section outlining termination procedures or payment adjustments. If these relationships are not correctly linked, the interpretation of contractual obligations could be incomplete. The global context layer ensures that such cross-references are properly identified, allowing the model to extract contractual relationships as a cohesive whole rather than as fragmented details.

Additionally, the global context layer helps prevent inconsistent or inaccurate extractions when contractual obligations are spread across multiple clauses. For example, a contract may define payment terms (‘Payment–Pay_Due’) in one section while specifying penalty conditions (‘Penalty–Pay_Due’) elsewhere. Without considering the full context, the model might misinterpret these as separate obligations rather than interrelated terms. By integrating local and global contexts, the model ensures that contractually linked entities remain connected, enabling the comprehensive and structured extraction of legal relationships.

4.3. Relationship Classification

Once the hierarchical transformer layers generate refined entity representations, the model classifies relationships using a three-stage process:

• Embedding fusion: the outputs from both local and global layers are concatenated to form a single entity representation, which combines clause-level precision with document-wide relational context.
• Feedforward neural network (FFNN): The fused embeddings are passed through a fully connected neural network, which learns higher-level features for relationship classification. This network helps distinguish between similar relationship types (e.g., distinguishing ‘has obligation’ from ‘has liability’ in contractual text).
• SoftMax-based classification: The final layer applies a softmax function to predict the most probable relationship between each entity pair. This step ensures that the model assigns an appropriate label from a predefined set of legal relationship categories.

The overall architecture is presented in Figure 6.

Our model was compared with existing document-level RE models, including DocRED [15], which encompasses various architectures, such as the convolutional neural network (CNN)-based model [52], long short-term memory (LSTM)-based model [53], and bidirectional LSTM (BiLSTM)-based model [54]. Additionally, we compared our approach with ATLOP [20], whose implementations are built on Hugging Face’s Transformers framework [55] utilizing either cased BERT-base (https://huggingface.co/bert-base, accessed on 6 May 2025) [56] or RoBERTa (https://huggingface.co/roberta-base, accessed on 6 May 2025) [57]. In our experiments, we also tried a variation of the ATLOP model by using Contracts-BERT-base, a specialized version of BERT that is fine-tuned for legal text. These models served as benchmarks to highlight the performance and computational efficiency of our hierarchical transformer approach.

4.4. Hyperparameter Settings

In our implementation, we used Hugging Face’s Transformers framework [25] to implement the hierarchical transformer model, which incorporates two layers for capturing both local and global contexts. The first layer focuses on the local context around each entity, while the second layer captures dependencies across the entire document, enabling the model to understand relationships within complex legal texts.

Following this, we implemented several models utilizing Hugging Face’s Transformers framework, incorporating cased BERT-base [26], RoBERTa-large [27], and Contracts-BERT-base (https://huggingface.co/nlpaueb/legal-bert-base-uncased, accessed on 6 May 2025). Contracts-BERT-base is part of the Legal-BERT family [58], which comprises six domain-specific variants of BERT-base pre-trained on various types of legal texts. These models are useful for various NLP tasks within the legal domain. Each variant of Legal-BERT is trained on different types of legal data, with Contracts-BERT-base specifically pre-trained on U.S. contracts. The hyperparameter settings for the transformer models are reported in

Table 4. Hyperparameter settings.

Hyper Param	BERT	RoBERTa	Contracts-BERT-Base	Hierarchical
Batch size	8	8	8	16
NB Epochs	30	30	30	30
LR for encoder	3 × 10−5	3 × 10−5	1 × 10−4	3 × 10−5
LR for classifier	1 × 10−4	1 × 10−4	1 × 10−4	1 × 10−4
Optimizer	Adam	Adam	Adam	Adam

.

Our experimental setup involved two distinct systems, each designed to meet specific computational demands. Firstly, we utilized Google Colab Pro (https://colab.research.google.com, accessed on 6 May 2025), a cloud-based platform optimized for GPU-accelerated computing, powered by NVIDIA GPUs such as the Tesla A100 and V100. This environment facilitated expedited model training and inference phases, leveraging the GPUs’ robust parallel processing capabilities. The GPU-centric configuration effectively managed large-scale text datasets, enhancing model accuracy while ensuring computational efficiency. Additionally, our setup featured a GPU equipped with 40 GB of video memory, complemented by an Intel Xeon CPU with 12 cores and 84 GB of RAM, providing an adequate platform for our computational experiments.

Secondly, we used a Cisco C240 M5 server for evaluation. This type of server is equipped with 2 Xeon 6150 processors totaling 36 cores and 768 GB of RAM and also features a Titan RTX GPU with 24 GB of VRAM. This high-performance computing environment facilitated the efficient execution of machine learning algorithms and deep learning models, enabling a thorough evaluation and analysis of the proposed methodologies.

5. Results

Performance Metrics

In this section, we describe the performance of our proposed hierarchical transformer-based model and compare it with that of existing models on the ContRE dataset. We used the F1 and IgnF1 scores for evaluating the performance of the models in the RE task. The F1 score represents the harmonic mean of precision and recall, providing a comprehensive evaluation of the model’s ability to identify relevant relations and entities. In contrast, IgnF1 measures performance by excluding relations present in the annotated training set from the development and test sets.

The performance of the models on the ContRE dataset is summarized in

Table 5. The primary outcomes (%) on both the development and the test sets of ContRE.

Methods	Model	Dev		Test
		F1	Ign F1	F1	Ign F1
DocRED [15]	CNN	34.33	33.65	36.23	34.34
	BiLSTM	44.57	45.09	44.76	39.46
ATLOP [20]	BERT	61.24	58.65	60.87	59.67
	RoBERTa	60.35	59.00	61.63	60.21
ATLOP variation	Contracts-BERT-base	70.82	68.88	69.7	69.45
Our model	Hierarchical transformer-based	71.32	70.5	70.8	71

, showing both the F1 and the IgnF1 scores on the development and test sets.

We compared our model with several baselines, including models from the DocRED benchmark, which comprises traditional architectures (CNN and BiLSTM) as well as transformer-based models. Additionally, we included results from ATLOP [20], a state-of-the-art transformer-based document-level RE model, using both BERT and RoBERTa backbones. To ensure a fair comparison within the legal domain, we also implemented a variant of ATLOP using Contracts-BERT-base, a domain-specific model pre-trained on legal contracts.

We also evaluated a variation of the ATLOP model using Contracts-BERT-base, a model fine-tuned specifically on legal domain data. This variant achieved a strong performance, with an F1 score of 69.7% on the test set, outperforming the general BERT and RoBERTa variants. The Contracts-BERT-base model was a significant contributor to our work, as it uses domain-specific knowledge of legal language to improve relation extraction tasks.

The hierarchical transformer-based model builds upon these advancements by introducing a two-layer hierarchical structure designed to process both local and global contexts. The first layer focuses on understanding the immediate context of entities within a clause, while the second layer captures long-range dependencies across the document. This approach is essential for legal documents, where entities and relationships may span across multiple sections and clauses.

As shown in Table 5, our hierarchical transformer-based model achieved an F1 score of 71.32% on the development set and 70.8% on the test set (shown in bold), outperforming all other models, including Contracts-BERT-base (69.7%) and other transformer-based models. The model also achieved IgnF1 scores of 70.5% (development) and 71% (test), further validating its robustness.

Beyond numerical performance, we further examined the types of relationships extracted by the model to evaluate its ability to process complex contractual logic.

Table 6. An example of intra- and inter-clause relations extracted from a contract.

Clause Type	Text Extract
Preamble	This Service Agreement is entered into on 1 April 2024, by and between Metro and Horizon
Term	The initial term of this agreement shall last for 5 years from the effective date
Payment	Metro agrees to pay $600,000 per quarter, due within 30 days of the invoice date
Penalty	In the event of a late payment, a penalty of 2% per month shall be applied
Notification	Either party may terminate this agreement by providing written notice 60 days prior to the expiration
Relation Label	Entity 1	Entity 2	Relation Type
Effective–Term	1 April 2024	5 years	Inter-clause
Contractor–Counterparty	Metro	Horizon	Intra-clause
Payment–Pay_Due	$600,000 per quarter	30 days	Intra-clause
Payment–Penalty	$600,000 per quarter	2% per month	Inter-clause
Penalty–Pay_Due	2% per month	30 days	Inter-clause
Notification–Term	60 days	5 years	Inter-clause

presents a real example from our dataset, illustrating both intra- and inter-clause relations extracted from a contract. The model successfully identified cross-clause relationships such as ‘Payment–Penalty’ and ‘Notification–Term’, where the related entities are defined in separate sections. These examples highlight the model’s ability to handle real-world contractual logic involving dispersed yet semantically linked clauses.

The success of our model can be attributed to its ability to seamlessly combine both local and global contexts. It captures the subtle interactions between entities within individual clauses while also recognizing the long-range dependencies that connect different parts of the document. This dual focus makes our architecture ideal for complex tasks like contract analysis, where understanding entity relationships requires both a detailed view of each section and an overall understanding of the entire document. This level of precision and contextual understanding is particularly valuable for automating the generation of smart contracts in blockchain platforms. By accurately extracting multi-clause relationships and dependencies, the model supports the reliable and transparent execution of contractual terms, reducing manual intervention and improving operational efficiency in supply chain systems.

6. Smart Contract Generation

This step aims to automate the creation of smart contracts by converting traditional legal contracts (written in natural language) into computer code. This represents a significant challenge, because legal contracts are often complex, ambiguous, and full of conditional rules that must be precisely translated into executable code.

By using the NER and RE methods, we extracted the relevant entities and relations between those entities. The business rules were then created automatically by using this extracted information. The underlying principle is that business rules within contracts are often well defined and structured, which makes them suitable for automation. By using AI-driven solutions, these rules can be automatically extracted, interpreted, and smoothly integrated into supply chain information systems with a high degree of accuracy. This automation significantly reduces reliance on manual modifications, thereby minimizing the risk of human errors and inconsistencies.

Furthermore, by simplifying the process of rule implementation, organizations can achieve greater operational efficiency, ensure adherence to contractual obligations, and enhance overall productivity. Automating the integration of business rules into supply chain systems also facilitates real-time decision-making, improves data accuracy, and optimizes resource utilization. Ultimately, this approach contributes to a more flexible, transparent, and stronger supply chain ecosystem, benefiting all stakeholders involved.

The system employs a business rules representation framework, originally developed in Python3.11, to structure and define business rules. The process begins by inserting extracted entities into predefined conditions, such as execution dates, payment due dates, and other contractual parameters, along with their corresponding actions. Once these conditions and actions are identified, they are converted into structured rules in JSON format. These generated rules are then seamlessly integrated into relevant data models within the system, facilitating automated decision-making based on predefined business logic, thereby enhancing efficiency, accuracy, and consistency in rule execution. A concrete example of this process is demonstrated in the following use case, where we show how a payment clause from a real-world contract is transformed into a machine-readable business rule.

6.1. Illustrative Use Case: Automating Payment Compliance

To demonstrate the practical applicability of our approach, we present an illustrative use case derived from a distribution agreement between two companies: Stellar Distribution LLC and Nova Logistics Inc. The objective was to automate compliance with payment terms using our NER and RE pipeline, thereby reducing manual oversight and ensuring adherence to contractual obligations.

Clause: payment terms

’Nova agrees to pay Stellar $500,000 per quarter for distribution services, due within thirty (30) days of the invoice date. If payment is not received within the stipulated time frame, a penalty of $50 shall be applied to the outstanding balance.’

From this clause, our model extracted the following key named entities:

• Payment: $500,000 per quarter.
• Pay_Due: 30 days.
• Penalty: $50.

It also identified the following relations between these entities:

• Payment–Pay_Due: links the payment obligation to its due date.
• Payment–Penalty: connects the payment clause with the penalty clause.
• Penalty–Pay_Due: relates the penalty condition to the due date.

These extracted entities and their relationships were then passed to the rule generation module, which constructs machine-executable business logic. Figure 7 illustrates the resulting JSON-based smart contract rule, which enforces a penalty if payment is not received on time.

This rule can be seamlessly deployed within a blockchain-enabled supply chain system, enabling the autonomous execution of financial penalties and ensuring contractual compliance without manual intervention. This use case highlights how our approach effectively bridges unstructured legal language and structured smart contract rules, thereby aligning legal obligations with automated enforcement mechanisms.

In addition to payment compliance, legal contracts often contain a variety of clauses that reflect broader business goals and legal commitments. To show how our approach can handle these diverse scenarios,

Table 7. Mapping business objectives to contractual clauses and smart contract rules.

Business Objective	Clause	Business Rule
Support early contract termination	‘Either party may terminate this agreement with ninety (90) days’ written notice.’
Ensure contract renewal transparency	‘The agreement will automatically renew for successive four-year terms unless terminated with written notice 90 days prior to expiration.’
Define the governing law for dispute resolution	‘This agreement shall be governed by the laws of Texas.’

provides a few representative examples. Each row links a business objective to a specific clause in the contract, along with the corresponding business rule generated by our system, in a structured, machine-readable format.

6.2. Sustainability in Smart Contract-Enabled Supply Chains

The use of smart contracts in a blockchain-enabled supply chain contributes to sustainability in several ways:

• Reduction in paper-based contracts: smart contracts eliminate the need for physical documentation, reducing paper consumption and the associated carbon footprint from printing, storage, and disposal.
• Energy-efficient transactions: Traditional contract execution involves multiple intermediaries, lengthy approval chains, and manual verifications, all of which consume time and energy. Automated smart contracts streamline these processes, reducing wasted administrative resources.
• Minimization of legal disputes and waste: Contract disputes often lead to lengthy litigations, increasing resource use in terms of legal fees, paperwork, and human efforts. Blockchain’s immutability and real-time enforcement significantly reduce conflicts, ensuring fewer wasteful legal proceedings.
• Optimized supply chain processes: Traditional contract management in supply chains leads to delays, errors, and inefficiencies that result in wasted resources (e.g., fuel from unnecessary transport, storage inefficiencies, and production delays). Smart contracts enable real-time automated transactions, optimizing inventory management, reducing overproduction, and minimizing excess transportation emissions.
• Secure and fraud-resistant transactions: Fraudulent contract manipulations often lead to product recalls, incorrect shipments, and redundant logistics, all of which negatively impact the environment. Smart contracts ensure secure, tamper-proof transactions, reducing wastage caused by fraud or mismanagement.
• Decentralization and energy optimization: unlike traditional centralized contract processing that relies on multiple third parties, smart contracts reduce the need for intermediaries, leading to a more efficient, less resource-intensive system.

7. Discussion

From Table 5, it is clear that among the DocRED models, BiLSTM outperforms CNN, primarily due to its bidirectional learning process, which enables better contextual understanding. Furthermore, all transformer-based models, especially those based on BERT, demonstrate significant improvements over traditional architectures like CNN and BiLSTM. This can be attributed to the transformer architecture’s ability to leverage self-attention mechanisms and stacked encodings, providing a deeper, more flexible contextual representation.

Among these transformer models, our hierarchical transformer-based approach achieved the best performance. This improvement stems from its dual attention mechanism, which captures both intra- and inter-sentence relationships. Legal contracts frequently contain long-distance dependencies, where related entities appear in different parts of the document. Traditional models, which focus only on localized context or treat the document as a flat sequence, often fail to capture such connections. Our model addresses this issue by using a local context layer to extract fine-grained relationships within individual clauses and a global context layer to recognize broader relationships across multiple clauses. This hierarchical structure aligns naturally with the complex legal contract structure, allowing our model to more accurately capture the true semantics of legal relationships. As a result, it not only improves relation extraction performance, but also enhances the completeness and reliability of the extracted contractual information.

By excelling at both detailed and document-wide analysis, our hierarchical transformer model addresses many of the complexities inherent in legal contracts. However, despite these advancements, automating contract analysis using machine learning models remains a challenging task. Contracts often differ significantly from a general text in terms of their language, structure, and contextual dependencies. Below, we discuss some of the key challenges that need to be addressed for further improvements in this field and their implications for model development and performance.

• Contractual language complexity: Contracts often incorporate specialized language and terms that are not typical in everyday English or other general texts. This specialized language complexity poses challenges for ML models trained on general English datasets, requiring a deeper understanding and interpretation of the context.
• Using existing generic models poses challenges, especially in tasks like NER and RE: Predefined entities and relations, such as dates, may not be adequately captured by these models. For instance, in contracts, various types of dates, such as expiration dates and effective dates, serve distinct purposes. Generic NER models like SpaCy and NLTK struggle to differentiate between these specialized dates. Thus, training custom models is imperative to achieve optimal performance in such contexts.
• Dataset size and confidentiality: Another critical factor affecting the performance of machine learning models on contract datasets is the size of the dataset and the presence of confidential information. If a substantial portion of the dataset contains masked or confidential data, this can limit the data available for training and validation. This missing or obscured information can be either replaced with random values, which would potentially cause misrepresentation of the actual data, or ignored altogether, which would lead to incomplete or biased training.

Practical Relevance and Key Contributions

While the challenges discussed herein remain significant, our proposed approach offers both practical and research-driven contributions to the field of contract automation. As industries increasingly embrace automation, particularly in legal and supply chain contexts, there is a growing need to transform complex contractual texts into structured formats that machines can understand.

The key contributions of our study include the following:

• Enhanced legal information extraction: we propose a hierarchical transformer model designed to handle intra- and inter-clause relation extraction, addressing the complex structure of legal contracts.
• Specialized dataset creation (ContRE): We introduce ContRE, a dataset specifically designed for contractual relation extraction. We plan to make this dataset publicly available to support ongoing research in legal NLP.
• Support for smart contract transformation: our approach facilitates the conversion of traditional legal contracts into blockchain-based smart contracts that are transparent, tamper-resistant, error-reducing, and self-executing.
• Real-world supply chain relevance: by enabling automation in contract enforcement, our approach improves efficiency, traceability, and sustainability across supply chain processes.
• Broad applicability: this work has potential applications in legal tech startups, regulatory tools, and smart contract platforms, which highlights its academic and industrial significance.

These contributions underscore the scientific relevance and practical utility of our work and establish a foundation for future developments in automated legal contract analysis and execution.

8. Conclusions and Future Work

In modern supply chains, efficient contract management is critical to ensuring transparency, compliance, and sustainability. However, traditional contract management methods are often manual, error-prone, and inefficient, leading to financial risks, delays, and disputes. The adoption of AI-driven automation and blockchain-enabled smart contracts presents a transformative opportunity to address these challenges. By extracting key legal entities and their relationships, contracts can be converted into structured, self-executing agreements, enhancing operational efficiency, reducing intermediaries, and promoting sustainable supply chain practices.

As part of this broader effort to automate contract management and enable smart contract execution, this paper introduces a novel method for relation extraction from identified legal entities in contracts. Expanding on our prior work in NER for contract analysis, we propose a BERT-based hierarchical transformer model that captures both local and global contexts to extract complex relationships spanning multiple sections and sentences. This approach enables document-level and multi-sentence relation extraction, providing a more comprehensive understanding of contractual relationships. Thus, we effectively addressed the challenges posed by the specialized and structured language of legal contracts, significantly enhancing the accuracy and efficiency of relation extraction.

The extracted entities and relationships are then used to create structured business rules, forming the foundation for automated smart contract generation. This transformation ensures that contractual obligations and conditions are accurately represented in computer-readable formats, enabling seamless smart contract execution on blockchain platforms. Automating this process eliminates manual intervention, reduces errors, and enhances integration with supply chain information systems, leading to greater efficiency, transparency, and sustainability in contract management. By bridging the gap between traditional legal contracts and blockchain-based smart contracts, our approach paves the way for a more secure and autonomous contract execution framework within modern supply chains.

To support the RE task, we developed ContRE, a specialized dataset designed specifically for relation extraction from legal contracts. ContRE provides a tailored resource for evaluating document-level relation extraction, ensuring that our model can accurately interpret contractual relationships and legal structures. By combining our hierarchical transformer approach with ContRE, we contribute significantly to the advancement of legal text processing and contract automation.

Moving forward, we aim to expand ContRE, further refine our hierarchical model, and enhance business rule automation techniques to improve the integration of AI-driven contract intelligence with blockchain-based smart contracts. Through this continuous improvement, our research contributes to the long-term goal of fully automated, secure, and efficient contract management in supply chains.

Furthermore, to ensure real-world relevance, we are actively engaging with a prominent public-sector organization in Qatar, which has expressed strong interest in piloting our system for automating clauses in supply chain contracts. This prospective deployment reflects the practical viability of our approach and opens avenues for operational testing.

We also acknowledge that our dataset is currently limited to contracts filed with the U.S. SEC, which may reflect jurisdiction-specific structures and terminology. As legal frameworks vary across jurisdictions, generalizing the proposed approach to contracts governed by different legal systems (e.g., civil law or multilingual contracts) remains an important area of future research. We plan to explore domain adaptation, multilingual fine-tuning, and cross-jurisdictional benchmarking to assess the model robustness in diverse legal environments.