Intelligent Transaction Scheduling to Enhance Concurrency in High-Contention Workloads

Abstract

Concurrency control (CC) scheme based on transaction decomposition has significantly enhanced the concurrency performance of multicore in-memory databases, surpassing traditional CC schemes such as two-phase locking (2PL) or optimistic concurrency control (OCC), particularly in high-contention scenarios. However, this performance improvement introduces new challenges, as balancing transaction dependency constraints with enhanced concurrency optimization remains a persistent issue, especially with the increased number of concurrent client requests, which can lead to complex transaction dependencies. To address these challenges, we propose Dynamic Contention Scheduling (DCoS), a novel method that enhances transaction concurrency via a dual-granularity architecture. DCoS integrates a deep reinforcement learning (DRL)-based executor to schedule high-contention transactions while preserving dependency correctness. DCoS employs a one-shot execution model that enables fine-grained scheduling in high-contention scenarios, while retaining lightweight in-partition execution under low-contention conditions. The experimental results on both micro- and macro-benchmarks demonstrate that DCoS achieves a throughput up to three times higher than state-of-the-art CC protocols under high-contention workloads.

1. Introduction

With the exponential growth of multicore architectures in modern servers, optimizing concurrency control (CC) has become a critical challenge for online transaction processing (OLTP) systems. As the demand for higher throughput and lower latency intensifies, efficiently scaling transaction processing across cores is essential for latency-sensitive applications such as financial trading, e-commerce, and distributed databases. However, the performance of these protocols degrades rapidly on workloads with high contention [1], where hot data items are frequently accessed and modified by many concurrent transactions. Existing approaches to addressing high-contention workloads can be broadly classified into two main paradigms:

1. Partition-based execution. Partitioning techniques aim to colocate contention transactions within the same partition to minimize cross-partition communication and CC overhead. By assigning transactions to fixed threads or cores, partition-based strategies reduce inter-partition contention and facilitate execution with minimal coordination, making explicit CC mechanisms unnecessary in ideal deployment scenarios. However, they inherently limit intra-transaction concurrency and lack flexibility in the presence of dynamic or skewed workloads. Moreover, many real-world workloads are not naturally partitionable [2,3], which further constrains their applicability.

2. Fine-grained CC. Although hot data may constitute only a small portion of a transaction’s execution time, traditional CC schemes like two-phase locking (2PL) [4] and optimistic concurrency control (OCC) [5] operate at the transaction level, either holding locks or aborting entire transactions, leaving much of the potential concurrency unexploited. To overcome this problem, techniques such as transaction chopping [6,7], IC3 [8], and runtime pipelining [9] decompose transactions into smaller pieces or operations. These techniques enable the early exposure of intermediate results, but they require careful scheduling to prevent deadlocks and cyclic dependencies. For example, if one transaction accesses Table A before B and another accesses them in reverse order, IC3 merges these operations into a single execution unit to avoid cycles—sacrificing parallelism for safety. In real-world OLTP applications, transaction dependencies become increasingly complex due to high-frequency access to hot items and fluctuating client request patterns [10]. Existing methods face fundamental challenges in balancing transaction dependencies and concurrency, which urges the design of dynamic, dependency-aware scheduling frameworks.

Deep reinforcement learning (DRL) has shown great promise in solving complex combinatorial optimization problems such as graph routing and job scheduling [11,12,13]. However, its application to contention-aware scheduling in OLTP systems remains relatively unexplored.

In this paper, we propose Dynamic Contention Scheduling (DCoS), a DRL-based framework designed to enhance transaction concurrency by adaptively resolving both inter-transaction and intra-transaction contentions. Rather than developing a new high-performance concurrency control scheme, we improve the effectiveness of transaction processing through dependency-aware scheduling, with the capability to adapt to varying levels of contention. We implement a flexible contention identification mechanism based on hot data, enabling DCoS to dynamically switch between lightweight execution for low-contention workloads and DRL-based fine-grained scheduling for high-contention scenarios. Specifically, DCoS introduces a two-stage transaction processing architecture:

• Transactions are first dynamically partitioned. Transactions without cross-partition contention are executed independently and in parallel across cores.
• Transactions with potential cross-partition contentions are further decomposed into fine-grained operations. A DRL agent is then used to learn contention-aware execution orders, optimizing core utilization and minimizing aborts under dynamic workloads.

Our contributions are summarized as follows:

• We formulate complex-dependency transaction scheduling as a Markov Decision Process (MDP), enabling adaptive execution through learned scheduling policies.
• We propose an Adaptive Placement Graph structure to jointly encode intra-transaction and inter-transaction dependencies, as well as partition-to-core mappings.
• We develop a scalable, architecture-agnostic DRL executor with efficient feature embeddings, capable of generalizing across diverse concurrency scenarios and processing environments.
• We conduct an extensive experimental evaluation to demonstrate the effectiveness of the proposed approach, showing consistent improvements over both existing transaction-scheduling strategies and state-of-the-art CC methods without scheduling.

2. Related Work

2.1. Transaction Partitioning

Transaction partitioning has been introduced to address scalability and performance bottlenecks in multicore database systems, particularly under high-concurrency workloads. By dividing transactions based on data access patterns, partitioning minimizes inter-core synchronization and coordination overhead, thereby improving throughput and simplifying CC within the system. Static transaction-partitioning methods include range-based partitioning, which divides data based on value ranges (e.g., timestamps or IDs), and hash-based partitioning, which distributes data evenly across cores using hash functions.

As data distributions and access frequencies evolve, dynamic partitioning adjusts real-time data allocation to balance load and optimize performance. Horticulture [2] develops an analytical cost model that allows for the rapid estimation of relative coordination costs and skews associated with a specific workload and candidate database design. Horticulture facilitates an informed exploration of the vast solution space through large neighborhood search techniques. Schism [3] partitions the co-access graph among data items and uses these partitions to infer the static partition predicates. Clay [14] is an elasticity algorithm that balances load and minimizes distributed transactions simultaneously, by creating dynamic blocks on-the-fly while monitoring the workload. Prasaad et al. [15] propose a transaction-processing protocol for high-contention workloads, which identifies hot records in the database during runtime, clusters transactions that access them together, and executes them without CC protocol. The effectiveness of transaction partitioning is highly dependent on workload distribution. Imbalanced partitioning schemes can lead to over-provisioned partitions with unnecessary resource allocation, or suffer from load imbalances, resulting in underutilized compute resources.

2.2. Concurrency Control

Conventional CC approaches enforce serializability through strict access ordering mechanisms:

• 2PL [4]: Enforces exclusive write locks until a transaction completes. Conflicting reads must wait for locks to be released, leading to sequential bottlenecks.
• OCC [5]: Delays the visibility of writes until a validation phase, resulting in write-set propagation delays for dependent transactions.
• Multi-Version CC (MVCC) [16]: Utilizes timestamp-based version control, restricting reads to committed versions available before the transaction start time.

2.2.1. Transaction Decomposition

For two transactions with inter-transaction dependencies, all three above approaches require one transaction critical path to wait for the other to complete. To address these limitations, transaction decomposition-based methods have been proposed [6,7,8,9]. Transaction decomposition minimizes contention and reduces the duration of locks on commonly accessed data items by analyzing programs to divide a transaction into independent sub-transactions, which enables controlled access to uncommitted writes through dependency tracking. IC3 [8] integrates a static analysis of transaction workloads with runtime techniques that manage and enforce dependencies among concurrent transactions, ensuring serializability while permitting parallel execution of certain conflicting transactions. Callas [9] partitions a transaction into groups, then employs different CC protocols for groups to execute independently.

2.2.2. Transaction Dependency Management

Mitigating transaction contention and optimizing CC synchronization overhead are critical challenges in high-performance transaction processing systems. Effective management of inter-transaction dependency and intra-transaction dependency can significantly improve system throughput and reduce latency.

• Inter-transaction dependency: Calvin [17] maintains dependency information between transaction groups, ensuring that dependent transactions only proceed after the completion of prerequisite transactions. Ding et al. [18] propose a thread-aware policy to reorder transactions efficiently in a multi-threaded OLTP architecture based on OCC. AlNiCo [19] utilizes SmartNICs to route transaction requests to CPU cores, reducing contention with low latency intelligently.
• Intra-transaction dependency: Polyjuice [18] leverages RL model to explore a policy space of fine-grained actions for individual operations. Bamboo [20] introduces a CC protocol that extends traditional 2PL by dynamically assigning timestamps, allowing locks on hot data items to be retired early while preserving serializability. Chiller [21] modifies lock order to reduce lock hold times for hot records.

2.3. Transaction Assignment

Most transaction-processing systems employ a thread-to-transaction assignment strategy, limiting their ability to utilize intra-transaction parallelism effectively. To address this limitation, several studies have explored techniques to harness such parallelism [8,22,23,24,25]. QueCC [25] adopts a queue-to-thread assignment strategy, dynamically parallelizing operations through two phases of priority-based planning. Similarly, LADS [24] separates contention resolution and transaction execution into two stages, enabling the decentralized and dynamic partitioning of dependency graphs into load-balanced subgraphs at the record level. Traditional thread-to-queue assignment methods struggle to adapt to fluctuating transaction priorities and varying access patterns. When high-priority transactions encounter unresolved contention, static queue-based architectures experience prolonged tail latency, leading to system inefficiencies. DCoS proposes a novel execution model which adapts to dynamic workloads to reduce the latency of transaction processing.

3. Preliminaries

In this section, we establish the formal foundations of transaction processing and then introduce key concepts for reinforcement learning.

3.1. Formalism of Transaction Processing

Definition 1

(Transaction). A transaction $T_i$ constitutes a finite sequence of database operations $\{O_{i1},\dots ,O_{in}\}$ executing as an atomic unit. Each operation $O_{ij}$ represents either a $READ\left(d\right)$ or $WRITE\left(d\right)$ action on data item $d\in \mathcal{D}$.

Operations within the same transaction can exhibit two types of dependencies, which arise based on the transaction’s logical structure. We categorize these as intra-transaction dependency, which are further divided into (1) data dependency and (2) commit dependency.

Definition 2

(Intra-Transaction Dependency). For operations $O_{ij}$ and $O_{ik}$ within transaction $T_i$, the partial order ${\prec }_{\mathrm{intra}}$ satisfies:

$$\forall O_{ij},O_{ik}\in T_i,(j<k)\Rightarrow \left(O_{ij}{\prec }_{\mathrm{intra}}{O}_{ik}\right)$$

(1)

• Intra-transaction data dependency exists when $O_{ik}$ relies on data computed by $O_{ij}$. Formally, $O_{ik}$ is said to be data-dependent on $O_{ij}$ if it requires a value produced by $O_{ij}$.
• Intra-transaction commit dependency ensures transaction atomicity when certain operations may be subject to logic-induced aborts. Logic-induced aborts result from violating integrity constraints defined by applications, which are captured by the set of constraints ${\prec }_{\mathrm{intra}}$. Intuitively, if an operation is associated with at least one constraint that may not be satisfied after the execution of the operation, then it is abortable.

Another type of dependency exists between operations of different transactions, which the execution model induces.

Definition 3

(Inter-Transaction Dependency). For distinct transactions $T_i\ne T_k$, a dependency $O_{ij}{\prec }_{\mathrm{dep}}{O}_{kl}$ exists iff:

$$\exists d\in \mathcal{D}:(O_{ij}=\mathrm{WRITE}\left(d\right))\wedge (O_{kl}\in \{\mathrm{READ}\left(d\right),\mathrm{WRITE}\left(d\right)\})$$

(2)

This general dependency can be classified into two specific cases:

• Write–read dependency: If $O_{kl}=\mathrm{READ}\left(d\right)$, then $O_{ij}$’s commit must precede $O_{kl}$’s execution to prevent cascading aborts.
• Write–write dependency: If $O_{kl}=\mathrm{WRITE}\left(d\right)$, then $O_{ij}$ and $O_{kl}$ require mutual exclusion on d, meaning they must be serialized to avoid data inconsistency. Unlike write–read dependency, this type of contention does not enforce a strict execution order.

Transaction contention. Transaction contention primarily arises from inter-transaction dependencies, where multiple transactions contend for shared data. A transaction $T_i$ is considered conflict-free with another transaction $T_j$ if there are no contentions between them. The nature of these contentions is determined by the isolation level enforced by the CC protocol. For example:

• Under serializability, contention occurs when two transactions access the same data item, and at least one modifies it.
• Under snapshot isolation, contention occurs if two transactions attempt to write the same data item.

Effectively managing transaction contentions is essential for maintaining database integrity, ensuring correctness, and optimizing performance in a concurrent environment.

3.2. Reinforcement Learning

Reinforcement learning is a practical approach for adaptive decision making in dynamic environments. In this framework, an agent iteratively refines its actions based on feedback from the system state. This learning paradigm is typically modeled as a Markov Decision Process, represented by the tuple $(\mathcal{S},\mathcal{A},\mathcal{P},\mathcal{R},{\mathcal{S}}^{\prime })$.

The agent interacts with the environment at discrete time steps. At each step t, the agent observes a state $s^t$ from the state space $\mathcal{S}$ and selects an action $a^t$ from the action space $\mathcal{A}$. After executing the action, the environment provides a reward $r^t$, and the agent transitions to a new state $s^{t+1}$ according to the transition probability distribution $\mathcal{P}$.

The process of selecting actions is guided by a policy $\pi \left(a\right|s)$, which specifies the probability of choosing an action a in a given state s. The objective of the learning-based optimization process is to derive an optimal policy $\pi$ that maximizes the expected cumulative reward:

$$R\left(\pi \right)={\mathbb{E}}_{\pi }\left[\sum _t{\gamma }^t{r}_t\right]$$

(3)

where $\gamma \in [0,1]$ is the discount factor that controls the importance of future rewards.

4. Design of DCoS

DCoS adopts a one-shot transaction-processing model, in which a transaction begins execution only after all its inputs are ready. Many modern in-memory systems employ similar models, including H-Store [26], Calvin [17], and Silo [27], which leverage pre-declared access patterns to enable efficient scheduling and deterministic execution in some cases. DCoS adopts two coordinated phases to optimize transaction processing. Figure 1 shows the workflow of DCoS.

• (Section 4.1) Conflict-free partitioning phase: Transactions are partitioned based on high-contention data items.
• (Section 4.2) Dynamic execution phase: The CC granularity of the transactions is determined dynamically according to data access patterns.

4.1. Conflict-Free Partitioning Phase

Given a transaction batch $\mathcal{B}$, DCoS applies a partitioning function $\Phi$ to minimize concurrency control to divide it into:

• p conflict-free clusters $\{Q_1,Q_2,\dots ,Q_p\}$;
• A residual set $\mathcal{T}$, where $\left|\mathcal{T}\right|\le \alpha \left|\mathcal{B}\right|$, and $\alpha \in [0,1]$ serves as the residual threshold parameter.

The transaction-partitioning problem is generally NP-hard, and extensive research has been conducted in this domain [3,15,24]. We adopt the dynamic partitioning algorithm following [15], and DCoS is also compatible with other partitioning functions.

DCoS follows a proactive framework that leverages the knowledge of the complete database scheme and the complete set of stored procedures in OLTP applications. DCoS analyzes the contention among transactions based on their read–write sets by creating the data access graph from the read–write set of transactions. Specifically, the partitioning phase consists of three key processes:

1. Identifying hot data items. Hot data items $\{d_1,\dots ,d_{\delta }\}$ are probabilistically detected by randomly sampling transactions for $\eta$ trials. The data items accessed by these sampled transactions are merged into a designated hot cluster, ensuring that only $\eta$ transactions are processed and $\delta$ hot clusters are formed in this step.

2. Expanding the hot cluster with cold items. For the remaining transactions, if at most one of their accessed data items belongs to an existing hot cluster, the data set of the entire transaction is merged into that cluster. However, if a transaction accesses data items spanning multiple hot clusters, it is skipped to prevent cross-cluster conflicts.

3. Optimizing the volume of the residual set. The system continuously monitors the residual growth ratio ${\displaystyle \frac{\left|\mathcal{T}\right|}{\left|\mathcal{B}\right|}}$. If the residual volume exceeds the threshold $\alpha \left|\mathcal{B}\right|$, a hot cluster merging process is triggered to balance the trade-off between reducing CC synchronization overhead and maintaining thread concurrency efficiency.

4.2. Dynamic Execution Phase

4.2.1. Execution Model

Existing solutions [15,28] employ a two-phase execution method, as shown in Figure 2b. In the first phase, conflict-free clusters are executed serially without requiring any CC protocol on m available cores $\mathcal{C}=\{C_1,\dots ,C_m\}$. We consider the situation in which each core is assigned to a single thread and use thread worker and core assignment interchangeably in the following paper. In the second phase, residual transactions involving cross-partition data items are executed using conventional CC protocols across all cores. However, this approach has inherent limitations: transactions in the residual set limit the entire transaction processing latency due to cross-partition dependencies (see Section 4.2.2 for details). The system can manage conflicts more effectively and improve concurrency control by decomposing transactions into smaller pieces based on their contention data set.

DCoS solution. Given the varying workloads and processing times of conflict-free clusters, particularly for residual transactions, DCoS introduces a fine-grained scheduling approach for cross-partition transactions. Firstly, transactions accessing multiple high-contention data items (i.e., cross-partition transactions) are decomposed into transactions pieces leveraging runtime pipelining (RP) [9]. RP organizes these pieces based on data dependencies and assigns each group a rank. Depends on whether access hot data item in different partition, the transaction pieces can be further divided into contention-intensive transaction pieces and non-contentious transaction pieces. Among the same group, existing RP-based methods mainly merge these accesses into a single execution unit to avoid deadlocks, which limits the level of parallelism that can be potentially exploited. We employ fine-grained scheduling with a DRL-based executor to explore more concurrency among the same group (see Section 4.2.3 for details).

Then DCoS employs an adaptive two-level queue execution model as shown in Figure 2a.

• Hot queue: Designed to handle contention-intensive transaction pieces requiring fine-grained scheduling at the operation level. A DRL-based executor is employed to maximize concurrency within these contention-intensive transaction pieces.
• Cold queue: Designed to handle transactions (or transaction pieces) that do not involve cross-partition data. Utilize a coarse-grained scheduling approach (e.g., conventional methods such as round-robin allocation), where entire transactions (pieces) are assigned to threads.

The adaptive two-level queue execution model adheres to the following scheduling principles:

• Principle 1: Concurrent execution and scheduling. DCoS executes cold-queue transactions with hot-queue transactions scheduling concurrently. The fine-grained scheduling process dynamically adapts to execution progress according to thread state.
• Principle 2: Priority-based scheduling. Transactions in the hot queue are assigned higher execution priority, while cold-queue transactions are assigned lower priority. If no hot transactions can proceed due to dependency constraints, the thread first fetches from the cold queue. Instead of forcing hot-queue transactions to wait idly, the system dynamically schedules transactions in the cold queue to execute first.
• Principle 3: Dependent enforcing. Hot-queue transactions are often dependent on cold-queue transactions. Since cold-queue transactions access low-contention data but may still hold locks on critical data items, hot-queue transactions that require those items must wait until the locks are released. In addition, execution dependencies between transaction pieces within the same transaction are enforced via the lock mechanism defined in the RP framework [9]. Specifically, a later transaction piece cannot acquire the locks until all earlier pieces have released theirs.

Compared with the two-phase execution model, the two-level queue execution model dynamically adjusts execution order based on dependencies and thread status with the following objectives:

• Reduced lock contention: Hot transactions experience fewer conflicts through fine-grained scheduling to reduce the probability of aborts.
• Higher resource utilization: Cores remain active by executing transactions in the cold queue when hot transactions wait for a lock.

4.2.2. Contention Transaction Patterns

Before introducing the fine-grained scheduling design, we first analyze the fundamental dependency patterns within the residual set $\mathcal{T}$ using two representative examples, as illustrated in Figure 3. $W(·)$ denotes a write operation, and $R(·)$ represents a read operation.

Example 1

(Direct dependency). Figure 3a illustrates a direct dependency scenario in which transactions $T_1$, $T_2$, and $T_3$ access shared data items $G_2$, $G_3$, and $G_4$ with overlapping write operations. To ensure serializability under such contention, transaction execution can be coordinated by assigning different start times based on dependency order. For instance, by applying proactive delayed data access [28], $T_3$’s write to $G_2$ can be scheduled first, followed by $T_2$, and finally $T_1$, respecting a write sequence to avoid conflicts.

Example 2

(Transitive dependency). In real-world workloads, more complex dependency chains frequently arise, as shown in Figure 3b. Here, transitive dependencies exist among $T_1$, $T_2$, and $T_3$. Unlike direct dependencies, simply adjusting the execution order of transactions cannot resolve the scheduling problem. An inappropriate sequence may still result in transaction aborts, which imposes a more sophisticated scheduling solution.

4.2.3. Fine-Grained Scheduling

To effectively schedule contention-intensive transactions, we design a DRL-based executor to determine the access sequence on hot data items (i.e., operation sequence) and their corresponding thread assignment, as shown in Figure 4. The scheduling process satisfies the following constraints:

• Minimizing cross-partition coordination: Scheduling operations while respecting inter-transaction and intra-transaction dependency constraints to reduce synchronization overhead.
• Optimizing core utilization: Ensuring proper core assignment to prevent resource underutilization and avoid excessive load concentration on specific cores.

Problem formulation. Given a residual transaction set $\mathcal{T}=\{T_1,\dots ,T_n\}$ consisting of variable-length operation sequences ($|T_i|\ne |T_j|$) over a data subset $\overline{\mathcal{D}}\subseteq \mathcal{D}$, the executor schedules these transactions across m available processing threads, $\mathcal{C}=\{C_1,\dots ,C_m\}$. We assume $n\le m$, as DCoS processes residual transactions concurrently within a single period across available cores. If residual transactions exceed the available cores ($n>m$), transactions are grouped and scheduled across multiple periods.

Residual transaction processing follows the same mapping function as conflict-free transaction processing to ensure the serial execution of operations on the same data. Let $\Phi :\mathcal{T}\to \mathcal{Q}=\{Q_1,\dots ,Q_p\}$ represent the partitioning function that assigns transactions to p disjoint subsets. Additionally, a one-to-many mapping $\Gamma :\mathcal{Q}\to \mathcal{C}$ allocates these partitions to thread groups. Consequently, the mapping of operations to threads is formally defined as follows:

$$\mathcal{M}=\Phi (\Gamma ):O_{ij}\to \overline{\mathcal{C}}\subseteq \mathcal{C}$$

(4)

where $O_{ij}$ represents an operation from transaction $T_i$.

The makespan refers to the maximum completion time across all threads, i.e., the time by which the last thread completes its assigned operations. The objective of the scheduler is to minimize the total makespan required to process $\mathcal{T}$. Specifically, the overall makespan is given by:

$$M_{max}=max\{M_i\mid i\in \{1,\dots ,m\}\}$$

(5)

where $M_i$ denotes the makespan of the i-th thread. By optimizing the operations sequence and operations to threads, the scheduler aims to achieve balanced workload distribution while reducing contention and synchronization overhead.

5. Drl-Based Executor

DCoS implements a DRL agent to act as the executor and solve the scheduling problem above. We propose the Adaptive Placement Graph (APG) as the input to the scheduling process, providing a structured way to represent both intra-transaction and inter-transaction dependencies.

Graph structure. The APG is defined as:

$$G=(\mathcal{T}=\mathcal{T}\cup \{{\mathcal{O}}_{start},{\mathcal{O}}_{end}\},\omega ,\phi )$$

(6)

where ${\mathcal{O}}_{start}$ and ${\mathcal{O}}_{end}$ are the dummy starting and ending of scheduling sequence. The set $\omega$ consists of intra-transaction dependency edges, capturing the execution order of operations within a single transaction. The set $\phi$ represents resource edges, which link pairs of operations requiring the same thread worker for execution, as dictated by $\mathcal{M}\left(O_{ij}\right)$. Resource edges consist of a mix of directed and undirected arcs:

• Undirected arcs: Represent intra-transaction dependency, indicating that two operations contend for the same resource without enforcing a strict execution order.
• Directed arcs: Represent write–read dependency, ensuring that a write operation must be committed before a dependent read operation is executed.

Figure 5 illustrates how DCoS schedules contention-intensive transactions in Example 2. By updating the resource edges (from undirected to directed), DCoS can generate a scheduling solution that optimizes the processing of transactions across available threads, ensuring minimal makespan while respecting the dependencies between transactions.

Moreover, DCoS leverages historical workload traces containing transaction-processing metadata (e.g., procedure parameters, execution timestamps, query parameters). Using this data, DCoS estimates per-thread operation processing times set $\epsilon =\left\{{\rho }_{ijk}\right\}(O_{ij}\in \mathcal{T},C_k\in \mathcal{C})$ based on current thread-worker state $S{\left(C\right)}^t=\{S{\left(C_1\right)}^t,\dots ,S{\left(C_m\right)}^t\}$.

5.1. MDP Formulation

The scheduling process comprises $\left|\mathcal{T}\right|$ consecutive decision steps. We formalize the MDP defined by a tuple $(\mathcal{S},\mathcal{A},\mathcal{P},\mathcal{R},{\mathcal{S}}^{\prime })$ as follows:

States $\mathcal{S}$. The global state $s^t$ at timestep t consists of a APG state $s_{sc}^t=\langle S{\left(\mathcal{O}\right)}^t,{\kappa }^t,{\xi }^t\rangle$ and a thread-worker state $s_c^t=(S{\left(C\right)}^t,\epsilon )$:

• APG State: ${\kappa }^t$ and ${\xi }^t$ denote scheduled/unscheduled set of resource edges. For each $O_{ij}$, $S{\left(O_{ij}\right)}^t=\{T{\left(O_{ij}\right)}^t,X{\left(O_{ij}\right)}^t\}$, representing operation state, where $X{\left(O_{ij}\right)}^t$ is a binary indicator (1 for scheduled operation, 0 otherwise), $T{\left(O_{ij}\right)}^t$ is scheduled time or the estimated minimum completion time defined as:

  $$T{\left(O_{ij}\right)}^t=\left\{\begin{array}{cc}\mathrm{Completion}\mathrm{time}\hfill & \mathrm{if}\mathrm{scheduled}\hfill \\ T{\left(O_{i,j-1}\right)}^t+\underset{C_k\in \mathcal{C}}{min}{\rho }_{ijk}\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

  (7)

Action $\mathcal{A}$. The hierarchical action space contains:

• Operation selection ($a_1^t$): Choose eligible operation $O_{ij}$;
• Thread-worker assignment ($a_2^t$): Choose one thread worker for $O_{ij}$ from candidate set $\mathcal{M}\left(O_{ij}\right)$.

Reward $\mathcal{R}$. The incremental reward at step t is:

$$r(s^t,a_1^t,a_2^t)=-\left[M\left(s^{t+1}\right)-M\left(s^t\right)\right]$$

(8)

The objective is to minimize makespan $M={max}_{O_{ij}}T\left(O_{ij}\right)$ by maximizing cumulative rewards ${\sum }_{t=1}^T{r}^t$.

APG Transition $\mathcal{P}$. At timestep t, after the agent acts with updating one directed arc, a new APG is generated with a new global state ${\mathcal{S}}^{\prime }$ with $s^{t+1}=(s_{sc}^{t+1},s_c^{t+1})$.

5.2. Network Architecture

We employ distinct networks, ${\pi }_{{\theta }_1}\left(a_1\right|s)$ for operation selection policy and ${\pi }_{{\theta }_2}\left(a_2\right|s,a_1)$ for thread-worker assignment policy, to guide the transaction-scheduling process. We use a distinct embedding layer for two policies as follows.

Graph embedding. We adopt a L-layer graph isomorphism network (GIN) [29], a variant of graph neural network (GNN), to compute structural embeddings for $s_{sc}^t$. A GNN is a neural network architecture that operates on graph-structured data by iteratively aggregating feature information from a node’s neighbors. In our case, the embedding of each operation is updated at iteration $l\in \{1,2,\dots ,L\}$ as follows:

$${\mathbf{h}}_l^{O_{ij},t}={\mathrm{MLP}}_{{\theta }_l}\left(\left(1+{ϵ}^{\left(l\right)}\right)·{\mathbf{h}}_{(l-1)}^{O_{ij},t}+\sum _{u\in \mathcal{N}\left(O_{ij}\right)}{\mathbf{h}}_{(l-1)}^{u,t}\right)$$

(9)

where $\mathcal{N}\left(O_{ij}\right)$ denotes the set of operations that have intra-transaction dependencies with $O_{ij}$, and u is one such operation. ${\theta }_l$ represents the learnable parameters of layer l, and ${ϵ}^{\left(l\right)}$ is a trainable scalar that controls the relative weight of self-information and neighbor information during aggregation.

After L iterations, we obtain a global representation of the APG using an average pooling function that aggregates the embeddings of all operations into a vector:

$${\mathbf{h}}_{\mathcal{G}}^t={\displaystyle \frac{1}{\left|\mathcal{T}\right|}}\sum _{O_{ij}\in \mathcal{T}}{\mathbf{h}}_L^{O_{ij}}$$

(10)

We adopt a fully connected layer to encode the thread-worker state $s_c^t$ by computing the embedding vector of each thread worker $h^{C_k,t}$ and the pooling vector $u^t$.

Arc transformation. APG is a mixed graph containing undirected arcs and directed arcs. For operations with no inter-transactions dependency that has undirected edges, we replace each undirected arc with two directed arcs in opposite directions. We ignore undirected disjunctive arcs in the initial state to reduce computational complexity [30], with $s_{sc}^t=\{S{\left(\mathcal{O}\right)}^t,{\kappa }^t\}$ to avoid APG too dense to be efficiently processed by GIN. With the transaction scheduling process, ${\kappa }^t$ becomes larger since more directed arcs will be added.

Action selection. We implement MLP layers as action decoders based on state embeddings. First, we compute the action score of operation selection action ${\Phi }_1^{O_{ij},t}$ and thread-worker assignment action ${\Phi }_2^{C_k,t}$:

$$\begin{array}{c}\hfill {\Phi }_1^{O_{ij},t}=ML{P}_{{\pi }_{{\theta }_1}}(h_{\left(l\right)}^{O_{i,j},t}\left|\right|{\mathbf{h}}_{\mathcal{G}}^t\left|\right|u^t)\\ \hfill {\Phi }_2^{C_k,t}=ML{P}_{{\pi }_{{\theta }_2}}(h^{C_k,t}\left|\right|{\mathbf{h}}_{\mathcal{G}}^t\left|\right|u^t)\end{array}$$

(11)

For operations that are already scheduled or violate precedence constraints, or thread workers that are incompatible with the selected operation. We set their score to $-\infty$. Finally, we normalize the scores using the softmax function, a standard transformation that converts real-valued inputs into a probability distribution. This enables the computation of selection probabilities and supports a sampling-based strategy for action prediction.

$$\begin{array}{c}\hfill P_{O_{ij}}\left(a_1^t\right)={\displaystyle \frac{e^{{\Phi }^{O_{ij},t}}}{{\sum }_{\mathcal{T}}{e}^{{\Phi }^{O_{ij},t}}}}\\ \hfill P_{C_k}\left(a_2^t\right)={\displaystyle \frac{e^{{\Phi }^{C_k,t}}}{{\sum }_{\mathcal{C}}{e}^{{\Phi }^{C_k,t}}}}\end{array}$$

(12)

5.3. Training Algorithm

To train the policy network, we employ the Proximal Policy Optimization (PPO) algorithm [31] with a Sequence Actor for operation selection policy, a Load-Balancing (LB) Actor for thread-worker assignment policy, and a global critic, as shown in Figure 6. Actors running in the environment collect training experience tuples, and then the two sub-policies ${\pi }_{{\theta }_1}$ and ${\pi }_{{\theta }_2}$ are updated via the collected samples.

Objectives of actor network. We implement identical objective computation mechanisms for both actor networks. The overall objective function combines the clipped surrogate objective and entropy regularization:

$$\begin{array}{cc}\hfill L\left(\theta \right)& =\alpha L_{\mathrm{CLIP}}\left(\theta \right)+\beta L_{\mathrm{E}}\left(\theta \right),\hfill \\ \hfill L_{\mathrm{CLIP}}\left(\theta \right)& ={\mathbb{E}}_t\left[min\left({\delta }_{\theta }^t{\widehat{A}}^t,\mathrm{clip}({\delta }_{\theta }^t,1-ϵ,1+ϵ){\widehat{A}}^t\right)\right],\hfill \\ \hfill L_{\mathrm{E}}\left(\theta \right)& ={\mathbb{E}}_t\left[\mathrm{Entropy}\left(\pi \left(a^t\right|s^t)\right)\right],\hfill \\ \hfill {\delta }_{\theta }^t& ={\displaystyle \frac{\pi \left(a^t\right|s^t)}{{\pi }_{\mathrm{old}}\left(a^t\right|s^t)}},\hfill \end{array}$$

(13)

where $\theta =\{{\theta }_1,{\theta }_2\}$ denotes the parameters of both actor networks. Here, $L_{\mathrm{CLIP}}\left(\theta \right)$ represents the clipped surrogate objective, $L_{\mathrm{E}}\left(\theta \right)$ is the entropy regularization term, and ${\delta }_{\theta }^t$ indicates the probability ratio between the updated and old policies. The hyperparameters $ϵ$, $\alpha$, and $\beta$ control the clipping range and objective weighting.

Objective of critic network. Both actors share a common critic network with state-value function $v_{\varphi }\left(s^t\right)$. The advantage function estimator is calculated as follows:

$${\widehat{A}}^t=\sum _{t^{\prime }=t}^T{\gamma }^{t^{\prime }}{r}^{t^{\prime }}-v_{\varphi }\left(s^t\right),$$

(14)

where ${\widehat{A}}^t$ serves as a variance-reduced advantage estimator. The critic network is optimized by minimizing the mean squared error (MSE) between predicted and actual returns:

$$L_{\mathrm{MSE}}\left(\varphi \right)={\mathbb{E}}_t\left[{\left(\sum _{t^{\prime }=t}^T{\gamma }^{t^{\prime }-t}{r}^{t^{\prime }}-v_{\varphi }\left(s^t\right)\right)}^2\right].$$

(15)

6. Deployment of DCoS

DCoS functions as an intelligent scheduling layer within the transaction processing pipeline. It builds upon a correctness enforcement layer that guarantees serializability through partitioning methods and runtime pipelining (RP). Figure 7 depicts the hierarchical interaction among partitioning, CC protocols, and scheduling methods. DCoS has the following characteristics:

Adaptive execution under varying contention levels. In low-contention scenarios, coarse-grained execution is sufficient. Transactions that do not access hot data and are free from contention can be executed entirely within a single partition. This approach eliminates coordination overhead and avoids the complexity associated with fine-grained control mechanisms. When contention is detected, typically through cross-partition accesses to hot data items, DCoS responds by activating a fine-grained scheduler based on DRL. The scheduler reorders and interleaves contention-intensive transaction pieces in a way that minimizes conflicts and improves concurrency.

Complementarity with existing CC protocols. DCoS is designed to serve as a complement to existing correctness mechanisms. Rather than replacing CC protocols that optimize lock acquisition order or validation timing, DCoS enhances them by operating at the scheduling layer. DCoS focuses on improving execution order to reduce conflicts and transaction aborts. DCoS assumes the presence of a base CC protocol, such as RP, to enforce correctness, while it adaptively applies fine-grained scheduling only when it is beneficial.

Limitations and Practical Considerations

DCoS introduces several practical considerations:

• Assumption of known access patterns. DCoS relies on a one-shot transaction processing model where access patterns are known ahead of execution, which may limit its applicability in dynamic or ad hoc workloads with unpredictable access patterns.
• Accuracy in hot data and contention detection. The integration of a DRL-based scheduler requires offline training. The effective activation of fine-grained scheduling depends on the accurate identification of hot data items and contention-intensive transaction pieces.

A promising direction for future work is the tighter integration of DCoS-style scheduling with enhanced CC mechanisms, enabling adaptive selection of scheduling strategies tailored to workload characteristics. Furthermore, exploring lightweight online learning and transfer learning techniques [32] could reduce training overhead and improve scheduler adaptability, broadening DCoS’s applicability across diverse OLTP environments.

7. Experimental Results

In this section, we comprehensively evaluate DCoS by comparing it with existing systems. Specifically, we focus on the following aspects:

• (Section 7.2) First, we assess the effectiveness of the DCoS executor for handling high-contention workloads.
• (Section 7.3 and Section 7.4) Then, we evaluate DCoS against state-of-the-art CCs for handling various workloads.
• (Section 7.5) Finally, we analyze the overhead introduced by DCoS in high-contention scenarios to provide insights into the trade-offs involved in employing DRL-based scheduling.

7.1. Experimental Setup

We implement a prototype of DCoS within the DBx1000 [33] transaction-processing engine and integrate a partitioning algorithm following [15]. Our experiments are conducted on two machines equipped with two 16-core AMD 7302 processors and an NVIDIA 24GB Ampere A30 GPU. One machine initializes the OLTP application, while the other serves as the transaction-processing engine.

Hyperparameters. The embedding network of the executor consists of two layers, each containing two hidden layers with a hidden dimension of 128 neurons. Following common practice in DRL tasks [31], we set the coefficients for clipping, policy loss, value function, and entropy to 0.2, 2, 1, and 0.01, respectively. During training, we use the widely adopted Adam optimizer [34] with a learning rate of $lr=1\times {10}^{-3}$, which is a standard choice in policy gradient algorithms.

Metrics. We measure the performance of the transaction-processing system using the following metrics:

• Makespan: The total execution time of a transaction batch, highlighting the effectiveness of scheduling under contention.
• Throughput: The number of transactions committed per second, reflecting overall system efficiency.

7.2. Microbenchmark Evaluation

In this section, we evaluate the effectiveness of the DCoS DRL-based scheduler by comparing it against three state-of-the-art scheduling methods:

• Simulated Annealing (SA) [10] explores scheduling configurations by probabilistically accepting higher-cost states.
• Dynamic Priority (Priority) [33] prioritizes transactions based on the largest number of pending operations.
• Greedy Scheduling (Greedy) selects immediately available thread workers using a first-fit allocation strategy.

We design a workload generator to evaluate the performance of DCoS under high-contention with varying workload skew. We sample a set of hot data items from the database and then test a set of transactions $|\overline{\mathcal{D}}|$ that will operate on these hot data items. With the increase of concurrent contention-intensive transactions, the contention rate also increases. We manage workload variance by adjusting the number of cold items, while the processing time for hot transactions, denoted as $t_p$, can vary with the cold item distribution. Specifically, we define three workload states among cores based on average transaction-processing time: $t_p\sim \mathcal{U}(100,120)\mathsf{\mu }\mathrm{s}$ for low workload variance, $t_p\sim \mathcal{U}(100,150)\mathsf{\mu }\mathrm{s}$ for medium workload variance, and $t_p\sim \mathcal{U}(100,200)\mathsf{\mu }\mathrm{s}$ for high workload variance.

Varying contention levels. We set the available thread workers to 5 and the number of concurrent contention-intensive transactions (10, 15, 20) to simulate increasing contention rates. The DCoS model was trained for 1000 episodes until convergence was reached, ensuring stable policy learning. We generated 500 distinct transaction workload sets not encountered during training for testing. The mean value of the results is used for comparison; we collect the makespan of processing the batch of hot items as shown in Figure 8a. DCoS consistently outperforms other algorithms across all scenarios, particularly under high workload variance and large concurrent transactions. Compared to the best-performing SA, DCoS demonstrates a makespan reduction of 8.8% to 12.8% under low workload variance and 14% to 15.5% under high workload variance as the number of concurrent transactions increases. The Greedy algorithm shows performance decreases in handling increased concurrency, which aligns with the inherent weakness of greedy approaches, which make locally optimal but globally suboptimal decisions. The priority algorithm performs comparably to SA under moderate conditions; however, with workload variance increases, the performance among SA, Greedy, and Priority is affected due to their lack of adaptability to dynamic adjustments, which limits their effectiveness.

Generalization. To evaluate cross-scale generalization, we conduct transfer learning experiments where DCoS trained on small-scale configurations (five thread workers, 20 concurrent contention-intensive transactions) is directly deployed in large-scale environments (ten thread workers, 20–30 concurrent high-contention transactions). This zero-shot adaptation test eliminates parameter fine-tuning to isolate model generalization capacity. The evaluation protocol maintains 10 worker threads while scaling contention-intensive transactions to represent 100–150% of the original training workloads. Figure 8b shows the testing results, we observe a similar trend to small-scale transactions, with DCoS demonstrating a 12.57% makespan reduction compared to the best-performing SA. Since we implement embedding layers to enable DCoS to generalize effectively to different sizes of concurrent transaction sets and thread workers, even without additional training, this allows DCoS to adapt to various transaction workload scenarios.

7.3. TPC-C Benchmark Evaluation

DBx1000 features a pluggable lock manager that supports multiple CC schemes, enabling a direct comparison of DCoS against various baseline approaches within the same system. Specifically, we implement and evaluate the following three methods:

• NO_WAIT is a variant of 2PL where any conflict leads to the immediate abortion of the requesting transaction.
• Silo [27] is a representative OCC protocol.
• IC3 [8] combines the static analysis of the transaction workload with runtime techniques that track and enforce dependencies among concurrent transactions.

To ensure an industry-standard evaluation, we conduct comprehensive experiments using the TPC-C benchmark [35] with enhanced contention control analysis. We maintain a balanced transaction mix of 50% New-Order and 50% Payment transactions to model real-world order-processing workloads closely. These two transactions comprise the vast majority of the benchmark and have been used previously to evaluate high-contention workloads to stress concurrency control. Our evaluation framework extends the classical TPC-C benchmark by varying two key factors: (i) the number of warehouses while maintaining a constant thread count, and (ii) the number of worker threads while keeping the warehouse count.

We fix the number of worker threads at 32 and vary the number of warehouses from 1 (high contention) to 32 (low contention). As shown in Figure 9, DCoS achieves the highest throughput across all warehouse configurations, significantly outperforming others, especially under high contention. At one warehouse, DCoS is three times higher than Silo and $1.5$ times higher than IC3. These results highlight the effectiveness of DCoS in mitigating contention bottlenecks and maintaining robust performance under varying workloads, since it can avoid unnecessary waiting through intelligent scheduling based on runtime information. At the same time, IC3 only leverages the static information.

To assess system scalability, we vary the number of worker threads from 1 to 32 while keeping the warehouse count fixed at 1, as shown in Figure 10. The performance advantage of DCoS becomes more pronounced as the number of threads increases. At 16 threads, DCoS reaches 3.2 M txns/s, surpassing IC3 by 25% and significantly outperforming NoWait and Silo.

7.4. TPC-E Benchmark Evaluation

To evaluate how well DCoS adapts to a more complex contention scenario, we conduct experiments using the TPC-E benchmark [36]. This benchmark simulates transaction workloads with varying data access patterns. We control contention by changing the frequency of updates to the SECURITY table, using a Zipfian distribution for these updates. We adjust the skew parameter $\theta$ from 0 to 4. Higher values of $\theta$ indicate more significant data access skew and increased contention.

As shown in Figure 11, throughput decreases for all methods as $\theta$ increases, reflecting the impact of contention due to skewed data access. DCoS maintains the highest throughput across all skew levels. At $\theta =0$, DCoS achieves 1.2 M txns/s, slightly behind Silo (1.3 M txns/s), since DCoS adopts a dependency analysis, but at $\theta =4$, DCoS still maintains 0.6 M txns/s, whereas Silo drops to 0.35 M txns/s. Silo performs best under low-skew scenarios but degrades significantly as contention increases. NoWait and IC3 consistently underperform, struggling to handle skewed workloads efficiently. These findings highlight that DCoS is more resilient to data skew than traditional CC methods, as it dynamically adapts to runtime contention.

To analyze the combined effect of parallelism and contention, we vary the number of worker threads while maintaining a Zipfian skew of $\theta =3$. The results, shown in Figure 12, indicate that DCoS significantly outperforms Silo and NoWait beyond 16 threads. At 32 threads, DCoS achieves a throughput that is $1.63$ times higher than Silo, $1.48$ times higher than NoWait, and $1.94$ times higher than IC3.

In summary, the findings from TPC-C and TPC-E demonstrate that DCoS is particularly effective at managing data skew, sustaining high throughput even in scenarios of extreme contention. These results highlight the significance of runtime-adaptive scheduling strategies employed in DCoS for optimizing transaction throughput in OLTP systems.

7.5. Overhead Analysis

To evaluate the runtime overhead of the DRL-based scheduling executor, we analyze the overhead in Section 7.4, where $\theta =4$ represents a highly skewed, high-contention scenario. Our overhead analysis focuses on high-contention workloads, where both the benefits and costs of DCoS are most evident. Figure 13 presents the execution profile across different phases of transaction processing. Note that for IC3, which uses an active waiting mechanism under high contention, we include its associated costs within the analysis phase for comparison. DCoS spends the largest percentage of execution time in the execute phase. Although DCoS introduces an additional analysis phase responsible for hot data item identification and DRL-based scheduling decisions, the abort-related overhead stays below 20%, which is significantly lower than the abort-related overhead observed in NoWait (40%) and Silo (45%). We also observe that IC3, which fully decomposes transactions, incurs even higher analysis overhead while achieving lower execution efficiency. This comparison illustrates that DCoS strikes a more favorable balance between coordination cost and execution throughput.

8. Conclusions

In this paper, we propose DCoS, a transaction-scheduling method based on DRL. DCoS is compatible with existing partitioning methods that divide transactions into conflict-free sets, and adopts a fine-grained scheduling architecture to enable efficient concurrency for high-contention transaction sets. DCoS addresses the transaction-scheduling problem by updating the undirected arcs in the novel APG structure. Our experimental results demonstrate that DCoS improves the system throughput compared to state-of-the-art scheduling approaches across diverse workload scenarios and maintains robust generalization capabilities when exposed to unseen transaction workload patterns. The fine-grained scheduling mechanism is selectively applied in high-contention scenarios, thereby avoiding unnecessary overhead in low-contention settings. Future work involves extending DCoS to distributed settings and examining its compatibility with emerging CC schemes across varying deployment architectures.