Boosting the Performance of CDCL-Based SAT Solvers by Exploiting Backbones and Backdoors

Abstract

Boolean structural measures were introduced to explain the high performance of conflict-driven clause-learning (CDCL) SAT solvers on industrial SAT instances. Those considered in this study include measures related to backbones and backdoors: backbone size, backbone frequency, and backdoor size. A key area of research is to improve the performance of CDCL SAT solvers by exploiting these measures. For the purpose of guiding the CDCL SAT solver for branching on backbone and backdoor variables, this study proposes low-overhead heuristics for computing these variables. Through these heuristics, a set of modifications to the Variable State Independent Decaying Sum (VSIDS) decision heuristic is suggested to exploit backbones and backdoors and potentially improve the performance of CDCL SAT solvers. In total, fifteen variants of two competitive base solvers, MapleLCMDistChronoBT-DL-v3 and LSTech, were developed. Empirical evaluation was conducted on 32 industrial families from 2002–2021 SAT competitions. According to the results, modifying the VSIDS heuristic in the base solvers to exploit backbones and backdoors improves its performance. In particular, our new CDCL SAT solver, LSTech_BBsfcr_v1, solved more industrial SAT instances than the winning CDCL SAT solvers in 2020 and 2021 SAT competitions.

1. Introduction

The Boolean satisfiability problem (SAT) [1] is a fundamental NP-complete problem in automated reasoning and mathematical logic. As NP-complete problems can be reduced to SAT in polynomial time, SAT is applicable to a wide range of fields [2,3,4].

The annual SAT competitions have become an essential event for the distribution of SAT benchmarks and the development of new SAT-solving methods [5]. Sequential SAT solvers compete mainly in three categories: industrial, crafted, and random tracks. The SAT competitions have demonstrated how difficult it is for SAT solvers to perform well across all categories. Results show that conflict-driven clause-learning (CDCL) SAT solvers were most performant for solving industrial and crafted SAT benchmarks, whereas look-ahead and Stochastic Local Search (SLS)-based SAT solvers have dominated the random category [5]. Modern implementations of CDCL SAT solvers employ a lot of heuristics. Some of them can be considered baseline, such as the Variable State Independent Decaying Sum (VSIDS) [6], restarts [7], and Literal Block Distance (LBD) [8]. Several others were incorporated recently, including: Learnt Clause Minimization (LCM) [9], Distance (Dist) heuristic [10], Chronological Backtracking (ChronoBT) [11], duplicate learnts heuristic [12], Conflict History-Based (CHB) heuristic [13], Learning Rate-based Branching (LRB) heuristic [14], and the SLS component [15]. The results of the SAT competitions have led researchers to conclude that (1) industrial, crafted, and random SAT instances have distinct structures, and (2) SAT-solving methods could exploit such structures.

Boolean structural measure proposed for SAT include the phase transition [16], backbone size [17], and backdoor size [18,19]. We propose three new related measures to the backbones and backdoors: the backbone frequency, backbone coverage, and backdoor coverage (The reader is referred to Appendix A where the evidence of the backbone and backdoor-related measures on industrial, crafted, and random benchmark instances drawn from 2002–2020 SAT competitions are investigated. In particular, we evaluated the backbone size, LSR backdoor size, and backbone/backdoor variable overlap size, along with the three proposed new related measures: backbone frequency, backbone coverage, and backdoor coverage.). Empirical results indicate that most random benchmarks have no backbones, whereas on average, industrial and crafted benchmark instances have small backbone sizes [20,21]. The frequency of backbones is low for all benchmark categories. As for the backbone coverage, industrial and crafted benchmarks have higher coverages, on average, than random ones. In both crafted and random benchmark instances, the backdoor size and coverage are greater than those in the industrial category [20,21]. Additionally, across all SAT benchmarks, there tends to be a little overlap between backbone and backdoor variables [20,21].

A fundamental problem of interest and practical importance concerns the possibility of enhancing the performance of SAT solvers by exploiting the inherent structure of the instances. Several studies have shown, for example, that backbone/backdoor-guided branching heuristics improve the solver performance [18,22,23,24]. Despite these efforts, this problem remains an open issue [19,25]. One reason is that the computational estimation of most Boolean structural measures (e.g., backbone and backdoor) is intractable [26]. Second, as a consequence, the structural measures of SAT and their impact on the behavior of SAT solvers have not been fully investigated [19,25].

The present study examines the relevance of Boolean structural measures on the performance of CDCL SAT solvers. The aim is to improve the performance of CDCL SAT solves by exploiting Boolean structural measures. In achieving this, low-overhead heuristics for computing backbones and backdoors are proposed. Using these heuristics, the VSIDS heuristic is extended to exploit backbones and backdoors. In particular, upon restart, conflict, and/or backtrack, variables that are likely to be backbones and/or backdoors are bumped. Accordingly, competitive CDCL SAT solver(s) are developed by improving two state-of-the-art CDCL SAT solvers, MapleLCMDistChronoBT-DL-v3 and LSTech, to exploit backbones and backdoors. The main contributions of this work are threefold:

• To compute backbones and backdoors based on a low-overhead computational heuristic, in order to guide the CDCL SAT solver to branch on backbone/backdoor variables (Section 4.1).
• To extend the VSIDS variable decision heuristic to exploit backbones and backdoors (Section 4.2).
• To develop a competitive CDCL based SAT solver by improving two state-of-the-art solvers, MapleLCMDistChronoBT-DL-v3 and LSTech, to exploit backbones and backdoors (Section 4 and Section 5).

The rest of the paper is organized as follows: In Section 2, the definitions and notations used throughout the paper are introduced. In Section 3, a review of state-of-the-art CDCL SAT solvers is provided. In particular, the base solvers that are the focus of this study, MapleLCMDistChronoBT-DL-v3 and LSTech, are described. In addition, algorithms for computing backbone and backdoor variables are reviewed. In Section 4, a description of the proposed CDCL SAT solvers that exploit backbones and backdoors is presented. In Section 5, results of the experimental evaluation are reported. This is followed, in Section 6, by a discussion of the main findings and issues. Finally, Section 7 contains concluding remarks and suggests future directions.

2. Definitions and Notations

The Boolean satisfiability problem is defined in this section, which is followed by formal definitions of backbones and backdoors.

2.1. Boolean Satisfiability Problem

A Boolean variable x may take the value $true$ or $false$. A literal is a Boolean variable x or its negation $\overline{x}$. A clause C is a disjunction of literals. Given a CNF formula $\varphi$ defined over X, where $X=\{x_1,x_2,\cdots ,x_n\}$ is a set of Boolean variables and $L=\{x_i,{\overline{x}}_i|x_i\in X,1\le i\le n\}$ is a set of literals over X, the following definitions hold:

Definition 1

(Conjunctive Normal Form (CNF) formula). A CNF formula ϕ is a conjunction of clauses, which is represented as a multiset of clauses $\{C_1,C_2,\cdots ,C_m\}$.

Definition 2

(k-CNF formula). A k-CNF formula is a CNF formula with exactly k literals per clause.

Definition 3

(Assignment). An assignment is a mapping from each variable $x_i$ to $\{true,false\}$. This is denoted by:

$${\varphi |}_{\{x_i=valu{e}_i|\forall x_i\in X,valu{e}_i\in \{true,false\}\}}$$

Definition 4.

An assignment satisfies a literal $x_i$ if $x_i=true$, and satisfies a literal ${\overline{x}}_i$ if $x_i=false$.

Definition 5.

An assignment satisfies a clause $C_i$ if it satisfies at least one literal in $C_i$.

Definition 6.

An assignment satisfies a CNF formula if it satisfies all clauses.

Definition 7.

A CNF formula ϕ is satisfiable if there exists an assignment that satisfies ϕ, that is:

$${\varphi |}_{\{x_i=valu{e}_i|\forall x_i\in X,valu{e}_i\in \{true,false\}\}}=true$$

Otherwise, the formula ϕ is unsatisfiable:

$${\varphi |}_{\{x_i=valu{e}_i|\forall x_i\in X,valu{e}_i\in \{true,false\}\}}=false$$

Definition 8

(SAT problem [1]). Given a CNF formula ϕ over X, the SAT problem asks whether there is an assignment that satisfies ϕ, or it decides that the formula is unsatisfiable.

Definition 9

(SAT solver). A SAT solver is a computer program which aims to solve the SAT problem.

2.2. Boolean Structural Measures

Definition 10

(Backbone literal [17]). $x_b$ is a backbone literal of ϕ if for all satisfying assignments of ϕ, the value of $x_b$ is fixed.

Definition 11

(Backbone of a CNF formula [17]). The backbone of ϕ is the set of all backbone literals, which is denoted $B{B}_{\varphi }$.

Definition 12

(Backbone size of a CNF formula [17]). The backbone size of ϕ is the cardinality of $B{B}_{\varphi }$, which is denoted $|B{B}_{\varphi }|$. The normalized backbone size of ϕ is the ratio of the backbone size to the number of variables $\left|X\right|$, which is denoted $BBsiz{e}_{\varphi }$:

$$BBsiz{e}_{\varphi }=\frac{|B{B}_{\varphi }|}{\left|X\right|}$$

(1)

Definition 13

(Backbone frequency). The backbone frequency of a backbone literal $x_b\in B{B}_{\varphi }$ is the ratio of the total number of occurrences of $x_b$ in ϕ to the total number of clauses (after CNF formula simplification, which involves reducing the number of variables and/or clauses in a CNF formula to a logically equivalent formula. It is part of the preprocessing phase that takes place before the solving phase.), which is denoted $BBfr{e}_{x_b}$. This can be expressed as follows:    

$$BBfr{e}_{x_b}=\frac{{\displaystyle \sum _{\begin{array}{c}i=1\\ x_b\in C_i,C_i\in C\end{array}}^{\left|C\right|}}1}{\left|C\right|}$$

(2)

The backbone frequency of ϕ is the percentage average of the backbone frequencies of all backbone literals, which is denoted $BBfr{e}_{\varphi }$. This is expressed as follows:

$$BBfr{e}_{\varphi }=\frac{{\displaystyle \sum _{\begin{array}{c}b=1\\ x_b\in B{B}_{\varphi }\end{array}}^{|B{B}_{\varphi }|}}BBfr{e}_{x_b}}{|B{B}_{\varphi }|}\times 100$$

(3)

Definition 14

(SAT sub-solver [18]). A SAT sub-solver Γ is an algorithm that takes an input CNF formula ϕ and satisfies the following:

1. 

(Trichotomy) Γ correctly determines ϕ. If satisfiable it returns a solution; otherwise, it is unsatisfiable.

2. 

(Efficiency) Γ runs in polynomial time.

3. 

(Trivial-solvability) Γ can determine whether ϕ is trivially

• $true$, that is, ϕ has no clauses; or
• $false$, that is, ϕ has an empty clause.

4. 

(Self-reducibility) If Γ determines ϕ, then $\forall x_i\in X$, Γ determines:

$${\varphi }^{*}{|}_{\{x_i=valu{e}_i|valu{e}_i\in \{true,false\}\}}.$$

(4)

where the latter denotes a simplified formula by fixing the value of any $x_i$ to $true$ or $false$.

Definition 15

(Weak backdoor [18]). A non-empty subset of variables $B{D}_{\varphi }\subseteq X$ is a weak backdoor with respect to sub-solver Γ if there exists a truth assignment of $B{D}_{\varphi }$ s.t. Γ returns a satisfying assignment.

Definition 16

(Strong backdoor [18]). A subset of variables $B{D}_{\varphi }\subseteq X$ is a strong backdoor with respect to sub-solver Γ if for all truth assignments of $B{D}_{\varphi }$, Γ returns a satisfying assignment or concludes unsatisfiability.

Definition 17

(Learning-Sensitive (LS) backdoor [27]). A subset of variables $B_{LS}$ is an LS backdoor with respect to sub-solver Γ if there exists a search tree exploration order such that a CDCL SAT solver branches only on variables in $B_{LS}$, and with Γ and learnt clauses at the leaves of the search tree, it either finds a satisfying assignment for ϕ, or it proves that ϕ is unsatisfiable.

Definition 18

(Learning Sensitive with Restarts (LSR) backdoor [19]). A subset of variables $B_{LSR}$ is an LSR backdoor with respect to sub-solver Γ if there exists a search tree exploration order with restarts such that a CDCL SAT solver branches only on variables in $B_{LSR}$, and with a sub-solver Γ and learnt clauses at the leaves of the search tree, it either finds a satisfying assignment for ϕ or proves that ϕ is unsatisfiable.

Definition 19

(Backdoor size of a CNF formula [18]). The backdoor size of ϕ is the cardinality of $B{D}_{\varphi }$, denoted by $|B{D}_{\varphi }|$. The normalized backdoor size of a CNF formula ϕ is the ratio of the backdoor size to the number of variables of $\left|X\right|$, which is denoted $BDsiz{e}_{\varphi }$. This can be expressed as follows:

$$BDsiz{e}_{\varphi }=\frac{|B{D}_{\varphi }|}{\left|X\right|}$$

(5)

Definition 20

(Backbone/backdoor variable overlap [20]). (To ensure the correctness of computing the backbone/backdoor variable overlap set, we relax the set $B{B}_{\varphi }$ to be the set of variables, not literals. As such, for all backbone literals in $B{B}_{\varphi }$, their corresponding variables are considered.) Let $B{B}_{\varphi }$ and $B{D}_{\varphi }$ be the backbone and backdoor of a CNF formula ϕ, respectively. The backbone/backdoor variable overlap is referred to as $BB{D}_{\varphi }$, and it is given by:

$$BB{D}_{\varphi }=B{B}_{\varphi }\cap B{D}_{\varphi }$$

(6)

The backbone/backdoor variable overlap size of a CNF formula ϕ is the cardinality of $BB{D}_{\varphi }$, which is denoted by $|BB{D}_{\varphi }|$. The normalized backbone/backdoor size is the ratio of $|BB{D}_{\varphi }|$ to the number of variables $\left|X\right|$, which is denoted by $BBDoverla{p}_{\varphi }$:

$$BBDoverla{p}_{\varphi }=\frac{|BB{D}_{\varphi }|}{\left|X\right|}$$

(7)

3. Related Work

CDCL SAT solvers follow primarily the Davis–Putnam–Logemann–Loveland (DPLL) algorithm [28], and they incorporate a number of effective techniques, including clause learning [29], lazy data structures [6], deletion policies [30], and restarts [31]. Typically, a CDCL SAT solver is organized into three main modules [6,29,32]: the decision module, used for branching; the deduction module, used for unit propagation and the identification of unsatisfied clauses (or conflicts); and the diagnosis module, which is used for conflict analysis and clause learning (see Figure 1). Accordingly, the main runtime breakdown of a CDCL SAT solver is: 10% decisions, 80% propagation, and 10% account for conflict analysis [6].

3.1. CDCL Solvers in SAT Competitions

The state of the art in CDCL SAT solvers can be thought of as the solvers that have participated in recent SAT competitions [5]. Most participating CDCL SAT solvers typically include at least one version or hack (To hack a base solver means to improve the solver by making only minor changes to its source code.) of MiniSat [32], Glucose [8], CryptoMiniSat [33], CaDiCaL [34], CoMiniSatPS [35], or more often, the most recent SAT competition(s) winners (e.g., [12,34]). For the latter to show an improvement of a CDCL SAT solver with a new heuristic, the solver must be compared to the base solver without any modifications.

Table 1. The configuration of CDCL SAT solvers considered in this study. In the last column, under SAT competition rank, the SAT, UNSAT, or ALL indicate the type of track, which is either satisfiable, or unsatisfiable, or both, respectively.

CDCL Solver	Base Solver	Decision Heuristic	Restart Heuristic	Backtracks	Improvements	SAT Competition Rank
MapleLCM DistChronoBT -DL-v3 [12]	MapleLCM DistChronoBT	LRB and VSIDS	Luby restarts for LRB Glucose-style restarts for VSIDS	ChronoBT	Duplicate learnts heuristic Deterministic LRB-VSIDS switching	Main track ALL/UNSAT 1st (2019) Main track SAT 2nd (2019)
Relaxed LCMDCBDL- newTech [15]	MapleLCMDist- ChronoBT-DL	LRB and VSIDS	Luby restarts for LRB Glucose-style restarts for VSIDS	ChronoBT	Relaxed CDCL approach Probability Based Phase Saving	Main track ALL 2nd (2020) Main track SAT 1st (2020)
Kissat_MAB [39]	Kissat	VSIDS and CHB	Infrequent restarts for SAT mode Frequent restarts for UNSAT mode	ChronoBT	Multi-Armed Bandit framework which combines VSIDS and CHB	Main track ALL/SAT 1st (2021)
Kissat_gb [38]	Kissat_SAT	VSIDS and CHB	Infrequent restarts for SAT mode Frequent restarts for UNSAT mode	ChronoBT	Glue Bumping (GB) based on glue centrality of glue variables	Main track ALL/SAT 3rd (2021)
LSTech [37]	Relaxed LCMDCBDL- newTech	LRB and VSIDS	Luby restarts for LRB Glucose-style restarts for VSIDS	ChronoBT	Relaxed CDCL approach based on number of restarts	Main track SAT 2nd (2021)

shows the configuration of the winning CDCL SAT solvers used in this study (MapleLCMDistChronoBT-DL-v3 and LSTech are selected as the base solvers. In addition to the base solvers, the proposed solvers were evaluated against Relaxed_LCMDCBDL_newTech, Kissat_GB, and Kissat_MAB.).

MapleLCMDistChronoBT-DL-v3 [12] is based on the winner of the 2018 SAT competition, MapleLCMDistChronoBT [36], and augmented with duplicate learnt heuristic. In particular, Kochemazov et al. [12] improved the three-tier clause management by persisting additional clauses through the hash-based detection of repeatedly learnt clauses. They presented their solver MapleLCMDistChronoBT-DL-v3 in the 2019 SAT Race.

Relaxed_LCMDCBDL_newTech, a relaxed variant of MapleLCMDistCBT-DL, was first introduced in the 2020 SAT competition [15]. Basically, the idea is to relax the backtracking by protecting promising partial assignments from being pruned. Specifically, during the search, whenever a node corresponding to a promising assignment is reached, the algorithm enters a non-backtracking stage (under some conditions); this leads to a complete assignment, which is fed to an SLS solver to search for a solution nearby. In the 2021 SAT competition, its variant LSTech [37] showed a good performance especially on satisfiable instances [37].

The CDCL solver Kissat [34] is another popular base SAT solver that placed first in the 2020 SAT competition main track. Kissat is a low-level re-implementation of CaDiCaL that features improved data structures, better scheduling of inprocessing, and optimized algorithms and implementation. One configuration of Kissat is Kissat_SAT that targets satisfiable instances. Kissat_SAT is the base solver of Kissat_GB [38] and Kissat is the base of Kissat_MAB [39]. Kissat_GB bumps the variables based on glue centrality of glue variables. Kissat_MAB was augmented with the CHB decision heuristic as specified in [13]. The solver incorporates a reinforcement learning technique under the Multi Armed Bandit (MAB) framework that combines the VSIDS and the CHB branching heuristics by adaptively choosing the relevant heuristic at each restart using the Upper Confidence Bound (UCB) strategy.

3.2. Algorithms for Computing Backbones and Backdoors

Throughout the literature, many algorithms for backbone computation have been proposed. To begin with, Kaiser and Küchlin [40] proposed three backbone computation algorithms based on model enumeration and SAT testing. To compute backbones, Climer and Zhang [41] proposed a graph reduction technique called limit crossing. Janota et al. [26] computed backbones based on enumerating implicants and iterative SAT testing and optimizations with calls to a CDCL SAT solver. The work of Previti and Järvisalo [42] follows the idea of enumerating implicants in [26], but it differs in terms of using a CDCL SAT solver with preferences. Zhang et al. [43] suggested three sets of filtering optimization heuristics to improve the performance of the iterative SAT testing method for backbone computation. Dequen and Dubois [24] introduced a heuristic for backbone computation. The heuristic was incorporated into the DPLL solver, kcnfs, to encourage branching on backbone variables. Experimental results indicated that kcnfs performed well on random k-CNF formulas [5]. Wu [44] applied machine learning to optimize the values of the branching variables in MiniSat [32]. Experimental results confirmed that the solver managed to set on average 78% of the backbones correctly. The solver did reduce conflicts; however, the long preprocessing time outweighed the decrease in runtime.

Many algorithms for backdoor computation have been proposed in the literature. Williams et al. [18] computed strong backdoors in several industrial instances. Experimental results indicated that the size of the backdoor is close to zero. The authors also investigated the relatedness between backdoors, restarts, and the heavy-tailed distribution phenomena. They suggested that backdoors with sizes near-zero lead to runtime distributions that are lower bounded by heavy tails. This led Williams et al. [18] to hypothesize that SAT solvers that are effective in solving industrial SAT instances exploit backdoors. Kilby et al. [20] concluded that strong backdoors seem to be correlated with problem hardness on random 3-CNF formulas, whereas this was not observed for weak backdoors. Gregory et al. [21] studied weak backdoors for random and crafted instances. They observed that backdoor values for crafted instances are close to zero. Moreover, the authors discovered that when clause learning is enabled, the average backdoor size decreases. This is of interest because modern CDCL solvers implicitly exploit the backdoor structure. Zulkoski et al. [19] computed weak backdoors for all SAT categories on benchmark instances from 2009–2014 SAT competitions. Experiments concluded that weak backdoors are hard to compute and small for all instance categories. Dilkina et al. [45] extended traditional backdoors to LS backdoors to take advantage of clause learning [29] during the search performed by a CDCL SAT solver. Experiments showed that LS backdoors are exponentially smaller than traditional strong backdoors on mixed integer programming SAT instances [46]. Overall, experiments on instances from all SAT categories have reported a near-zero backdoor size. Zulkoski et al. [19] extended the notion of LS backdoors to allow restarts by introducing the concept of LSR backdoors. The results confirmed that industrial instances indeed appear to have smaller LSR backdoor sizes compared to random instances. Zulkoski et al. [19] observed that the number of LSR backdoors that have been computed are of twice the instances of weak backdoors. Zulkoski et al. [19] concluded that LSR backdoor sizes are larger compared to weak backdoors. However, weak backdoors are harder to compute.

3.3. MapleLCMDistChronoBT-DL-v3 and LSTech

The base solvers considered in this study, MapleLCMDistChronoBT-DL-v3 and LSTech, are described in the following section (The reader is referred to Appendix B.1 for a justification of the selection of base solvers.). Both solvers follow the general CDCL framework (A general framework for a CDCL solver is provided in Appendix C.1.). We specifically discuss the branching decision heuristic augmented in both solvers, as the proposed improvements are focused on it (A justification for choosing the branching decision heuristic to exploit backbones in order to improve the base solvers is described in Appendix B.2. In addition, Appendix C.2 provides the algorithm for the decision heuristic VSIDS implemented in Maple-based series SAT solvers).

Algorithm 1 describes a top-level implementation for MapleLCMDistChronoBT-DL-v3. There are three decision heuristics augmented in it: Dist, VSIDS, and LRB. For the first 50,000 conflicts, the solver branches using the Dist heuristic (lines 18–19). After the first 50,000 conflicts, the solver branches based on a deterministic LRB/VSIDS switching strategy (lines 21–25). It starts from LRB and switches each time the number of propagations since the last switch exceeds a threshold value. This value is initially set to 30,000,000 propagations; then, it increases by 10% with every switch (line 27).

A top-level implementation for LSTech SAT solver is presented in Algorithm 2. As the Dist heuristic did not improve the performance of the solver, the developers removed it. The solver switches based on a new deterministic restart-based strategy (lines 35–41). Each time the solver loops a threshold value, it will switch between VSIDS and LRB once. This threshold value is set to 500 (line 35). In addition, LSTech relaxes the backtracking process for protecting promising partial assignments (lines 33–34). When the solver reaches a promising partial assignment, it enters a non-backtracking stage until it obtains a full assignment (line 34). The SLS solver, CCAr, is then called every number of restarts ($sthreshold$) in order to find a satisfying assignment close to the full assignment (lines 16–28). This value of $sthreshold$ is set at 300, but if the SLS solution does not improve, it increases; otherwise, it decreases, keeping the value above 300 (lines 24–27). If the SLS solver fails to find a satisfying assignment within certain limits, then the solver goes back to where it was interrupted (line 28).

Algorithm 1: Pseudocode for top-level implementation of MapleLCMDistChronoBT-DL-v3.

Algorithm 2: Pseudocode for top-level implementation of LSTech.

4. The Proposed CDCL SAT Solvers

A description of the proposed CDCL SAT solvers that exploit backbones and backdoors is presented in this section. To begin with, for the purpose of guiding the CDCL SAT solver to branch on backcbones/backdoors, two low-overhead heuristics are proposed for computing backbones and backdoors. They are described in the following subsection.

4.1. Backbone and Backdoor Computational Heuristics

This section details the proposed heuristics for computing the backbone and backdoor sets: backbone low-overhead (BBLO) heuristic and backdoor low-overhead (BDLO) heuristic. Both heuristics are designed based on relevant features of the backbone and backdoor while having low computational overhead. As described below, this trade-off between accuracy and computational time is motivated by the work of [47]. It should be noted that these heuristics are not intended to be standalone heuristics for computing the backbones and backdoors. Rather, they are incorporated within the CDCL SAT solver, in particular the branching heuristic, to encourage the solver to branch on these variables.

4.1.1. The BBLO Heuristic

The BBLO heuristic was inspired by the low-overhead computation of backbones in [47]. Menai and Batouche [47] proposed a backbone-based co-evolutionary algorithm for the partial maximum satisfiability problem guided by the estimated backbone literals of the problem. That is, literals that are set to the same value on each run will be considered backbones; otherwise, they will not. BBLO adopts a similar concept except that local search runs are the search tree trails and non-backbone literals are identified in each run (A trail is the partial assignment that represents the current path in the search tree produced by a CDCL SAT solver.). This process is repeated for each restart.

The BBLO heuristic is presented in Algorithm 3. It is called during every restart procedure in Algorithms 1 and 2. At the first call of the Algorithm 3, all variables in $list$ are marked as undefined. Upon completing a trail, the initial value of the $list$ is its final value of the previous trail. For each assigned variable (line 3), the heuristic determines if it is not a backbone literal whenever a variable has a different assignment than the previous trail (lines 7 and 8). At the end, all assigned variables in $List$ are moved as literals to the backbone set.    

Algorithm 3: A backbone-based low-overhead computational heuristic (BBLO).

4.1.2. The BDLO Heuristic

The BDLO heuristic is inspired by the findings in [20,21] as well as our findings in Appendix A that show experimentally the near-zero overlap between backbone and LSR backdoor variables on all benchmark categories. Consequently, the BDLO heuristic computes the backdoor variables from the backbones in Algorithm 3. As computed in Algorithm 4, backdoor variables are those that are not backbones (line 9).

Algorithm 4: A backdoor-based low-overhead computational heuristic (BDLO).

4.2. VSIDS Variants

We chose to exploit backbone and backdoor variables by extending the VSIDS heuristic (A detailed description of the VSIDS heuristic is provided in Appendix C.2, along with a justification for choosing the branching heuristic to improve the solver is given in Appendix B.2). The following is a summary of VSIDS’s policy implemented in MapleLCMDistChronoBT-DL-v3 and LSTech:

• Initialization: Each variable has a floating point number, called activity, which is initialized to 0.
• Additive bump: Following a conflict analysis phase, the activities of all variables that led to a conflict are additively bumped (increased), typically by 1, if their decision levels are greater than the backtrack level; otherwise, they are bumped by 0.5.
• Decision: The (unassigned) variable with the highest activity is chosen at each decision.
• Multiplicative decay: All variables are periodically decremented by multiplying their activities by a constant $0<Decay<1$ called the multiplicative decay factor.

To exploit backbone and backdoor variables, the VSIDS heuristic was extended, specifically, the additive bump policy. We considered the following cases: when to bump, which variables to bump, and with what value to bump. Accordingly, six variants of the VSIDS heuristic were derived: VSIDS_BBsize_restart, VSIDS_BBsize_BBfreq_restart, VSIDS_BBsize_BBfreq_backtrack, VSIDS_BBsize_BBfreq_conflict, VSIDS_BBsize_BBfreq_ restart_conflict, and VSIDS_BDsize_restart. Algorithms 5–10 are variants of the VSIDS, and throughout, we highlight only the added policies to the VSIDS implemented in Maple-based series SAT solvers (see Appendix C.2). For all algorithms, all variables’ activities are initialized to zero. In addition, the backbones and backdoors are computed on every restart by calling BBLO or BDLO. The policy of each variant is detailed below:

1.

VSIDS_BBsize_restart (Algorithm 5): Following a restart, the activities of the backbone literals computed in Algorithm 3 are additively bumped, typically by 1 (lines 4–5).

Algorithm 5: VSIDS_BBsize_restart a variant of the VSIDS decision heuristic.

2.

VSIDS_BBsize_BBfreq_restart (Algorithm 6): Following a restart, the activities of the backbone literals computed in Algorithm 3 are additively bumped typically by their frequencies (lines 4–6).

Algorithm 6: VSIDS_BBsize_BBFreq_restart a variant of the VSIDS decision heuristic.

3.

VSIDS_BBsize_BBfreq_backtrack (Algorithm 7): Following a backtrack, the activities of the backbone literals computed in Algorithm 3 are additively bumped typically by their frequencies (lines 3–4).

Algorithm 7: VSIDS_BBsize_BBFreq_backtrack a variant of the VSIDS decision heuristic.

4.

VSIDS_BBsize_BBfreq_conflict (Algorithm 8): Following a conflict, the activities of the backbone literals computed in Algorithm 3 that led to the learnt clause (including those in the learnt clause) are additively bumped typically by their frequencies (lines 3–6).

Algorithm 8: VSIDS_BBsize_BBfreq_conflict a variant of the VSIDS decision heuristic.

5.

VSIDS_BBsize_BBfreq_restart_conflict (Algorithm 9): In combination of variant 2 and 4.

Algorithm 9: VSIDS_BBsize_BBfreq_restart_conflict a variant of the VSIDS decision heuristic.

6.

VSIDS_BDsize_restart (Algorithm 10): Following a restart, the activities of the backdoor variables computed in Algorithm 4 are additively bumped typically by 1 (lines 4–5).

Algorithm 10: VSIDS_BDsize_restart a variant of the VSIDS decision heuristic.

4.3. The CDCL SAT Solvers

To demonstrate that the six proposed VSIDS decision heuristics (Algorithms 5–10) contribute to the state of the art, we implemented each of them on top of two base CDCL SAT solvers, MapleLCMDistChronoBT-DL-v3 and LSTech. In doing so, three different versions of the six extended CDCL SAT solvers were developed. In version 1, during restarts/conflicts/backtracks, bumping occurs only in the VSIDS phase. In version 2, bumping during conflicts is only for the VSIDS phase. But, for restarts/backtracks the bumping occurs in both VSIDS and LRB phases (As the bump in VSIDS_BBsize_BBfreq_conflict occurs only during conflicts, it is the same for versions 1 and 2.). Version 3 is the same as version 1, except that in computing the bump with the backbone frequency, the polarity of variables is taken into account (Version 3 is not applicable to the variants VSIDS_BBsize_restart and VSIDS_BDsize_restart.). For convenience, we name each solver by starting with the base solver name, which is followed by the VSIDS variant, and then the version number, e.g., MapleLCMDistChronoBT-DL-v3_VSIDS_BDsize_restart_version1.

5. Performance Evaluation

5.1. Benchmarks and Experimental Setup

Experimental evaluation was performed on industrial benchmark instances drawn from 2002–2021 SAT benchmark instances. On the basis of

Table 2. A summary of industrial benchmarks used in the performance evaluation drawn from 2002–2021 SAT benchmark instances.

#families (satisfiable-unsatisfiable-both)	32 (12-5-15)
#instances	329
#satisfiable instances (with backbones-without backbones)	189 (185-4)
#unsatisfiable instances	140

, we can see that there are 32 industrial families out of which 12 are satisfiable, 5 are not, and 15 are both. For the purpose of reducing bias, each family contains the same number of satisfiable or/and unsatisfiable instances, seven, and almost the same values of backbone-related measures: backbone size, frequency, and coverage.

The CDCL SAT solvers were evaluated on Shaheen II [48], which is a supercomputer at King Abdullah University of Science and Technology (KAUST). Each node of the cluster is equipped with 128 GB of DDR4 memory and a dual CPU based on 16-core Intel Haswell processors running at 2.3 GHz. In accordance with the literature, the time limit for solving each instance was set at 3600 s. A comparison is presented here between the extended proposed CDCL SAT solvers and their counterpart bases as well as three state-of-the-art winning solvers: RelaxedLCMDCBDL_newTech [15], Kissat_GB [38], and Kissat_MAB [39]. The primary metric is the number of solved instances along with the PAR-2 score. The PAR-2 score is the sum of runtimes for all solved instances and twice the timeout for each unsolved instance, so that a lower score is better.

5.2. Results

The performance results of the six proposed VSIDS extensions implemented in 15 variants of MapleLCMDistChronoBT-DL-v3 and LSTech are presented in

Table 3. Performance results of the CDCL SAT variants on industrial benchmark instances. The versions 1, 2, and 3 are explained in Section 4.3. The base solver Maple is short for MapleLCMDistChronoBT-DL-v3.

		Number of Solved Industrial Instances
	VBS 1	329
		Version 1		Version 2		Version 3
	Base Solver	Maple	LSTech	Maple	LSTech	Maple	LSTech
		302	308	302	308	302	308
1	VSIDS_BBsize_restart	301	309	301	310	— 3
2	VSIDS_BBsize_BBfreq_restart	302	304	307	308	305	309
3	VSIDS_BBsize_BBfreq_backtrack	296	305	290	302	299	302
4	VSIDS_BBsize_BBfreq_conflict	303	308	— 2		299	308
5	VSIDS_BBsize_BBfreq_restart_conflict	300	312	301	310	302	311
6	VSIDS_BDsize_restart	299	310	302	308	— 3

¹ Virtual Best Solver; ² Implementation of version 1 and 2 for this variant is the same, since this variant does not bump variables during restarts; ³ Version 3 is not applicable, since this variant does not bump variables based on the backbone frequencies.

. Each entry represents the number of solved instances. Bold values indicate that the variant performed better than the base solver. MapleLCMDistChronoBT-DL-v3 and LSTech solved 302 and 308, respectively, out of 329 industrial instances. The best performing variant for each base solver is MapleLCMDistChronoBT-DL-v3_VSIDS_BBsize_BBfreq_restart_version2 (Maple_VBBsfr_v2) and LSTech_VSIDS_BBsize_BBfreq_restart_conflicts_version1 (LSTech _VBBsfrc_v1). Maple_VBBsfr_v2 solved 307 instances, which was five more than the base solver. With LSTech_VBBsfrc_v1, four more instances were solved than with the base solver. For the solvers based on VSIDS_BDsize_restart, the only variant that improved is LSTech_VSIDS_ BDsize_restart_version1. Furthermore, note that the solver variants based on backtracks (row number 3) did not perform well.

The scatter plots shown in Figure 2a,b detail performance comparisons of Maple_ VBBsfr_v2 and LSTech_VBBsfrc_v1 to their base solvers. Each improved variant solved more instances and achieved a lower PAR-2 score than its base counterpart.

Table 4. Performance evaluation of CDCL SAT solvers on industrial instances from the 2002–2021 SAT competitions. #SAT (#UNSAT) denotes the number of solved satisfiable (unsatisfiable) instances. Appendix D reports the results in detail.

	#SAT	#UNSAT	Total
VBS	189	140	329
MapleLCMDistChronoBT-DL-v3	175	127	302
Maple_VBBsfr_v1	177	130	307
LSTech	179	129	308
LSTech_VBBsfrc_v2	183	129	312
Relaxed_LCMDCBDL_newTech	178	128	306
Kissat_GB	166	131	297
Kissat_MAB	183	128	311

and Figure 3 report runtime results of Maple_VBBsfr_v2 and LSTech_VBBsfrc_v1 compared to state-of-the-art CDCL SAT solvers on the whole benchmark instances in Table 2. It is observed that LSTech_VBBsfrc_v1 scored 312, which is one instance more than the number of instances solved by Kissat_MAB, the winner of the 2021 SAT competition. According to Table 4 and Figure 4a,b, Maple_VBBsfr_v2 showed improvements on both satisfiable and unsatisfiable benchmarks, while LSTech_VBBsfrc_v1 showed improvements on satisfiable benchmarks.

6. Discussion

Backbones and backdoors have been introduced for over two decades, but the current state of the art of CDCL SAT solvers [17,49] shows limited evidence of their use to improve CDCL SAT solvers. Among the few is Dequen and Dubois’ work [24]. The backbone-guided decision heuristic they proposed was heavily dependent on unit propagation, which consequently added a significant overhead to the solver performance. It should be noted that early in our work, we have implemented the backbone guided decision heuristic [24] on top of MapleLCMDistChronoBT-DL-v3 to compute backbones; however, the solver performed poorly and never beat the base solver. In addition, it was demonstrated in [19,50] that it is difficult to build a solver that detects backdoors due to the difficulty of computing them. In this regard, we have proposed the BBLO and BDLO heuristics for computing backbones and backdoors. As far as backdoors, they are calculated under the assumption of no overlap between backbones and backdoors, which is only an experimental observation.

According to the performance evaluation in Section 5, both variants Maple_VBBsfr_v2 and LSTech_VBBsfrc_v1 have outperformed the base solvers on solving satisfiable benchmarks, that is, instances with backbones. Therefore, improvements in VSIDS have led to improved performance for both variants in solving more satisfiable instances. It is therefore evident that VSIDS is not only superior at solving unsatisfiable instances [51] but could even be improved to solve satisfiable instances as well.

Further observations include the improved performance of Maple_VBBsfr_v2 with both satisfiable and unsatisfiable benchmarks, while LSTech_VBBsfrc_v1 only improved with satisfiable benchmarks. Evidence suggests that the BBLO heuristic could assist the CCAr solver in finding an earlier solution by identifying promising partial assignments.

7. Conclusion and Future Work

In this study, we performed a set of modifications to the VSIDS decision heuristic to exploit backbones and backdoors. To guide the CDCL SAT solver to branch on backbone/backdoor variables, we proposed low-overhead heuristics to compute them. We implemented 15 variants on top of two popular state-of-the-art award-winning CDCL solvers, MapleLCMDistChronoBT-DL-v3 and LSTech. The variants were evaluated on 32 industrial families from 2002 to 2021 SAT competitions. Results showed that bumping the backbone and backdoor variables during the branching heuristic improved both solvers’ performance. In particular, the variant of LSTech that augmented the VSIDS decision heuristic to exploit the backbone size and frequency and bumped during restarts and conflicts solved more industrial instances compared to the best state-of-the-art CDCL solvers. In the future, we would like to assess the accuracy of the proposed backbone and backdoor heuristics and to investigate the possibility of any improvements. A further research goal would be to modify other decision heuristics, such as LRB and CHB. Moreover, we would like to conduct a larger study to include broader bases of CDCL SAT solvers and benchmarks.