All the experiments were run on a Linux system with a CPU running at 2.53 GHz, 60 min time limit and 2 GB memory limit. The programs were restricted to run on one core of the CPU by using the Linux command taskset. MCMC-Flat^{1/2} and MCMC-Flat^{1/t-pv} were run 10 times on each problem; their results were averaged on successful runs. In the following results, Vol. is short for volume, SR is short for success ratio, and RE is short for relative error. Let d′ be an estimation of a target value d, we define the relative error d to d′ as RE(d, d′) = (d′−d)/d. We use MO to indicate the memory out of the Exact Method and use “\” to indicate ‘time out’.

With structural instances, we want to see that whether the accuracies of MCMC-Flat methods improve with increasing size of volumes. The detailed results are shown in

Table 2. On instance

s1, the relative errors of MCMC-Flat

^{1/2} and MCMC-Flat

^{1/t-pv} are 302.69% and 297.97%, respectively. The errors of the two algorithms decrease consistently when the volumes of the target convex bodies increase. In addition, on the structured instance

s6 which is with the maximal volume 4096, both of our algorithms reach small relative errors within 10% (9.33% and 5.65% for MCMC-Flat

^{1/2} and MCMC-Flat

^{1/t-pv}, respectively). We can also see that MCMC-Flat

^{1/t−pv} leads to smaller relative errors, compared with MCMC-Flat

^{1/2}, on instances

s5,

s6 and

s7, though the running times of the former are more than the latter. Finally, we note that the Exact Method is the fastest among these three algorithms, with nearly neglectable running times. However, this advantage of the Exact Method will be lost on complex SMT(LIA) formulas contained in our random instances.

With random instances, the ability of MCMC-Flat methods on hard instances is verified (see

Table 3).

Table 3 shows that MCMC-Flat

^{1/t-pv} reaches good accuracy very fast, in contrast with the running time of MCMC-Flat

^{1/2}. For example, MCMC-Flat

^{1/2} takes more than 2000 s on instances

ran2,

ran5, and

ran9, whereas MCMC-Flat

^{1/t-pv} takes at most 205.18 s for

ran9. Note that the relative errors of MCMC-Flat

^{1/2} and MCMC-Flat

^{1/t-pv} on

ran2,

ran5, and

ran9 are close. It can also be seen that the volume estimations of MCMC-Flat

^{1/t-pv} on instances

ran2,

ran4,

ran7, and

ran10 are close to those of MCMC-Flat

^{1/2}. On all the random instances tested, the running times of MCMC-Flat

^{1/t-pv} varies from about ten seconds to a few hundreds of seconds, whereas those of MCMC-Flat

^{1/2} varies from 7.59 to 3266.07 s. Therefore, MCMC-Flat

^{1/t-pv} is more scalable than MCMC-Flat

^{1/2}. Next, let us look at the results of the Exact Method. The Exact Method runs out of memory on

ran2,

ran4,

ran7,

ran10 and

ran11, which is of dimension 15, 10, 8, 8, and 9 respectively (please refer to

Table 1). Seeing that the Exact Method succeeds on other instances with less than 8 dimensions, we can conclude that the Exact Method does not scale well. Overall, we see that MCMC-Flat

^{1/t-pv} has a good scalability and outperforms MCMC-Flat

^{1/2} and the Exact Method.

Another note is that both MCMC-Flat

^{1/2} and MCMC-Flat

^{1/t-pv} return greater than zero estimates on all the random instances, especially on those with 8 or more dimensions. The results indicate the ability of our MCMC-Flat algorithms in finding at least one solution. In contrast, the direct Monte Carlo method implemented in [

24] failed to find any solution for instances with dimensions greater than 7, which results in a relative error of 100% (please refer to Figure 10b in [

24]). We argue that this good ability of our MCMC-Flat algorithms is due to both the transition probability in Equation (2) and a mapping from the valuation space (with possibly exponentially number of states) of a SMT(LIA) formula to our defined energy space that is linear in the number of clauses.

#### More Results on Convex Bodies

In the second part of experiments, we focus on a restricted form of STM(LIA), the convex body. Specifically, using the volume computation problem of convex bodies, we compare our MCMC-flat methods with some state-of-the-art model counters. Instances of convex bodies were obtained from the benchmarks (

http://code.google.com/p/rvc/) used in [

24]. All the problems are extracted from the sub-folder named “Exp3_1”. A problem with name

m-

n-

i involves

m variables, and

n + 1 constraints where

i is the index of a variant. Each variable has a value bound [−15, 15]. Every constraint is an inequality, and coefficients of variables are rational numbers represented by fractions. For example, the first inequality of the problem “7-5-1” is:

A brief summary on properties of the selected instances are presented in

Table 4. “#Variable”, “#Constraint”, and “Space”, in

Table 4, stand for the number of variables, the number of LIA constraints, and the number of valuations in the space, respectively. The “Volume/Space” column is the proportion of the number of models to the size of the space, where “NA” represents an unknown property. Intuitively, a problem with lower Volume/Space is more constrained.

We take an exact model counter Volce+LattE [

9] and two approximate model counters SMTApproxMC [

19] and ApproxMC2 [

21] for comparison. Note that all the three are state-of-the-art model counters. To run them, the benchmark problems were converted to their corresponding formats. The problem format in [

24] is similar to that of our methods’ accepts. However, the format is quite different from the formats of Volce+LattE, SMTApproxMC, and ApproxMC2. Volce+LattE accepts an input format called the “Enhanced DIMACS CNF Format”, whereas SMTApproxMC accepts SMT-LIB2 format of Bit Vector theory. In our implementation, we first translate the instances from [

24] into the “Enhanced DIMACS CNF Format” of Volce+LattE. Specifically, denominators of coefficients in a LIA constraint are all multiplied by their least common multiple. For example, the translation result of (7) is,

In this way, every coefficient in an LIA constraint is a number, instead of a fraction. In addition, every value bound for a variable is translated into a corresponding inequality. Moreover, due to that each problem instance representing a convex body, our converter states that all constraints must be satisfied. For SMTApproxMC, equations in the form of (8) are translated into sentences in the Fixed Size Bit Vector theory of SMT-LIB2, where the size of a bit vector is 32.

To run ApproxMC2 [

21], the benchmark problems were bit-blasted by using an embedded function of the SMT solver Yices (

http://yices.csl.sri.com/). Specifically, for the problems described in the SMT-LIB2 Bit Vector theory (which is suited for SMTApproxMC), the SMT-LIB2 keywords are replaced by Yices’ corresponding ones, for example: “bvsle” in SMT-LIB2 is replaced with “bv-sle” in the Yices language. Furthermore, at the end of the problem, an “export-to-dimacs” command is set. The resulted problem file was then bit-blasted by Yices (with command-line option “–logic = QF_BV”). All the original and translated problems for SMTApproxMC and ApproxMC2 are publicly available (

https://github.com/PlanningByCai/MCMCFlat-Count-1-2).

This part of the test was carried out on a 64-bit Linux platform with a 2.6 GHz CPU, 4 GB memory limit. The time limit of each test is 12 h. Volce+LattE was executed once, since it is an exact method. MCMC-Flat

^{1/2} was given 5 runs on each instance and the averaged results are reported. A “\” indicates failure due to time out. SMTApproxMC was run with the default configuration (50 iterations, tolerance

ε = 0.8 and

δ = 0.2, for an instance). Note that these settings for

ε and

δ are the same as those used by the authors of SMTApproxMC, which make the value

v returned by SMTApproxMC satisfy

Note that RF is the real volume. ApproxMC2 was run with its default configuration as well (with the same ε and δ). So, return values of ApproxMC2 satisfy (9) as well.

Results regarding Volce+LattE and MCMC-Flat

^{1/2} are shown in

Table 5. We can observe that Volce+LattE fails on many instances with 6 and 7 variables, among which it only succeeds on 6-5-1, 6-5-2, and 6-5-4 with dozens of minutes. In contrast, our MCMC-Flat

^{1/2} method works on all the instances. On the other hand, MCMC-Flat

^{1/2} reports a result of 1.74 × 10

^{8} on the problem 6-5-1 with a relative error 6.21%. In summary, on the 13 instances on which Volce+LattE succeeds, MCMC-Flat

^{1/2} gives 3 results with RE beyond 20% (52.07% for 5-10-2, 48.17% for 5-20-5, and −25.17% for 6-5-4), and gives the other 10 results with RE in [−20%, 20%]. Therefore, results in

Table 5 show that MCMC-Flat

^{1/2} is more scalable than Volce+LattE.

Table 6 are results of MCMC-Flat

^{1/2}, SMTApproxMC and ApproxMC2; the results of MCMC-Flat

^{1/2} are the same as those in

Table 5. In

Table 6, a brief summary of the results is presented at the bottom: “#Solved” is the number of solved instances of an algorithm, “#BestRE” is the number of instances on which an algorithm obtained the best relative error, and “#AvgTime” is the averaged solving time over instances that are solved by all the three algorithms.

We make several observations from the results in

Table 6, as follows.

In terms of scalability, our MCMC-Flat

^{1/2} method is superior to SMTApproxMC and ApproxMC2. In [

9], the word-level hash-mapping-based method SMTApproxMC was shown to handle instances with less than 3.98 × 10

^{6} models (around 4,000,000) and failed on others. On our test platform, SMTApproxMC got a result of 1.04 × 10

^{7} on instance 6-5-5. However, SMTApproxMC can return a value 15 out of 22 instances, whereas MCMC-Flat

^{1/2} returns values on all the 22 instances. Notably, MCMC-Flat

^{1/2} reports a result of 1.74 × 10

^{8} on the problem 6-5-1 with a low relative error 6.21%. From the results, the scalability of SMTApproxMC is better than ApproxMC2, as the former solved 15 instances whereas the latter solved 11 instances. Therefore, MCMC-Flat

^{1/2} is the best, with respect to scalability.

In terms of accuracy, MCMC-Flat

^{1/2} is better than SMTApproxMC and ApproxMC2. Among 13 instances for which the exact volume is known by Volce+LattE (see

Table 5), MCMC-Flat

^{1/2} returns best results on 8 instances, and ApproxMC2 does the best on the other 5 instances, in terms of relative error. The accuracy of SMTApproxMC is not competitive to either MCMC-Flat

^{1/2} or ApproxMC2. The variance in relative errors of ApproxMC2 is better than that of MCMC-Flat

^{1/2}, as we can observe from

Table 6 that on instances where ApproxMC2 returns none-zero results the relative error of ApproxMC2 is better than that of MCMC-Flat

^{1/2}. On the other hand, ApproxMC2 seems to have big difficulty in working on instances having volume greater than 1.0 × 10

^{6}, as it takes relatively long time on instances 5-10-1, 6-10-1, and 7-10-1.

In terms of efficiency, MCMC-Flat

^{1/2} is significantly better than SMTApproxMC and ApproxMC2. As can be seen from the bottom row in

Table 6, over the ten instances (5-10-1, 5-10-2, 5-10-4, 5-10-5, 5-20-1, 5-20-2, 5-20-3, 5-20-4, 5-20-5, 6-10-1) on which all the three methods return none-zero results, the averaged time costs of MCMC-Flat

^{1/2}, SMTApproxMC, and ApproxMC2 are 344.5, 955.80, and 463.33, respectively. Obviously, MCMC-Flat

^{1/2} is the fastest.

Regarding the ability of obtaining samples from the solution area for large size instances, MCMC-Flat

^{1/2} is superior to the direct Mont Carlo method in [

24]. Note that, all the test instances are took from [

24], and the direct Mont Carlo method in [

24] is reported to give results of RE of 100% on all instances with 7 variables (We failed to run that method due to that its call on Matlab 2010b leads to segment faults). That is, the direct Mont Carlo method failed to sample any solution when the problem size is large. In contrast, MCMC-Flat

^{1/2} is able to reach the solution area on all the tested instances with 7 variables.

A final note for this part of experiments is that MCMC-Flat^{1/t-pv} is not good at these instances. Actually, we also run several tests for MCMC-Flat^{1/t-pv} on instances with 5 variables. However, its success rate is very low. The underlying reason is to be explored in our future work.