A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic Complexity

We investigate the properties of a Block Decomposition Method (BDM), which extends the power of a Coding Theorem Method (CTM) that approximates local estimations of algorithmic complexity based on Solomonoff–Levin’s theory of algorithmic probability providing a closer connection to algorithmic complexity than previous attempts based on statistical regularities such as popular lossless compression schemes. The strategy behind BDM is to find small computer programs that produce the components of a larger, decomposed object. The set of short computer programs can then be artfully arranged in sequence so as to produce the original object. We show that the method provides efficient estimations of algorithmic complexity but that it performs like Shannon entropy when it loses accuracy. We estimate errors and study the behaviour of BDM for different boundary conditions, all of which are compared and assessed in detail. The measure may be adapted for use with more multi-dimensional objects than strings, objects such as arrays and tensors. To test the measure we demonstrate the power of CTM on low algorithmic-randomness objects that are assigned maximal entropy (e.g., π) but whose numerical approximations are closer to the theoretical low algorithmic-randomness expectation. We also test the measure on larger objects including dual, isomorphic and cospectral graphs for which we know that algorithmic randomness is low. We also release implementations of the methods in most major programming languages—Wolfram Language (Mathematica), Matlab, R, Perl, Python, Pascal, C++, and Haskell—and an online algorithmic complexity calculator.


I. MEASURING COMPLEXITY
Capturing the 'complexity' of an object, for purposes such as classification and object profiling, is one of the most fundamental challenges in science.This is so because one has to either choose a computable measure (e.g.Shannon entropy) that is not invariant to object descriptions and lacks an invariance theoremwhich forces one to decide on a particular feature shared among several objects of interest-or else estimate values of an uncomputable function in applying a 'universal' measure of complexity that is invariant to object description (such as algorithmic complexity).
Here we study a measure that lies half-way between two universally used measures that enables the action of both at different scales by dividing data into smaller pieces for which the halting problem involved in an uncomputable function can be partially circumvented, in exchange for a huge calculation based upon the concept of algorithmic probability.The calculation can, however, be precomputed and hence reused in future applications, thereby constituting a strategy for efficient estimations-bounded by Shannon entropy and by algorithmic (Kolmogorov-Chaitin) complexity-in exchange for a loss of accuracy.
In the past, lossless compression algorithms have dominated the landscape of applications of algorithmic complexity.When researchers have chosen to use lossless compression algorithms for reasonably long strings, the method has proven to be of value (e.g.[26]).Their successful application has to do with the fact that compression is a sufficient test for non-algorithmic randomness (though the converse is not true).However, implementations of lossless compression algorithms are based upon estimations of entropy rate, and are therefore no more closely related to algorithmic complexity than is Shannon entropy by itself.They can only account for statistical regularities and not for algorithmic ones, though accounting for algorithmic regularities ought to be crucial, since these regularities represent the main advantage of using algorithmic complexity.
One of the main difficulties with computable measures of complexity such as Shannon entropy is that they are not robust enough [32], [35].For example, they are not invariant to different descriptions of the same object-unlike algorithmic complexity, where the so-called invariance theorem guarantees the invariance of an object's algorithmic complexity.This is due to the fact that one can always translate a lossless description into any other lossless description simply with a program of a fixed length, hence in effect just adding a constant.Computability theorists are not much concerned with the relatively negligible differences between evaluations of Kolmogorov complexity which are owed to the use of different descriptive frameworks (e.g.different programming languages), yet these differences are fundamental in applications of algorithmic complexity.
Here we study a Block Decomposition Method (BDM) that is meant to extend the power of the so-called Coding Theorem Method (CTM).CTM has been used to numerically estimate and apply Kolmogorov Complexity to graph theory [31], image classification [19] and visual cognition [15]- [17].In these applications, other complexity measures, including entropy and lossless compressibility, have been outperformed by CTM.
After introducing the basics of algorithmic complexity, algorithmic probability, Shannon entropy and other entropic measures, and after exploring the limitations and abuse of the use of lossless compression to aproximate algorithmic complexity, we introduce CTM on which BDM heavily relies upon.After introducing BDM, we thoroughly study its properties and parameter dependency on size, the problem of the boundaries in the decomposition process, we prove theoretical bounds and provide numerical estimations after testing on actual data (graphs) followed by error estimations for the numerical estimations by BDM.We also introduce a normalized version of BDM and a logical depth measure based on BDM.In the Sup.Inf.we also provide tools, and information about an online algorithmic complexity calculator free to use, and several implementations in major computer programming languages.

A. Algorithmic Complexity
The Coding Theorem Method was first introduced as a method for dealing with the problem of compressing very short strings, for which no implementation of lossless compression gives reasonable results.CTM exploits the elegant and powerful relationship between the algorithmic frequency of production of a string and its algorithmic complexity [22].
The algorithmic complexity [7], [18] K(s) of a string s is the length of the shortest program p that outputs the string s, when running on a universal (prefix-free 1 ) Turing machine U .
A technical inconvenience of K as a function taking s to be the length of the shortest program that produces s, is that K is lower semi-computable.In other words, there is no effective algorithm which takes a string s as input and produces the integer K(s) as output.This is usually considered a major problem, but the theory of algorithmic randomness [13] ascribes uncomputability to any universal measure of complexity, that is, a measure that is at least capable of characterizing mathematical randomness [24].However, because it is lower semi-computable, K(s) can be approximated from above, or in other words, upper bounds can be found, for example, by finding and exhibiting a small computer program (measured in bits) relative to the length of a bit string.

B. Algorithmic Probability
The classical probability of production of a bit string s among all possible 2 n bit strings of length n is given by P r(s) = 1/2 n .The concept of algorithmic probability (also known as Levin's semi-measure) replaces the random production of outputs by the random production of programs that produce an output.The algorithmic probability of a string s is thus a measure that estimates the probability of a random program p producing a string s when run on a universal (prefix-free) Turing machine U .
The algorithmic probability m(s) of a binary string s is the sum over all the (prefix-free) programs p for which a universal Turing machine U running p outputs s and halts [7], [22], [30].It replaces n (the length of s) with |p|, the length of the program p that produces s: m(s) can be considered an approximation to K(s), because the greatest contributor to m(s) is the shortest program p that generates s using U .So if s is of low algorithmic complexity, 1 The group of valid programs forms a prefix-free set (no element is a prefix of any other, a property necessary to keep 0 < m(s) < 1).For details see [5], [23].
then |p| < n, and will be considered random if |p| ∼ n.
where c is a fixed constant, independent of s.
The Coding Theorem implies [5], [8] that the output frequency distribution of random computer programs to approximate m(s) can be clearly converted into estimations of K(s) using the following rewritten version of Eq. ( 2):

C. Convergence Rate and the Invariance Theorem
One fundamental property that provides the theory of algorithmic complexity with the necessary robustness to stand as a universal measure of (random) complexity is the so-called Invariance theorem [22], [30], which guarantees the convergence of values despite the use of different reference universal Turing machines or programming languages.
Theorem I.2.Invariance theorem [7], [18], [30]: If U 1 and U 2 are two UTMs and K U 1 (s) and K U 2 (s) the algorithmic complexity of s for U 1 and U 2 , there exists a constant c such that: where c U 1 ,U 2 is independent of s and can be considered to be the length (in bits) of a translating function between universal Turing machines U 1 and U 2 , or as a compiler between computer programming languages U 1 and U 1 .
In practice, however, the constant involved can be arbitrarily large, and the invariance theorem tells us nothing about the convergence (see Fig. 1).One may perform the calculation of K(s) for a growing sequence s under U 1 in the expectation that for long s, K U 1 (s) = K(s).However, there is no guarantee that this will be the case, and the size of c U 1 ,U 2 in general is unknown.
It is a question whether there can be a natural universal Turing machine U N such that K U N (s) converges faster for s than for any other universal Turing machine (UTM), or whether specific conditions must be met if U 1 is to generate "well-behaved" (monotonic) behaviour in c [28].
The Invariance theorem guarantees that such optimal 'well-behaved' machines U N always existindeed their existence is implicit in the very sense of the theorem (meaning any universal machine can be optimal in the sense of the theorem)-but it tells nothing about the rate of convergence or about transitional behaviour (see Fig. 1 for illustration).The longer the string, the less important c is (i.e. the choice of programming language or UTM).However, in practice c can be arbitrarily large, thus having a great impact, particularly on short strings, and never revealing at which point one starts approaching a stable K or when one is diverging before finally monotonically converging, as is seen in the different possible behaviours illustrated in the sketches in Fig. 1.

II. THE USE AND MISUSE OF LOSSLESS COMPRESSION
Lossless compression algorithms have traditionally been used to approximate the Kolmogorov complexity of an object (e.g. a string).Data compression can be viewed as a function that maps data onto other data using the same units or alphabet (if the translation is into different units or a larger or smaller alphabet, then the process is called an encoding).Compression is successful if the resulting data are shorter than the original data plus the decompression instructions needed to fully reconstruct said original data.For a compression algorithm to be lossless, there must be a reverse mapping from compressed data to the original data.That is to say, the compression method must encapsulate a bijection between "plain" and "compressed" data, because the original data and the compressed data should be in the same units.By a simple counting argument, lossless data compression algorithms cannot guarantee compression for all input data sets, because there will be some inputs that do not get smaller when processed by the compression algorithm, and for any lossless data compression algorithm that makes at least one file smaller, there will be at least one file that it makes larger.Strings of data of length N or shorter are clearly a strict superset of the sequences of length N − 1 or shorter.It follows therefore that there are more data strings of length N or shorter than there are data strings of length N − 1 or shorter.And it follows from the pigeonhole principle that it is not possible to map every sequence of length N or shorter to a unique sequence of length N − 1 or shorter.
Therefore there is no single algorithm that reduces the size of all data.
One of the more time consuming steps of implementations of, for example, LZ77 compression (one of the most popular) is the search for the longest string match.Most lossless compression implementations are based upon the LZ algorithm.The classical LZ77 and LZ78 algorithms enact a greedy parsing of the input data.That is, at each step, they take the longest dictionary phrase which is a prefix of the currently unparsed string suffix.LZ algorithms are said to be 'universal' because, assuming unbounded memory (arbitrary sliding window length), they asymptotically approximate the entropy rate of the generating source [21].Not only does lossless compression fail to provide any estimation of the algorithmic complexity of small objects [12], [28], it is also not more closely related to algorithmic complexity than Shannon entropy by itself, being only capable of exploiting statistical regularities.
Lossless compression is also subject to the constant involved in the invariance theorem, because there is no reason to choose one compression algorithm over another.
The greatest limitation of the most popular lossless compression algorithms, in the light of algorithmic complexity, is that their implementations only exploit statistical regularities (repetitions up to the size of the sliding window length).Thus in effect no general lossless compression algorithm does better than provide the Shannon entropy rate (c.f.Section II-A) of the objects it compresses.It is then obvious that an exploration of other possible methods for approximating K is not only desirable but needed, especially methods that can, at least in principle, and more crucially in practice, detect some truly algorithmic content.

A. Building upon Entropy Rate
The entropy of a discrete random variable s with possible values s 1 , . . ., s n and probability distribution P (s) as: Definition II.1.
In the case of P (s i ) = 0 for some i, the value of the corresponding summand 0 log 2 (0) is taken to be 0.
It is natural to ask how random a string appears when blocks of finite length are considered.
That is, how do finite-L approximations of entropy converges.
For example, the string 01010101 . . .01 is periodic, but for the smallest granularity (1 bit) or 1-symbol block, the sequence has maximal entropy, because the number of 0s and 1s is the same assuming a uniform probability distribution for all strings of the same finite length.Only for longer blocks of length 2 bits can the string be found to be regular, identifying the smallest Entropy value for which the granularity is at its minimum.
The following function R gives what is variously denominated as rate or block entropy and is Shannon entropy over blocks or subsequences of s of length l.That is, To determine the entropy rate of a finite sequence s, one estimates the limit when l → |s|.
If the sequence is not statistically random, then H R (s) will reach a low value for some l, and if random, then it will be maximally entropic for all blocks l.H R (s) is computationally intractable as a function of l, and typically upper bounds are realistically calculated for a fixed value of l (e.g. a window length).
If the sequence is not statistically random, then H R (s) reaches a low value for some l, and if random, then it will be maximally entropic for all blocks l.H R (s) is computationally intractable as a function of sequence size, and typically upper bounds are realistically calculated for a fixed value of l (e.g. a window/block length/size).
BDM is based upon a similar decomposition approach based in blocks but combining the strengths (and limitations) of both Shannon entropy and algorithmic complexity with the advantage that upon improvements of estimations of algorithmic complexity BDM can incorporate them in a different way than updating the underlying probability distribution in the application of Shannon entropy alone.We first introduce the algorithm upon which BDM is based for the local estimations of algorithmic complexity.

III. THE CODING THEOREM METHOD (CTM)
A computationally expensive procedure that is nevertheless closely related to algorithmic complexity involves approximating the algorithmic complexity of an object by running every possible program, from the shortest to the longest, and counting the number of times that a program produces every string object.The length of the computer program will be an upper bound of the algorithmic complexity of the object, following the Coding theorem 3, and a (potentially) compressed version of the data itself (the shortest program found) for a given computer language or 'reference' UTM.This guarantees discovery of the shortest program in the given language or reference UTM but entails an exhaustive search over the set of countable infinite computer programs that can be written in such a language.A program of length n has asymptotic probability close to 1 of halting in time 2 n [6], making this procedure exponentially expensive, even assuming that all programs halt or that programs are assumed never to halt after a specified time, with those that do not being discarded.
As shown in [12] and [28], an exhaustive search can be carried out for a small-enough number of computer programs (more specifically, Turing machines) for which the halting problem is known, thanks to the Busy Beaver game [25].This is the challenge of finding the Turing machine of fixed size that runs for longer than any other machine of the same size, and for which values are known for Turing machines with 2 symbols and up to 4 states.One can also proceed with an informed runtime cut-off, well below the theoretical 2 n optimal runtime that guarantees an asymptotic drop of non-halting machines [6].[25]), but some shortcuts and optimizations are possible [12], [28].
CTM is rooted in the relation [12], [28] provided by algorithmic probability between frequency of production of a string from a random program and its algorithmic complexity as described by Eq. ( 3).Essentially it uses the fact that the more frequent a string is, the lower Kolmogorov complexity it has; and strings of lower frequency have higher Kolmogorov complexity.The advantage of using algorithmic probability to approximate K by application of the Coding Theorem3 is that m(s) produces reasonable approximations to K based on an average frequency of production, which retrieves values even for small objects.
Let (n, k) denote the set of all Turing machines with n states and k symbols using the Busy Beaver formalism [25], and let T be a Turing machine in (n, k) with empty input.Then the empirical output distribution D(n, k) for a sequence s produced by some T ∈ (n, k) gives an estimation of the algorithmic probability of s, D(n, k)(s) defined by: Definition III.1.
For small values n and k, D(n, k) is computable for values of the Busy Beaver problem that are known.The Busy Beaver problem [25] is the problem of finding the n-state, k-symbol Turing machine which writes a maximum number of non-blank symbols before halting, starting from an empty tape, or the Turing machine that performs a maximum number of steps before halting, having started on an initially blank tape.For n = 4 and k = 2, for example, the Busy Beaver machine has maximum runtime S(n) = 107 [4], from which one can deduce that if a Turing machine with 4 states and 2 symbols running on a blank tape hasn't halted after 107 steps, then it will never halt.This is how D was initially calculated-by using known Busy Beaver values.
However, because of the undecidability of the Halting problem, the Busy Beaver problem is only computable for small n, k values [25].Nevertheless, one can continue approximating D for a greater number of states (and colours), proceeding by sampling, as described in [12], [28], with an informed runtime based on both theoretical and numerical results.
Notice that 0 < D(n, k)(s) < 1, D(n, k)(s) and is thus said to be a semi-measure, just as m(s) is.
Now we can introduce a measure of complexity that is heavily reliant upon algorithmic probability m(s), as follows: Definition III.2.Let (n, k) be the space of all n-state k-symbol Turing machines, n, k > 1 and D(n, k)(s) = the function assigned to every finite binary string s.Then: where b is the number of symbols in the alphabet (traditionally 2 for binary objects, which we will take as understood hereafter).
That is, the more frequently a string is produced the lower its Kolmogorov complexity, with the converse also being true.
Table I shows the rule spaces of Turing machines that were explored, from which empirical algorithmic probability distributions were sampled and estimated.
Definition III.3.We will designate as base string, base matrix, or base tensor the objects of size d for which CTM values were calculated such that the full set of n d objects have CTM evaluations.In other words, the base object is the maximum granularity of application of CTM.
Table I provides figures relating to the number of base objects calculated.
Validations of CTM undertaken before show the correspondence between CTM values and the exact number of instructions used by Turing machines when running to calculate CTM [14](Fig  model of computation [25].The correspondence in values found between the directly calculated K and CTM by way of frequency production was near perfect.Delahaye [12].The section 'Robustness' in Soler-Toscano et al. [28] provides evidence relating to the behaviour of the invariance theorem constant for a standard model of Turing machines [25].

IV. THE BLOCK DECOMPOSITION METHOD (BDM)
Because finding the program that reproduces a large object is computationally very expensive and ultimately uncomputable, one can aim at finding smaller programs that reproduce smaller parts of the original object, parts that together compose the larger object.And this is what the BDM does.It decomposes the original data into fragments, the Kolmogorov complexity of which we have good approximations of.The sum of the programs that reconstruct the original data is an approximation of its Kolmogorov complexity.This also means that the method is local.BDM cannot, however, work alone.It requires a CTM, described in the next section.In Section 8 we study a method to extend its range of application, thereby effectively extending its power by combining it with Shannon entropy.
The BDM is a hybrid complexity measure that calculates global entropic and local algorithmic complexity.It is meant to extend the power of CTM, and consists in decomposing objects into smaller pieces for which exact complexity approximations have been numerically estimated using CTM, then reconstructing an approximation of the Kolmogorov complexity for the larger object by adding the complexity of the individual components of the object, according to the rules of information theory.For example, if s is an object and 10s is a repetition of s ten times smaller, upper bounds can be achieved by approximating K(s) + log 2 (10) rather than K(10s), because we know that repetitions have a very low Kolmogorov complexity, given that one can describe repetitions with a short algorithm.
Here we introduce and study the properties of this Block Decomposition Method based on a method advanced in [12], [28] that takes advantage of the powerful relationship established by algorithmic probability between the frequency of a string produced by a random program running on a (prefix-free) UTM and the string's Kolmogorov complexity.The chief advantage of this method is that it deals with small objects with ease, and it has shown stability in the face of changes of formalism, producing reasonable Kolmogorov complexity approximations.BDM must be combined with CTM if it is to scale up properly and behave optimally for upper bounded estimations of K. BDM + CTM is universal in the sense that it is guaranteed to converge to K due to the invariance theorem, and as we will prove later, if CTM no longer runs, then BDM alone approximates Shannon entropy.
Like compression algorithms, BDM is subject to a trade-off.Compression algorithms deal with the trade-off of compression power and compression/decompression speed.

A. l-overlapping String Block Decomposition
Let us fix values for n and k and let D(n, k) be the frequency distribution constructed from running all the Turing machines with n states and k symbols.Following Eq. ( 5), we have it that D is an approximation of K (denoted by CT M ).We define the BDM of a string or finite sequence s as follows, Definition IV.1.
where x i is the multiplicity of s i , and s i the subsequence i after decomposition of s into subsequences s i , each of length l, with a possible remainder sequence y < |l| if |X| is not a multiple of the decomposition length l.
The parameter m goes from 1 to the maximum string length produced by CTM, where m = l means no overlapping inducing a partition of the decomposition of X, m is thus an overlapping parameter when m < l for which we will investigate its impact on BDM (in general, the smaller m a greater overestimation of BDM).
The parameter m is needed because of the remainder.If |s| is not a multiple of the decomposition length l then the option is to either ignore the remainder in the calculation of BDM or define a sliding window with overlapping m − l.
The choice of n and k for CTM in BDM depend only on the available resources for running CTM, which involves running the entire (n, k) space of Turing machines with n symbols and k states.
The definition of BDM is interesting because one can plug in other algorithmic distributions, even computable ones, approximating some measure of algorithmic complexity even if it is not the one defined by Kolmogorov-Chaitin such as, for example, the one defined by Calude et al. [5] based upon Finite-state automata.BDM thus allows the combination of measures of classical informaiton theory and algorithmic complexity.

B. 2-and w-Dimensional Complexity
To ask after the likelihood of an array, we can consider a 2-dimensional Turing machine.The Block Decomposition Method can then be extended to objects beyond the unidimensionality of strings, e.g.arrays representing bitmaps such as images, or graphs (by way of their adjacency matrices).We would first need CTM values for 2-and w-dimensional objects that we call base objects (e.g.base strings or base matrices).
A popular example of a 2-dimensional tape Turing machine is Langton's ant [20].Another way to see this approach is to take the BDM as a way of deploying all possible 2-dimensional deterministic Turing machines of a small size in order to reconstruct the adjacency matrix of a graph from scratch (or smaller pieces that fully reconstruct it).Then, as with the Coding theorem method (above), the algorithmic complexity of the adjacency matrix of the graph can be estimated via the frequency with which it is produced from running random programs on the (prefix-free) 2-dimensional Turing machine.More specifically, where set Adj(X) {x i } is composed of the pairs (r, n), r is an element of the decomposition of M (as specified by a partition {x i }) in different sub-arrays of size up to d 1 × . . .× d w (where w is the dimension of the object) that we call base matrix (because CT M values were obtained for them) and s is the multiplicity of each component.CT M (r) is a computable approximation from below to the algorithmic information complexity of r, K(r), as obtained by applying the coding theorem method to w-dimensional Turing machines.In other words, {x i } is the set of base objects.
Because string block decomposition is a special case of matrix block decomposition, and square matrix block decomposition is a special case of w-block decomposition for objects of w dimensions, let us describe the way in which BDM deals with boundaries on square matrices, for which we can assume CTM values are known, and that we call base strings or base matrices.Applications to graph theory [31], image classification [15] and human behavioural complexity have been produced in the last few years [16], [17].

C. BDM Upper and Lower Absolute Bounds
In what follows we show the hybrid nature of the measure.We do this by setting lower and upper bounds to BDM in terms of the algorithmic complexity K(X), the partition size and the approximation error of CT M , such that these bounds are tighter in direct relation to smaller partitions and more accurate approximations of K.These bounds are independent of the partition strategy defined by {x i }.
Proposition IV.1.Let BDM be the function defined in Eq. 8 and let X be an array of dimension w.Then K(X) ≤ BDM (X, , where A is a set composed of all possible ways of accommodating the elements of Adj(X) {x i } in an array of dimension w, and is the sum of errors for the approximation CT M over all the sub-arrays used.
Let Adj(X) {x i } = {(r 1 , n 1 ), ..., (r k , n k )} and {p j }, {t j } be the sequences of programs for the reference prefix-free UTM U such that, for each (r j , n j ) ∈ Adj(X) {x i } , we have U (p j ) = r j , Let j be a positive constant such that CT M (r j ) + j = K(X); this is the error for each sub-array.Let be the sum of all the errors.
For the first inequality we can construct a program q w , whose description only depends on w, such that, given a description of the set Adj(X) {b i } and an index l, it enumerates all the ways of accommodating the elements in the set and returns the array corresponding to the position given by l.
Note that |l|, |Adj(X) {b i } | and all n j 's are of the order of log|A|.Therefore U (q w q 1 p 1 t 1 ...p j t j l) = X and which gives us the inequality.Now, let q X be the smallest program that generates X.For the second inequality we can describe a program q {x i } which, given a description of M and the index j, constructs the set Adj(X) {x i } and returns r j , i.e.U (q {x i } q X j) = r j .Note that each |j| is of the order of log |Adj(X) {x i } |.Therefore, for each j we have K(r j ) Finally, by adding all the terms over the j's we find the second inequality: Corollary IV.2.If the partition defined by Given the inequalities presented in proposition IV.1, we have it that and From [28], we can say that the error rate is small, and that by the invariance theorem it will converge towards a constant value.

V. DEALING WITH OBJECT BOUNDARIES
Because partitioning an object-a string, array or tensor-leads to boundary leftovers not multiple of the partition length, the only two options to take into consideration such boundaries in the estimation of the algorithmic complexity of the entire object is to either estimate the complexity of the leftovers or to define a sliding window allowing overlapping in order to include the leftovers in some of the block partitions.The former implies mixing object dimensions that may be incompatible (e.g.CTM complexity based on 1-dimensional TMs versus CTM based on higher dimensional TMs).Here we explore these strategies to deal with the object boundaries.
Here we introduce a strategy for partition minimization and base object size maximization that we will illustrate for 2-dimensionality.The strategies are intended to overcome under-or overfitting complexity estimations that are due to conventions, not just technical limitations (due to, e.g., uncomputability and intractability).
In Section IV-C, we have shown that using smaller partitions for BDM yields more accurate approximations to the algorithmic complexity K.However, the computational costs for calculating CT M are high.We have compiled an exhaustive database for square matrices of size up to 4 × 4. Therefore it is in our best interest to find a method to minimize the partition We define a function part(X, d, q), where d is a natural number and q 1 , . . ., q d ∈ quad d , as follows: where max(X, q, d) is the largest set of adjacent submatrices of size d × d that can be clustered in the corner corresponding to the quadrant q i , resR(X, d − 1, q i ) is the submatrix composed of all the adjacent rightmost cells that could not fit on max(X, q, d i ) and are not part of the leftmost cells, resL(X, d − 1, q i ) is an analogue for the leftmost cells and resLR(X, d − 1, q) is the submatrix composed of the cells belonging to the rightmost and leftmost cells.We call the last three submatrices residual matrices.
By symmetry, the number of matrices generated by the function is invariant with respect to any vector of quadrants q 1 , . . ., q d .However, the final BDM value can (and will) vary according to the partition chosen.Nevertheless, with this strategy we can evaluate all the possible BDM values for a given partition size and choose the partition that yields the minimum value, the maximum value, or compute the average for all possible partitions.
The partition strategy described can easily be generalized and applied to strings (1 dimension) and tensors (objects of n-dimensions).

B. Periodic Boundary Conditions
One way to avoid having remaining matrices (from strings to tensors) of different sizes is to embed a matrix in a topological torus (see Fig. 4 bottom) such that no more object borders are found.Then let X be a square matrix of arbitrary size m.We screen the matrix M for all possible combinations to minimize the number of partitions maximizing block size.We then take the combination of smallest BDM for fixed base matrix size d and we repeat for d − 1 until we have added all the components of the decomposed X.This procedure, will, however, overestimate the complexity values of all objects (in unequal fashion along the complexity spectra) but will remain bounded, as we will show in Section VI.
Without loss of generality the strategy can be applied to strings (1 dimension) and tensors (any larger number of dimensions, e.g.greater than 2), the former embedded in a cylinder while tensors can be embedded in n-dimensional tori (see Fig. 4).

VI. ERROR ESTIMATION
One can estimate the error in different calculations of BDM, regardless of the error estimations of CTM (quantified in [12], [28]), in order to calculate their departure and deviation both from granular entropy and algorithmic complexity, for which we know lower and upper bounds.For example, a maximum upper bound for binary strings is the length of the strings themselves.This is because no string can have an algorithmic complexity greater than its length, simply because the shortest computer program (in bits) to produce the string may be the string itself.
In the calculation of BDM, when an object's size is not a multiple of the base object of size d, boundaries of size < d will be produced, and there are various ways of dealing with them to arrive at a more accurate calculation of an object that is not a multiple of the base.First we will estimate the error introduced by ignoring the boundaries or dealing with them in various ways, and then we will offer alternatives to take into consideration in the final estimation of their complexity.• Full overlapping recursive decomposition: X is decomposed into (d − 1) 2 base matrices of size d × d by traversing X with a sliding square block of size d.This will produce a polynomial overestimation in the size of the object of up to (d − 1) 2 , but if consistently applied it will for the most part preserve ranking.
• Adding low complexity rows and columns (we call this 'add col'): If a matrix of interest is not multiple the base matrices, we add rows and columns until completion to the next multiple of the base matrix, then we correct the final result by substracting the borders that were artificially added.
The BDM error rate (see 4 top) is the discrepancy of the sum of the complexity of the missed borders, which is an additive value of, at most, polynomial growth.The error is not even for a different complexity.For a tensor of d dimensions, with all 1s as entries, the error is bounded by log(k d ) for objects with low algorithmic randomness and by k d d k for objects with high algorithmic randomness.
Ultimately there is no optimal strategy for making the error disappear, but in some cases the error can be better estimated and corrected VI and all cases are convergent, hence asymptotically negligible, and in all cases complexity ranking is preserved and under-and over-estimations bounded.

A. BDM Convergence towards Shannon entropy
Let {x i } be a partition of X defined as in the previous section for a fixed d.Then the Shannon entropy of X for the partition {x i } is given by: where P r(r j ) = n j |{x i }| and the array r j is taken as a symbol itself.The following proposition establishes the asymptotic relationship between H {x i } and BDM .
Proposition VI.1.Let M be a 2-dimensional matrix and {x i } a partition strategy with elements of maximum size d × d.Then: First we note that n j = |{x i }| and, given that the set of matrices of size d × d is finite, there exists a constant c d such that |Adj(X) {b i } |CT M (n j ) < c d .Therefore: Now, let's recall that the number of n j 's is bounded by c d .Therefore c d exists such that Finally, given that |{x i }| is of the order of |X|, we have the result.

VII. NORMALIZED BDM
A normalized version of BDM is useful for applications in which a maximal value of complexity is known or desired for comparison purposes.The chief advantage of a normalized measure is that it enables a comparison among objects of different sizes, without allowing size to dominate the measure.This will be useful in comparing arrays and objects of different sizes.
First, for a square array of size n × n, we define: where M d ({0, 1}) is the set of binary matrices of size d×d.For any n, M inBDM (n) d×d returns the minimum value of Eq. ( 10) for square matrices of size n, so it is the minimum BDM value  Secondly, for the maximum complexity, Eq. ( 10) returns the highest value when the result of dividing the adjacency matrix into the d × d base matrices contains the highest possible number of different matrices (to increase the sum of the right terms in Eq. ( 10)) and the repetitions (if necessary) are homogeneously distributed along those squares (to increase the sum of the left terms in Eq. ( 10)) which should be the most complex ones in M d ({0, 1}).For n, d ∈ N, we define a function that verifies: The value f n,d (r) indicates the number of occurrences of r ∈ M d ({0, 1}) in the decomposition into d × d squares of the most complex square array of size n × n.Condition Eq. ( 11) establishes that the total number of component squares is n/d 2 .Condition Eq. ( 12) reduces the square repetitions as much as possible, so as to increase the number of differently composed squares as far as possible and distribute them homogeneously.Finally, Eq. ( 13) ensures that the most complex squares are the best represented.Then, we define: Finally, the normalized BDM value of an array M is: This way we take the complexity of an array M to have a normalized value which is not dependent on the size of M but rather on the relative complexity of M with respect to other arrays of the same size.Fig. 5, provides an example of high complexity for illustration purposes.The use of M inBDM (n) d×d in the normalization is relevant.Note that the growth of M inBDM (n) d×d is linear with n, and the growth of M axBDM (n) d×d exponential.This means that for high complexity matrices, the result of normalizing by using just CT M (M )/M axBDM (n) d×d would be similar to N BDM (M ) d .But it would not work for low complexity arrays, as when the complexity of M is close to the minimum, the value of CT M (M )/M axBDM (n) d×d drops exponentially with n.For example, the normalized complexity of an empty array (all 0s) would drop exponentially in size.To avoid this, Eq. ( 14) considers not only the maximum but also the minimum.
Notice the heuristic character of f n,d .It is designed to ensure a quick computation of M axBDM (n) d×d , and the distribution of complexities of squares of size d ∈ {3, 4} in D(5, 2) ensures that M axBDM (n) d×d is actually the maximum complexity of a square matrix of size n, but for other distributions it could work in a different way.For example, condition (12) assumes that the complexities of the elements in M d ({0, 1}) are similar.This is the case for d ∈ {3, 4} in D(5, 2), but it may not be true for other distributions.But at any rate it offers a way of comparing the complexities of different arrays independent of their size.

VIII. BEHAVIOUR OF CTM TO BDM TRANSITION
How BDM scales CTM remains a question, as does the rate at which BDM loses the algorithmic estimations provided by CTM.Also unknown is what the transition between CTM and CTM+BDM looks like, especially in the face of applications involving objects of medium size between the range of application of CTM (e.g. 10 to 20 bit strings) and larger objects (e.g.longer sequences in the hundreds of bits).
We perform a Spearman correlation analysis to test the strength of a monotonic relationship between CTM values and BDM values with various block sizes and block overlap configurations in all 12 bit strings.We also test the strength of this relationship with CTM on Shannon entropy and compression length.
Fig. 6 shows the agreement between BDM and CTM for strings for which we have exact CTM values, against which BDM was tested.The results indicate an agreement between CTM and BDM in a variety of configurations, thereby justifying BDM as an extension of the range of application of CTM to longer strings (and to longer objects in general).
In the set of all 12 bit strings, the correlation is maximal when block size = 11 and overlap = 10 (b11o10, ρ = 0.69); Shannon entropy has ρ = 1 with BDM when strings are divided in blocks of size = 1 and overlap = 0 (b1o0, ρ = 0.42), as is expected from what is described in VI-A.CTM strings indicates that there is a greater difficulty in estimating the algorithmic complexity of highly irregular strings through either BDM, entropy, or compression length than in detecting their regularity.Fig. 6 shows that for block size > 6 the Spearman ρ of BDM is always higher than the correlation of CTM with either Shannon entropy or compression length.Some block configurations of size < 6 (e.g., b2o1) also have higher ρ than both Shannon entropy and compression.
While BDM approximates the descriptive power of CTM and extends it over a larger range, we prove in Subsection IV-C that BDM approximates Shannon entropy if base objects are no longer generated with CTM, but if CTM approximates algorithmic complexity, then BDM does.

A. Smooth BDM (and 'add col')
As an alternative method for increasing accuracy while decreasing computational cost, is the use of a weighted function as penalization parameter in BDM.Let the base matrix size be 4 × 4.
We first partition the matrix into sub matrices of the matrix base size 4 × 4. If the matrix size is not divisible by 4 we (1) use a smooth BDM with full overlap boundary condition (we call this method somply 'smooth' BDM) or (2) add an artificial low complexity boundary to 'complete' the matrix to the next multiple of 4 and apply 'smooth' (we call this approach 'add col' in future sections).
When using the BDM full overlap boundary condition, we screen the entire matrix by moving a sliding square of size 4×4 over it (as it is done for 'recursive BDM').When adding artificial low complexity boundaries we only calculate non overlapping sub-matrices of size 4 × 4 because the expanded matrix of interest is of multiple of 4. These artificial low complexity boundary are columns and rows of one symbols (zeroes or ones).We then correct the final result by subtracting the information added to the boundaries from log(|R|) + log(|C|).
To prevent the undesired introduction of false patterns in the 'completion' process (add col), we use the minimum BDM of the extended matrix for both cases (column and rows of zeroes and ones denoted by BDM 1 (X) and BDM 0 (X) respectively).
In both cases, to distinguish the occurrence of rare and thus highly complex patterns, we assign weights to each base matrix based on the probability of seeing each pattern, denoted by W i , where i is the index of the base matrix.We thereby effectively 'smooth' the transition to decide matrix similarity, unlike the previous versions of BDM which counts multiplicity of equal matrices.Thus the main difference introduced in the 'smooth' version of BDM is the penalization by base matrix (statistical) similarity rather than only perfect base matrix match.
To simplify notation, in what follows let us denote the adjacency matrix Adj(X) of a matrix M simply as M .The smooth version of BDM is then calculated as follows:

B. Weighted Smooth BDM with Mutual Information
The Smooth BDM version assigns a weight to each base matrix depending on its statistical likelihood, which is equivalent to assigning a weight based on the entropy of the base matrix over the distribution of all base matrices of 4 × 4.An equivalent version that is computationally more expensive is the use of classical mutual information.This is arrived at by measuring the statistical similarity between base matrices precomputed by mutual information.
Mutual information is a measure of the statistical dependence of a random variable X on a random variable Y in the joint distribution of X and Y relative to the joint distribution of X and Y under an assumption of independence.If M I(X, Y ) = 0, then X and Y are statistically independent, but if the knowledge of X fully determines Y , M I(X, Y ) = 1, then X and Y are knowing all about Y also implies knowing all about X.In one of its multiple versions MI of X and Y can be defined as: where H(X) is the Shannon entropy of X and H(X|Y ) the conditional Shannon entropy of X given Y .
In this way, statistically similar base matrices are not counted as requiring 2 completely different computer programs, one for each base matrix, but rather a slightly modified computer program producing 2 similar matrices accounting mostly for one and for the statistical difference of the other.More precisely, BDM can be defined by: where MIBDM is defined by: and where M I(B i , B j ) is a weight for each CTM value of each base matrix such that j is the index of the matrix that maximizes M I (or maximizes statistical similarity) over the distribution of all the base matrices such that M I(B i , B j ) ≥ M I(B i , B k ) for all k ∈ {1, . . ., N }.
Notice that because M I is symmetric, then M I(B i , B j ) = M I(B j , B i ), but the min in Eq. 19 is because we look for the minimum CTM value (i.e. the length of the shortest program) for the 2 cases in which one base matrix is the one helping define the statistical similarities of the other and vice versa.

IX. TESTING BDM AND BOUNDARY CONDITION STRATEGIES
A test for both CTM and BDM can be carried out using objects that have different representations or may look very different but are in fact algorithmically related.First we will prove some theorems relating to the algorithmic complexity of dual and cospectral graphs and then we will perform numerical experiments to see if CTM and BDM perform as theoretically expected.
A dual graph of a planar graph G is a graph that has a vertex corresponding to each face of Cospectral graphs, also called isospectral graphs, are graphs that share the same graph spectrum.The set of graph eigenvalues of the adjacency matrix is called the spectrum Spec(G) of the graph G.This cospectrality test for complexity estimations is interesting because two non-isomorphic graphs can share the same spectrum.
We have demonstrated that isomorphic graphs have similar complexity as a function of graph automorphism group size [31].We have also provided definitions for the algorithmic complexity of labelled and unlabelled graphs based on the automorphism group [33].In the Sup.Inf.
we prove several theorems and corollaries showing that dual and cospectral graphs should be expected to have similar algorithmic complexity values, and so we have a theoretical expectation of numerical tests with BDM to compare with.In [31], we showed that BDM behaves in agreement with the theory with respect to the algorithmic complexity of graphs and the size of the automorphism group to which they belong.This is because the algorithmic complexity K(G) of G is effectively a tight upper bound on K(Aut(G)).
In the next Section IX-A, we will see exactly how BDM differs from Shannon entropy, entropy rate, and compression.

A. BDM versus Entropy (Rate) and Compression
Let us address the task of quantifying how many strings with maximum entropy rate are actually algorithmically compressible, i.e., have low algorithmic complexity.That is, how many strings are actually algorithmically (as opposed to simply statistically) compressible but are not compressed by lossless compression algorithms, which are statistical (entropy rate) estimators.
We know that most strings have both maximal entropy (most strings look equally statistically disordered, a fact that constitutes the foundation of thermodynamics) and maximal algorithmic complexity (according to a pigeonhole argument, most binary strings cannot be matched to shorter computer programs as these are also binary strings).But the gap between those with maximal entropy and low algorithmic randomness diverges and is infinite at the limit (for an unbounded string sequence).That is, there is an infinite number of sequences that have maximal entropy but low algorithmic complexity.
The promise of BDM is that, unlike compression, it does identify some cases of strings with maximum entropy that actually have low algorithmic complexity.Fig. 7 shows that indeed BDM assigns lower complexity to more strings than entropy, as expected.Unlike entropy, and implementations of lossless compression algorithms, BDM recognizes some strings that have no statistical regularities but have algorithmic content that makes them algorithmically compressible.

X. A LOGICAL DEPTH-BASED MEASURE
Logical Depth is a measure that distinguishes "structure" from algorithmic randomness (i.e. high algorithmic complexity) through the introduction of computational time.CTM can also provide estimations based upon this logical depth.According to the concept of logical depth [3], the complexity of a string is best defined by the time that an unfolding process takes to reproduce the string from its shortest description.The longer it takes, the more complex.Hence complex objects are those that can be seen as "containing internal evidence of a nontrivial causal history" [3].It is thus more in keeping with our intuitive sense of the complexity of physical objects, because trivial and random objects are intuitively easy to produce, decompressing quickly in both cases.Bennett provides a careful development [3] of the notion of logical depth, taking into account near-shortest programs rather than the shortest one, in order to arrive at a more robust measure.The simplest version is defined as follows: Definition X.1.Let s be a string and d a significance parameter.A string's depth at significance d, is given by [3]: program p on a Turing machine U ; that is, a program that cannot be compressed by more than a fixed (small) number of d bits.
For algorithmic complexity the choice of UTM is bounded by an additive constant, as shown by the invariance theorem, but for logical depth it is bounded by a multiplicative factor [3].
We will denote by LD(s) the measure based upon the concept of logical depth for a string s as estimated by CTM by running an enumeration over all possible computer programs (Turing machines) increasing in size from shorter to longer.In this fashion CTM will eventually find a program p using the formalism (Rado's Busy Beaver [25]) that produces any string for the first time.The computing time taken by p to produce s is the approximation of LD(s) by CTM.
Definition X.2.We define a BDM version for logical depth LD BDM that applies a multiplicative log penalization instead of an additive one, as we did for previously studied versions of BDM.
where n i is the multiplicity of base stringx i .
In Fig. 8, a numerical experiment is undertaken between BDM and LD for CTM and lossless compression, with LD CT M and LD BDM performing as theoretically expected, i.e., assigning low logical depth to high complexity strings as estimated by BDM.LD CT M and LD BDM can also be defined in higher dimensions, as was done for CTM and BDM separately, based on the replacement of the unidimensional Turing machine tape by an n-dimensional tape.

XI. BDM ASYMPTOTIC TIME COMPLEXITY AND SCALABILITY
Consider the complexity of each step involved in the Block Decomposition Method for approximating the Kolmogorov complexity of a graph.The calculation of BDM for the Kolmogorov complexity approximation of graphs grows in proportion to the growth of the adjacency matrix of M , Adj(M ).That is, O(n 2 ) in the number of vertices n.For every element in an array with floating values between zero and one, one can first precompute the output of the expensive function, as we did with (5, 2), and store it in a minimal perfect hash table.The index of the array is associated with the value stored there, allowing us to look up the value in constant time.It is only the size of the adjacency matrix that registers an increase of n 2 in the number of vertices n that dominates the computational time required to compute K (K log m is also bounded by a small polynomial as it only involves multiplication, which is computationally efficient).The algorithm for tuples calculation involves either knowledge of the length of the object-to calculate the subsets-or traversing the object to perform the partition.
CTM, BDM and LD as functions of the object's size (and therefore the size of the Turing machine rule space that has to be explored) have the following time complexity: • CTM is uncomputable but for decidable cases runs in O(exp) time.
• Non-overlapping string BDM and LD runs in O(1) linear time and O(n d ) polynomial time for d-dimensional objects.
• Overlapping BDM runs in O(ns) time with m the overlapping offset.
• Full overlapping with m = 1 runs in O(2 n ) polynomial time as a function of the number of overlapping elements n.
• Mutual Information BDM runs in O(exp) time for strings and d exponential for dimension d.
Table III, summarizes the range of applications, with CTM and BDM preeminent in that they can more efficiently deal with short, medium and long sequences and other objects such as graphs, images, networks and higher dimensionality objects.

XII. CONCLUSION
We have introduced a well-grounded, theoretically sound and robust measure of complexity that beautifully connects 2 of the main branches of information theory, classical and algorithmic.
We have shown that the methods are scalable in various ways, including native n-dimensional variations of the same measure.The properties and numerical experiments are in alignment with theoretical expectations and represent the only truly different alternative and more accurate measure of algorithmic complexity currently available.We have also shown that BDM is computationally efficient, hence complementing the effective use of lossless compression algorithms for calculation of upper bounds of Kolmogorov complexity.
There are thus three methods available today for approximating K (two of which have been advanced by us, one being completely novel: BDM; and one that was known but had never been calculated before: CTM).Here they are described by their range of application: • CTM deals with all bit strings of length 1-12 (and for some 20-30 bits).
• BDM deals with 12 bits to hundreds of bits (with a cumulative error that grows by the length of the strings-if not applied in conjunction with CTM).The worst case occurs when substrings share information content with other decomposed substrings and BDM just keeps adding their K values individually.
• CTM+BDM (deals with any string length but it is computationally extremely expensive) • Lossless compression deals with no less than 100 bits and is unstable up to about 1K bits.
While CTM cannot produce estimations of longer bitstrings, estimating the algorithmic complexity of even bitstrings can be key to many problems.Think of the challenge posed by a puzzle of 1000 pieces, if you were able to put together only 12 local pieces at a time, you would be able to put all the puzzle together even without ever looking at the whole piece and thus not even requiring to see possible non-local long-range algorithmic patterns.
Because BDM locally estimates algorithmic complexity via algorithmic probability based upon CTM, it is slightly more independent of object description than computable measures such as Shannon entropy, though in the 'worst case' it behaves like Shannon entropy.We have also shown that the various flavours of BDM are extremely robust, both by calculating theoretical errors on tensors and by numerical investigation, establishing that any BDM version is fit for use in most cases.Hence the most basic and efficient one can be used without much concern as to the possible alternative methods that could have been used in its calculation, as we have exhaustively and systematically tested most, if not all, of them.We have shown that all implementations agree with each other in various degrees, with the   complexitycalculator.com.Full code for the R Shiny web server used is available at [27].

Fig. 1 :
Fig. 1: Illustration of hypothetical (non-)regular convergence rates of the constant involved in the invariance theorem.The theorem guarantees that complexity values for a string x measured with different reference UTMs U 1 and U 2 will only diverge by a constant c (the length between U 1 andU 2 ) independent of x.A non-regular rate of convergence behaviour only gives the impression of converging prior to a transitional divergence trend.The Invariance theorem only tells us that at the limit the curve will converge to a small and constant value c, but it tells us nothing about the rate of convergence or about transitional behaviour.

Sections 7 .
1.2 and 7.2 and Fig.10, 11, 12 and 15 in [19]  support the agreements in correlation using different rule spaces of Turing machines and different computing models altogether (cellular automata).Section 7.1.1 of the same paper provides a first comparison to lossless compression.The sections 'Agreement in probability' and 'Agreement in rank' provide further material comparing rule space (5,2) to the rule space (4,3) previously calculated inZenil and For example, for binary strings we can use n = 2 and k = 2 to produce the empirical output distribution (2, 2) of all machines with 2 symbols and 2 states by which all strings of size l = 12 are produced, except two (one string and its complement).But we assign them values max {CT M (y, 2, 2) + r : |y| = 12} where r is different from zero because the missing strings were not generated in (2, 2) and therefore have a greater algorithmic random complexity than any other string produced in (2, 2) of the same length.Then, for l = 12 and m = 1, BDM (X) decomposes s = 010101010101010101 of length |s| = 18 into the following subsequences: multiplicity 4 and 101010101010 multiplicity 3. We then get the CTM values for these sequences: CT M (010101010101, 2, 2) = 26.99073CT M (101010101010, 2, 2) = 26.99073To calculate BDM, we then take the sum of the CTM values plus the sum of the log b of the multiplicities, with b = 2 because the string alphabet is 2, the same as the number of symbols in the set of Turing machines producing the strings.Thus: log 2 (3) + log 2 (4) + 26.99073 + 26.99073 = 57.56642

Fig. 2 :
Fig. 2: Non-overlapping BDM calculations are invariant to block permutations (reshuffling base strings and matrices), even when these permutations may have different complexities due to the reorganization of the blocks that can produce statistical or algorithmic patterns.For example, starting from a string of size 24 (top) or an array of size 8 × 8 (bottom), with decomposition length l = 8 for strings and decomposition l = 4 × 4 block size for the array, all 6 permutations for the string and all 6 permutations for the array have the same BDM value regardless of the shuffling procedure.

Fig. 2
Fig.2shows that the number of permutations is a function of the complexity of the original object, with the number of permutations growing in proportion to the original object's entropybecause the number of different resulting blocks determines the number of different n objects to distribute among the size of the original object (e.g. 3 among 3 in Fig.2(top) or only 2 different 4 × 4 blocks in Fig.2 (bottom)).This means that the non-overlapping version of BDM is not invariant vis-à-vis the variation of the entropy of the object, on account of which it has a different impact on the error introduced in the estimation of the algorithmic complexity of the object.Thus, non-overlapping objects of low complexity will have little impact, but with random objects non-overlapping increases inaccuracy.Overlapping decomposition solves this particular permutation issue by decreasing the number of possible permutations, in order to avoid trivial assignment of the same BDM values.However, overlapping has the undesired effect of systematically overestimating values of algorithmic complexity by counting almost every object of size n, n − 1 times, hence overestimating at a rate of about n(n − 1) for high complexity objects of which the block multiplicity will be low, and by n log(n) for low complexity objects.
of a given matrix into squares of size up to d × d for a given d.A. Recursive BDMThe strategy consists in taking the biggest base matrix multiple of d × d on one corner and dividing it into adjacent square submatrices of the given size.Then we group the remaining cells into 2 submatrices and apply the same procedure, but now for (d − 1) × (d − 1).We continue dividing into submatrices of size 1 × 1.Let X be a matrix of size m × n with m, n ≥ d.Let's denote by quad = {U L, LL, DR, LR} the set of quadrants on a matrix and by quad d the set of vectors of quadrants of dimension d.

Fig. 3 :
Fig. 3: Error rate for 2-dimensional arrays.With no loss of generalization, the error rate for ndimensional tensors lim d→∞ k nn k = 0 is convergent and thus negligible, even for the discontinuities disregarded in this plot which are introduced by some BDM versions, such as non-overlapping blocks and discontinuities related to trimming the boundary condition.

Fig. 4 :
Fig. 4: One way to deal with the decomposition of n-dimensional tensors is to embed them in an n-dimensional torus(n = 2 in the case of the one depicted here), making the borders cyclic or periodic by joining the borders of the object.Depicted here are three examples of graph canonical adjacency matrices embedded in a 2-dimensional torus that preserves the object complexity on the surface, a complete graph, a cycle graph and an Erdös-Rényi graph with edge density 0.5, all of size 20 nodes and free of self-loops.Avoiding borders has the desired effect of producing no residual matrices after the block decomposition with overlapping.
for matrices with n nodes.It corresponds to an adjacency matrix composed of repetitions of the least complex d × d square.It is the all-1 or all-0 entries matrix, because 0 d,d and 1 d,d are the least complex square base matrices (hence the most compressible) of size d.

Fig. 5 :
Fig. 5: NBDM assigns maximum value 1 to any base matrix with highest CTM or any matrix constructed out of base matrices.In this case, the 4 base matrices on the left are those with the highest CTM in the space of all base matrices of the same size, while the matrix to the left is assigned the highest value because it is built out of the maximum complexity base matrices.

Fig. 6 :
Fig. 6: Spearman correlation coefficients (ρ) between CTM and BDM of all possible block sizes and overlap lengths for 12 bit strings, compared with the correlation between CTM and Shannon entropy, and the correlation between CTM and compression length (shown at the rightmost edge of the plot) in blue.ρ coefficients for the 2048 strings below and above the median CTM value are shown in green and orange, respectively.BDM block size and overlap increases to the left.Compression length was obtained using Mathematica's Compress[] function.All values were normalized as described in VII.

G
, and an edge joining two neighbouring faces for each edge in G.If G is a dual graph of G, then A(G ) = A(G), making the calculation of the Kolmogorov complexity of graphs and their dual graphs interesting-because of the correlation between Kolmogorov complexity and A(G ), which should be the same for A(G).One should also expect the estimated complexity values of graphs to be the same as those of their dual graphs, because the description length of the dual graph generating program is O(1).

Fig. 7 :
Fig. 7: Top left: Comparison between values of Entropy, compression (Compress[] and BDM over a sample of 100 strings of length 10 000 generated from a binary random variable following a Bernoulli distribution and normalized by maximal complexity values.Entropy just follows the Bernoulli distribution and, unlike compression that follows entropy, BDM values produce clear convex-shaped gaps on each side assigning lower complexity to some strings compared to both entropy and compression.Top right: The results confirmed using another popular lossless compression algorithm BZip2 on 100 random strings of 100 bits each (BZip2 is slower thanCompress but achieves greater compression).Bottom left: The CT M low (s) − H high (s) gap between near-maximal entropy and low algorithmic complexity grows and is consistent along different string lengths, here from 8 to 12 bits.This gap is the one exploited by BDM and carried out over longer strings, which gives it the algorithmic edge against entropy and compression.Bottom right: When strings are sorted by CTM, one notices that BZip2 collapses most strings to minimal compressibility.Over all 2 12 = 4096 possible binary strings of length 12, entropy only produces 6 different entropy values, but CTM is much more fine-grained, and this is extended to the longer strings by BDM, which succeeds in identifying strings of lower algorithmic complexity that have near-maximal entropy and therefore no statistical regularities.Similar results were obtained when using a third lossless compression algorithm, LZMA.
where T (p) is the number of steps in the computation U (p) = s, |p | is the length of the shortest program that produces s, thus effectively the algorithmic complexity K(s) of s.In other words, LD d (s) is the fastest computing time T (p) required to compute s from a d-incompressible The same decomposition method has allowed us to define a measure based on the seminal concept of logical depth as introduced by Bennett that, unlike algorithmic(Kolmogorov-Chaitin)    complexity, assigns low complexity to both simple and random objects, hence leaving only what appears to have non-trivial structure with high logical depth.Ready to use online and offline methods and data are being released alongside this paper.An animated video is also available at http://www.complexitycalculator.org/animationA.Duality and cospectral graph proofs and test complementLet G be the dual graph of G. Then K(G ) ∼ K(G).Let p denote the finite program that, for any graph G, replaces every edge in G by a vertex and every vertex in G by an edge.The resulting graph produced by p is then G (uniqueness), which implies that |K(G) − K(G )| < |p| because we did not assume that p was the shortest program.Thus, K(G)+ |p| = K(G ) or K(G) ∼ K(G ) up to a constant factor.Let K(Aut(G)) be the algorithmic complexity of the automorphism group Aut(G) of the graph G (i.e.all possible relabellings that preserve graph isomorphism), that is, the length of the shortest program that generates all the graphs in Aut(G).Theorem .1.Let G be an isomorphic graph of G. Then K(G ) ∼ K(G) for all K(G ) ∈ Aut(G),where Aut(G) is the automorphism group of G.The idea is that if there is a significantly shorter program p for generating G compared to a program p generating Aut(G), we can use p to generate Aut(G) via G and a relatively short program c that tries, e.g., all permutations, and checks for isomorphism.Let's assume that there exists a program p such that ||p |−|p|| > c, i.e. the difference is not bounded by any constant, and that K(G) = |p |.We can replace p by p + c to generate Aut(G) such that K(Aut(G)) = p + c, where c is a constant independent of G that represents the size of the shortest program that generates Auth(G), given any G.Then we have it that |K(Aut(G)) − K(G)| < c, which is contrary to the assumption.K(Aut(G)) < K(G ) + O(1) for any G ∈ Aut(G).Let G be in Aut(G) such that G = G .There exists 2 a computer program p that produces Aut(G) for all G.With this program we can construct Aut(G) from any graph G ∈ Aut(G) and G from Aut(G) and the corresponding label n.Therefore K(G ) ≤ |p| + K(Aut(G)) + log(n) + O(1) and K(Aut(G)) <= |p| + K(G ).If G and G are cospectral graphs, then |K(G) − K(G )| < c + log(n), i.e.K(G) ∼ K(G ) up to a constant and small logarithmic term.
only differences having to do with under-or over-estimated values and time complexity and scalability properties.They are extremely robust, thereby establishing that the use of the most basic versions (1D n-o, 2D n-o) are justified in most cases.'M-' stands for Mathematica, 'Online' for the online calculator, 'Cyc' for Cyclic, 'Norm' stands for normalized, 'Rec' for recursive, 'Smo' for 'Smooth', 'N-o' for Nonoverlapping and 'addcol' for the method that adds rows and columns of lowest complexity to the borders up to the base string/array/tensor size.If not stated then it supports overlapping.

Fig. 8 :
Fig. 8: Unlike approximations to algorithmic complexity by lossless compression, LD-based values (versus CTM and BDM) conform to theoretical expectation regarding LD behaviour.

TABLE II :
Spearman ρ values of various BDM versions tested on dual and cospectral graphs that theoretically have the same algorithmic complexity up to a (small) constant.Compression lengths and BDM values in TableII and Fig. 9 (Sup.Inf.) are obtained from the adjacency matrices of 113 dual graphs and 193 cospectral graphs from Mathematica's GraphData[] repository.Graphs and their dual graphs were found by BDM to have estimated algorithmic complexities close to each other.While Entropy and Entropy rate do not perform well in any test compared to the other measures, compression retrieves similar values for cospectral graphs as compared to BDM, but it is outperformed by BDM on the duality test.The best BDM version for duals was different from that for cospectrals.For the duality test, the smooth, fully overlapping version of BDM outperforms all others, but for cospectrality, overlapping recursive BDM outperforms all others.

TABLE III :
Summary of ranges of application and scalability of CTM and all versions of BDM.d stands for the dimension of the object.

TABLE IV :
Computer programs in different languages implementing various BDM versions.