#
Computational Creativity and Aesthetics with Algorithmic Information Theory^{ †}

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

#### 1.1. Motivation

#### 1.2. Summary of Results

## 2. Descriptional Complexity and Existence of an Artifact

#### 2.1. Kolmogorov Complexity

#### 2.2. Algorithmic Probability

## 3. The Order and Complexity of an Artifact

#### 3.1. Two-Part Code and Models

#### 3.1.1. Finite Set Model

#### 3.1.2. Total Recursive Function Model

#### 3.2. Randomness Deficiency and Typicality

**Definition**

**1.**

#### 3.3. Novelty

**Definition**

**2.**

## 4. Value as Computational Effort

#### 4.1. Logical Depth and Its Relation to Value

**Definition**

**3.**

#### 4.2. Compressibility ≠ Value

#### 4.3. The Logical Steps of a Creative Process

**Definition**

**4.**

## 5. Sophistication and the Creator

“We may call a novel complex if it has a great many different characters, scenes, subplots, and so on, so that the regularities of the novel require a long description.”

#### 5.1. Generative Attributes of a Creator

**Definition**

**5.**

#### Computability

#### 5.2. Non-Stochastic Objects or Masterpieces

**Definition**

**6.**

#### High Sophistication = Strong Depth

## 6. Related Works and Discussion

#### A Formal Framework

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

AIT | Algorithmic Information Theory |

## References

- Kolmogorov, A.N. On Tables of Random Numbers. Sankhyā Indian J. Stat. Ser. A
**1963**, 25, 369–376. [Google Scholar] [CrossRef][Green Version] - Solomonoff, R. A formal theory of inductive inference. Part II. Inf. Control
**1964**, 7, 224–254. [Google Scholar] [CrossRef][Green Version] - Chaitin, G.J. On the Length of Programs for Computing Finite Binary Sequences. J. ACM
**1966**, 13, 547–569. [Google Scholar] [CrossRef] - Vitányi, P.M.B. Tolstoy’s Mathematics in War and Peace. Math. Intell.
**2013**, 35, 71–75. [Google Scholar] [CrossRef][Green Version] - Ritchie, G. Some Empirical Criteria for Attributing Creativity to a Computer Program. Minds Mach.
**2007**, 17, 67–99. [Google Scholar] [CrossRef][Green Version] - Jordanous, A. Four PPPPerspectives on computational creativity in theory and in practice. Connect. Sci.
**2016**, 28, 194–216. [Google Scholar] [CrossRef][Green Version] - Li, M.; Vitányi, P.M. An Introduction to Kolmogorov Complexity and Its Applications, 4th ed.; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
- Cilibrasi, R.; Vitányi, P.M.B.; de Wolf, R. Algorithmic clustering of music. In Proceedings of the Fourth International Conference on Web Delivering of Music, 2004. EDELMUSIC 2004, Barcelona, Spain, 14 September 2004; IEEE: Manhattan, NY, USA, 2004; pp. 110–117. [Google Scholar]
- Mondol, T.; Brown, D.G. Grammar-based Compression and its use in Symbolic Music Analysis. J. Math. Music. Adv. Online Publ.
**2021**. [Google Scholar] [CrossRef] - Nikvand, N.; Wang, Z. Generic image similarity based on Kolmogorov complexity. In Proceedings of the 2010 IEEE International Conference on Image Processing, Hong Kong, China, 26–29 September 2010; pp. 309–312. [Google Scholar]
- Zhang, X.; Hao, Y.; Zhu, X.Y.; Li, M. New Information Distance Measure and Its Application in Question Answering System. J. Comput. Sci. Technol.
**2008**, 23, 557–572. [Google Scholar] [CrossRef] - Zenil, H.; Jean-Paul, D.; Gaucherel, C. Image Characterization and Classification by Physical Complexity. Complexity
**2012**, 17, 26–42. [Google Scholar] [CrossRef][Green Version] - Mondol, T.; Brown, D.G. Incorporating Algorithmic Information Theory into Fundamental Concepts of Computational Creativity. In the Proceedings of the Twelfth International Conference on Computational Creativity, Mexico City, Mexico, 14–18 September 2021; pp. 173–181.
- Turing, A.M. On Computable Numbers, with an Application to the Entscheidungsproblem. Proc. Lond. Math. Soc.
**1937**, s2-42, 230–265. Available online: https://academic.oup.com/plms/article-pdf/s2-42/1/230/4317544/s2-42-1-230.pdf (accessed on 27 February 2021). [CrossRef] - Church, A. An Unsolvable Problem of Elementary Number Theory. Am. J. Math.
**1936**, 58, 345–363. [Google Scholar] [CrossRef][Green Version] - Vitányi, P.M. How Incomputable is Kolmogorov Complexity? Entropy
**2020**, 22, 408. [Google Scholar] [CrossRef][Green Version] - Kraft, L.G. A Device for Quantizing, Grouping, and Coding Amplitude-Modulated Pulses. Master’s Thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 1949. [Google Scholar]
- Woo, C.H. Laws and Boundary Conditions. In Complexity, Entropy and the Physics of Information; Zurek, W.H., Ed.; CRC Press: Boca Raton, FL, USA, 2018; pp. 127–135. [Google Scholar] [CrossRef]
- Vitanyi, P.M. Meaningful Information. IEEE Trans. Inf. Theory
**2006**, 52, 4617–4626. [Google Scholar] [CrossRef][Green Version] - Vereshchagin, N.K.; Vitanyi, P.M. Kolmogorov’s Structure Functions and Model Selection. IEEE Trans. Inf. Theory
**2004**, 50, 3265–3290. [Google Scholar] [CrossRef] - Gacs, P.; Tromp, J.T.; Vitanyi, P.M.B. Algorithmic statistics. IEEE Trans. Inf. Theory
**2001**, 47, 2443–2463. [Google Scholar] [CrossRef] - Koppel, M. Structure. The Universal Turing Machine: A Half-Century Survey, 2nd ed.; Springer: Berlin/Heidelberg, Germany, 1995; pp. 403–419. [Google Scholar]
- Brown, D.G.; Mondol, T. On the Problem of Small Objects. Entropy
**2021**, 23, 1524. [Google Scholar] [CrossRef] - Boden, M.A. The Creative Mind: Myths and Mechanisms; Basic Books, Inc.: New York, NY, USA, 1991. [Google Scholar]
- McGregor, S. Algorithmic Information Theory and Novelty Generation. In Proceedings of the Fourth International Joint Workshop on Computational Creativity, Goldsmith College, University of London, London, UK, 17–19 June 2007; pp. 109–112. [Google Scholar]
- Bennett, C.H.; Gacs, P.; Ming, L.; Vitanyi, P.M.B.; Zurek, W.H. Information distance. IEEE Trans. Inf. Theory
**1998**, 44, 1407–1423. [Google Scholar] [CrossRef] - Cage, J. 4’33”; Edition Peters: Leipzig, Germany, 1952. [Google Scholar]
- Gann, K. No Such Thing as Silence: John Cage’s 4’33′′; Yale University Press: New Haven, CT, USA, 2011. [Google Scholar]
- Landauer, R. Irreversibility and Heat Generation in the Computing Process. IBM J. Res. Dev.
**1961**, 5, 183–191. [Google Scholar] [CrossRef] - Arora, S.; Barak, B. Computational Complexity: A Modern Approach, 1st ed.; Cambridge University Press: Cambridge, MA, USA, 2009. [Google Scholar]
- Antunes, L.; Fortnow, L.; van Melkebeek, D.; Vinodchandran, N.V. Computational Depth: Concept and Applications. Theor. Comput. Sci.
**2006**, 354, 391–404. [Google Scholar] [CrossRef][Green Version] - Bennett, C.H. Logical Depth and Physical Complexity. In A Half-Century Survey on the Universal Turing Machine; Oxford University Press, Inc.: New York, NY, USA, 1988; pp. 227–257. [Google Scholar]
- Colton, S. Creativity Versus the Perception of Creativity in Computational Systems. In Proceedings of the AAAI Spring Symposium: Creative Intelligent Systems, AAAI, Palo Alto, CA, USA, 26–28 March 2008; pp. 14–20. [Google Scholar]
- Adriaans, P.W. Between Order and Chaos: The Quest for Meaningful Information. Theory Comput. Syst.
**2009**, 45, 650–674. [Google Scholar] [CrossRef][Green Version] - Borges, J.L. Collected Fictions; Penguin Books: London, UK, 1998. [Google Scholar]
- Shannon, C.E. Prediction and Entropy of Printed English. Bell Syst. Tech. J.
**1951**, 30, 50–64. [Google Scholar] [CrossRef] - Gell-Mann, M.; Lloyd, S. Effective complexity. In Nonextensive Entropy; Santa Fe Institute Studies in the Sciences of Complexity; Oxford University Press: New York, NY, USA, 2004; pp. 387–398. [Google Scholar]
- Mota, F.; Aaronson, S.; Antunes, L.; Souto, A. Sophistication as Randomness Deficiency. In Descriptional Complexity of Formal Systems; Jurgensen, H., Reis, R., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 172–181. [Google Scholar]
- Koppel, M.; Schler, J.; Argamon, S. Computational methods in authorship attribution. J. Am. Soc. Inf. Sci. Technol.
**2009**, 60, 9–26. [Google Scholar] [CrossRef][Green Version] - Grover, L.K. A Fast Quantum Mechanical Algorithm for Database Search. In Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing, STOC ’96, Philadelphia, PA, USA, 22–24 May 1996; Association for Computing Machinery: New York, NY, USA, 1996; pp. 212–219. [Google Scholar] [CrossRef][Green Version]
- Koppel, M.; Atlan, H. An Almost Machine-Independent Theory of Program-Length Complexity, Sophistication, and Induction. Inf. Sci.
**1991**, 56, 23–33. [Google Scholar] [CrossRef] - Antunes, L.; Fortnow, L. Sophistication Revisited. Theor. Comp. Syst.
**2009**, 45, 150–161. [Google Scholar] [CrossRef] - Shen, A. The concept of (α,β)-stochasticity in the Kolmogorov sense, and its properties. Sov. Math. Dokl.
**1983**, 28, 295–299. [Google Scholar] - Joosten, J.; Soler-Toscano, F.; Zenil, H. Program-size versus Time Complexity Slowdown and Speed-up Phenomena in the Micro-cosmos of Small Turing Machines. Int. J. Unconv. Comput.
**2011**, 7, 353–387. [Google Scholar] - Lathrop, J.I.; Lutz, J.H. Recursive Computational Depth. Inf. Comput.
**1999**, 153, 139–172. [Google Scholar] [CrossRef][Green Version] - Teller, R.J. Tim’s Vermeer; Sony Pictures Home Entertainment: Culver City, CA, USA, 2014. [Google Scholar]
- Schütz, K. Vermeer: The Complete Works; Taschen: Köln, Germany, 2019. [Google Scholar]
- Jones, J. DIY Vermeer documentary utterly misses the point about old masters. The Guardian. 28 January 2014. Available online: https://www.theguardian.com/artanddesign/jonathanjonesblog/2014/jan/28/tims-vermeer-fails (accessed on 21 September 2021).
- Moles, A.A. Information Theory and Esthetic Perception; Cohen, J.E., Translator; University of Illinois Press: Urbana, IL, USA, 1966. [Google Scholar]
- Birkhoff, G.D. Aesthetic Measure; Harvard University Press: Cambridge, MA, USA, 1933. [Google Scholar]
- Rigau, J.; Feixas, M.; Sbert, M. Conceptualizing Birkhoff’s Aesthetic Measure Using Shannon Entropy and Kolmogorov Complexity. In Proceedings of the Third Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging, Computational Aesthetics’07, Banff, AB, Canada, 20–22 June 2007; Eurographics Association: Goslar, Germany, 2007; pp. 105–112. [Google Scholar]
- Kosheleva, M.; Kreinovich, V.; Yam, Y. Towards the Use of Aesthetics in Decision Making: Kolmogorov Complexity Formalizes Birkhoff’s Idea. Bull. Eatcs
**1998**, 66, 166–170. [Google Scholar] - Schmidhuber, J. Low-Complexity Art. Leonardo
**1997**, 30, 97–103. [Google Scholar] [CrossRef] - Cilibrasi, R.; Vitányi, P.M.B.; de Wolf, R. Algorithmic Clustering of Music Based on String Compression. Comput. Music J.
**2004**, 28, 49–67. [Google Scholar] [CrossRef] - Cilibrasi, R.; Vitanyi, P.M.B. Clustering by compression. IEEE Trans. Inf. Theory
**2005**, 51, 1523–1545. [Google Scholar] [CrossRef][Green Version] - Ens, J.; Pasquier, P. CAEMSI: A Cross-Domain Analytic Evaluation Methodology for Style Imitation. In Proceedings of the Ninth International Conference on Computational Creativity, Salamanca, Spain, 25–29 June 2018; pp. 64–71. [Google Scholar]
- Svangård, N.; Nordin, P. Automated Aesthetic Selection of Evolutionary Art by Distance Based Classification of Genomes and Phenomes Using the Universal Similarity Metric. In Applications of Evolutionary Computing; Springer: Berlin/Heidelberg, Germany, 2004; pp. 447–456. [Google Scholar]
- Vidal, C.; Delahaye, J.P. Universal Ethics: Organized Complexity as an Intrinsic Value. In Evolution, Development and Complexity; Georgiev, G.Y., Smart, J.M., Flores Martinez, C.L., Price, M.E., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 135–154. [Google Scholar]

**Figure 1.**(

**a**) is a simple highly compressible black canvas as compared to the random noise in (

**b**). Neither has any aesthetic value. However, their combination (

**c**) is also not valuable, illustrating the notion that raw complexity is not a measure of aesthetic quality: meaningless objects can be found across the spectrum of possible complexity scores. Image taken from: Zenil et al. [12].

**Figure 2.**The left column is dictionary M of MIDI pitch patterns collected from a corpus of Beatles songs; the patterns exceed a certain occurrence threshold—they are frequently repeated in the artist’s songs. The right column is a song “Let it Be” from the same artist, with highlighted pitch patterns from M and the individual patterns that can be regarded as extra information d.

**Figure 3.**The children playing “Duck, Duck, Goose” are much more regularly positioned (

**b**) than the randomly playing children (

**a**).

**Figure 4.**The mono-coloured image on the left is more compressible than the complex one on the right; yet, the latter takes more decompression time from its short program. Image taken from: Zenil et al. [12].

**Figure 5.**The image on the left is purely random noise. Hence, a suitable program that generates it will contain a lot of verbatim descriptions, taking little time to reproduce the image. The image on the right, by the virtue of being highly compressible, takes more decompression time from its short program. Image taken from: Zenil et al. [12].

Creative Entity | Attributes | Algorithmic Information Theory Notion |
---|---|---|

Artifact | Typicality | Randomness Deficiency |

Novelty | Mutual Information between model parameters | |

Order and Noise | Model and data-to-model codes | |

Creative Process | Non-randomness | Logical Steps of s-significant program |

Value (also of artifact) | s-significant Logical Depth | |

Creator | Skills and Style | Sophistication |

Masterpiece | Non-stochasticity |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mondol, T.; Brown, D.G.
Computational Creativity and Aesthetics with Algorithmic Information Theory. *Entropy* **2021**, *23*, 1654.
https://doi.org/10.3390/e23121654

**AMA Style**

Mondol T, Brown DG.
Computational Creativity and Aesthetics with Algorithmic Information Theory. *Entropy*. 2021; 23(12):1654.
https://doi.org/10.3390/e23121654

**Chicago/Turabian Style**

Mondol, Tiasa, and Daniel G. Brown.
2021. "Computational Creativity and Aesthetics with Algorithmic Information Theory" *Entropy* 23, no. 12: 1654.
https://doi.org/10.3390/e23121654