Lindy’s Law and the Longevity of Scientific Theories

Abstract

This work aims to summarize the history and mutations of Lindy’s Law (or the Lindy Effect)—a mathematical distribution that originated from television commentary—and to first test this principle in the context of a recent new iteration: Lindy’s Law as a proxy to describe the significance of longevity as a factor in the retention of scientific theories.

1. Introduction

The formulation of Lindy’s Law (also known as the Lindy Effect) is 60 years old. Its creator, American journalist Albert Goldman, used it as a social commentary on Broadway’s shows and comedians. Today, any reference to “Lindy” stems from the same underlying principle that Goldman laid out, a principle that—through a series of significant turns—has expanded into new domains.

The purpose of this work is to outline the history of Lindy’s Law from its beginnings to its success as a mathematical distribution, until its latest iteration: a metaphor for the role of longevity in the evaluation of ideas.

2. The History of Lindy’s Law

In an article written for The New Republic in April 1964, American journalist Albert Goldman illustrated a seemingly robust empirical rule that described the career longevity of TV comedians: “The life expectancy of a television comedian is proportional to the amount of his exposure on the medium” [1]. This formulation was based on an extensive observation of the conversations happening at Lindy’s Delicatessen, a small restaurant right across Broadway where show industry professionals gathered to discuss the successes and failures of comedy shows. Goldman named it “Lindy’s Law”.

This anecdote captured the attention of Benoit Mandelbrot, French-Polish mathematician and father of fractal geometry. In The Fractal Geometry of Nature (1982), Mandelbrot [2] mentions Lindy’s Law as one example of paradoxical hyperbolic distributions: longevity (of a comedian’s career) increasing (instead of decreasing) with the passing of time. To describe Lindy’s Law, Mandelbrot introduces another law: Lotka’s.

Alfred Lotka [3] had published in 1926 a seminal work called “The Frequency Distribution of Scientific Productivity”, in which—through the analysis of articles published in scientific journals between 1907 and 1916—he discovered that the number of articles published per author followed roughly an inverse-square power law ([4], e.g., there were ¼ as many authors publishing two articles as there were authors publishing only one. This probabilistic distribution was labeled “Lotka’s Law” and has been empirically tested—among others—by Nicholls (1986) [5].

In his work [2], Mandelbrot used Lotka’s Law to explain Lindy’s Law without elaborating on the mathematics of the two probabilistic distributions. The relationship between the two “laws” was not formalized until Eliazar ([6]), who demonstrated that Lindy’s Law is “synonymous” with Lotka’s and other power law distributions.

In particular, Eliazar describes how Lindy’s Law is an example of a power law distribution, which is a function that describes a relationship between two items where a relative change in one results in a proportional relative change in the other. In other words, one quantity varies as a power of another. In Lindy’s Law mathematical formulation, life expectancy is considered a random variable (with a probabilistic exponential distribution) whose mean value is proportional to longevity. Lotka’s Law is a specific example of Lindy’s Law.

Despite being part of the same mathematical distribution family, there is a key difference between Lindy’s Law and Lotka’s: Lindy was meant to describe the expected longevity of an individual career, while the Lotka distribution was meant to capture the different productivity (measured in number of papers published) of a multitude of careers. From a mathematical standpoint, Lindy’s Law is interested in describing the relationship between elapsed time and longevity (a time-average), while Lotka’s Law is interested in describing the expected value (of the productivity) through the observation of multiple individuals (the careers of several scholars)1 [7,8].

By putting together the two laws, Mandelbrot implies that productivity follows an inverse-square law distribution (Lotka) and that the length of one’s career increases its expected longevity (Lindy). The conclusion of the combination of the two “laws” is that, at any point during a comedian’s (or a scholar’s) career, one could assume that their productivity relative to their peers should last for as long as their career lasts.

To illustrate this principle, Mandelbrot tells a short story called the “Parable of the Young Poets’ Cemetery”:

“In the cemetery’s most melancholy section, among the graves of poets and scholars who had fallen unexpectedly in the flower of their youth, each monument is surmounted by a symbol of loss: one half of a book, of a column, or of a tool. The old groundskeeper, himself a scholar and a poet in his youth, urges visitors to take these funereal symbols most literally: “Anyone who lies here,” he proclaims, “had accomplished enough to be viewed as full of promise, and some monuments’ sizes reflect the accomplishments of those whose remains they shelter. But how can we assess their broken promise? A few of my charges may have lived to challenge Leonhard Euler or Victor Hugo in fecundity, if perhaps not in genius. But most of them, alas, were about to be abandoned by their Muses. Since promise and accomplishments are precisely equal in young life, we must view them as equal at the moment of sudden death.”

The conclusion of the combined Lindy–Lotka effect—and of the parable—is that anyone (a poet, a comedian, or a scholar) who at any point during their lifetime seems to be following an extremely successful path—will certainly continue to do so for the rest of their lives. This is curiously against Goldman’s (1964) [1] original interpretation of Lindy’s Law: in his article, he pointed out that the longevity of TV comedians’ careers was helped by a policy of “conservation of resources”, i.e., by not maximizing potential in the early days of the career. In other words, Goldman’s own explanation of Lindy’s Law was not compatible with the idea that Lotka’s law could be observed throughout an individual’s career.

Mandelbrot was certainly more interested in the mathematics of such probabilistic distributions rather than the validity of their conclusions: it is important to point out that the chapter dedicated to Lindy’s Law is called “paradoxical distributions”, which highlights the counterintuitive nature of their conclusions. Nevertheless, scholars have tested Lotka’s Law across different disciplines: an empirical test run by Huber (1998) [9] found that Lotka’s Law was indeed applicable to an individual’s career and that the product frequency of production and career duration did follow (roughly) an inverse-square rule. Huber’s conclusion is consistent with Simonton ([10]), who developed a model aimed at predicting the output of authors on the basis of initial creative potential, age, and elaboration rate (i.e., productivity). Simonton’s conclusions were substantially in line with the “Parable of the Young Poets’ Cemetery”: he found that the career length at which 50% of the initial creative potential had been exploited was, on average, 15.4 years for poets.2

3. From Mathematics to Philosophy

Didier Sornette and Daniel Zajdenweber ([11]) first mentioned Lindy’s Law outside of theoretical mathematics, citing Mandelbrot’s formulation as part of their research on the modeling of the economic returns of research funding. In their work, the authors argue that Lindy’s Law (and, in general, power law distributions) is a practical principle for understanding the limits of using option pricing techniques to evaluate the economic value of research for companies and governments.

Lebanese author Nicholas Nassim Taleb popularized Lindy’s Law in his 2012 book “Antifragile”. In his work, Taleb credits Mandelbrot as a source, but his argument uses as a main reference the research of John Richard Gott ([12]),3 who applied a simple formula to estimate the average outstanding life of an item.

Gott’s starting point is that an individual observer of a subject has no special insights into its lifespan. If this assumption is proven correct, one should assume that the observation takes place within the 95% confidence interval of a normal distribution (of the lifetime of the subject). By inferring the probabilistic distribution of the item, Gott’s formula may estimate its life expectancy. While technically not a power law (see [14]), under Gott’s formula, the expected lifetime of something increases with its longevity, as per Lindy’s.

Gott’s most famous contribution was probably his estimation of the expected lifetime of the human race (1993), which was substantially in line with Carter’s ([15]). Before leaving the domain of poetry for the remainder of this work, it is worth noting that a formulation of Gott’s principle can be found in Baudelaire ([16]): “The world is about to end. The only reason it may last, is that it exists”.

Taleb’s ([17]) main contribution to the subject was to expand the reach of Lindy’s Law, claiming that it is applicable to “informational” non-perishable domains such as technology, concepts, and ideas (in general, on “intellectual production”). He wrote: “For the perishable, every additional day in its life translates into a shorter additional life expectancy. For the nonperishable, every additional day may imply a longer life expectancy”.

It is important to note that Taleb’s interpretation of the principle goes back to the initial formulation of Lindy’s Law—the one coined by Goldman. This is because Taleb does not make assumptions on the power exponent (as per Lotka’s) nor on the distributions of outcomes (as Gott’s). Taleb’s formulation is rooted in the original principle in its most open interpretation.

It is also noteworthy that Taleb’s shift of focus towards the non-perishable probably arises from the backlash (see [18]) triggered by the popularization (see [19]) of Gott’s ideas, which had resulted in the life expectancy of everything (including perishable items) being calculated on the basis of its current age. Moreover, Taleb goes beyond the statistical nature of Lindy’s Law—extrapolating the mathematical principle into a qualitative epistemological principle.

Taleb’s formulation brings Lindy’s Law to a novel territory for the first time: outside of the observation of individuals (where research has focused) and directly into the philosophy of science. His suggested applications of Lindy’s Law are technology (i.e., old technologies still in use are expected to be used for at least as long) and, more crucially, scientific knowledge. Is longevity a variable when it comes to the evaluation of a theory? Will older theories last longer compared to comparatively newer ones?

Before continuing on this path, it is worth noting that—despite transitioning from the mathematical domain to philosophy—Lindy’s Law is meant to assess probabilities. Its proponents are interested in understanding the probability of a certain theory’s disappearance (or persistence), not in using it as an instrument for theory selection or falsification.

4. The Role of Lindy’s Law in the Development of Scientific Theories

The widened formulation of Lindy’s Law is substantially compatible with the framework outlined by Paul Feyerabend in his work Against Method (1975).

Feyerabend’s main thesis was that science is only a “tradition”, largely equivalent to other ones such as religion and aesthetics. Crucially, he refused to follow “well-defined rules” for the “evaluation of ideas”: in his framework, science is just another source of knowledge (not inherently better or worse than religion, witchcraft, or ancient medicine), and scientific breakthroughs are often influenced by unforgotten yet non-scientific concepts. In his main work, the philosopher discussed the influence of ancient beliefs (e.g., voodoo) on the development of modern psychology, as well as the influence of Hermetic writings on Copernicus’ theories.

The Austrian philosopher noted (1975) that there were ideas that survived for centuries yet “[only] now are said to be in agreement with reason”, and welcomed the “prejudice, passion, conceit, errors, sheer pig-headedness” that allow non-scientific theories to survive and influence new discoveries. Feyerabend was not interested in why (or how) old ideas and theories are still remembered (he even mentioned “prejudice” and “conceit” as valid reasons for ideas to survive), and spent a long time explaining how ancient, seemingly unscientific ideas continue to inspire (through “backwards movements”) modern, more “scientific” domains.

Garcia Da Silva Oliveira ([20]) studied Feyerabend’s principle of “retainment” of ideas, finding that under Feyerabend’s framework, “there is no definite elimination of theories”. By not accepting any “rule” on the selection of theories (the “proliferation” method), Feyerabend is implicitly advocating for ideas to last forever.

Tambolo ([21]) emphasizes how “radical” (and widely criticized) Feyerabend’s theory of progress is, where “science is in fact a battlefield” and “victory of one of the competing alternatives” has to be interpreted as a “temporary outcome”. He states that Feyerabend simply “never accepted the claim that truth, or an approximation to the truth, ought to play a regulative role within scientific inquiry”.

It must be highlighted that Feyerabend did not make any prediction on how long ancient ideas are expected to survive, nor did he make any claim that ancient ideas were supposed to last longer than newer ones. Nevertheless, he outlined a framework in which no idea is expected to be overlooked, inviting scientists “to retain the theories of man and cosmos that are found in Genesis, or in the Pimander”. Moreover, it is clear by reading the examples made to outline his framework ([22]) that he was concerned about retaining ancient, seemingly unscientific ideas as opposed to prioritizing comparatively recent theories (which were deemed more acceptable by the scientific community). In Tambolo’s words ([21]), in Against Method (1975), the importance of the “criticism from the past” (i.e., refuted theories experiencing a comeback) is “glorified”.

Although many applications of Lindy’s Law focus on its significance in physics and economics (see [23,24]), a recent contribution from Denic, Souid, and Nicholls ([25]) provides a good example of this process. The authors focus on the history of a routine laboratory test known as the “automated complete blood cell count” (CBC). By tracing back its history (which started from the Egyptians to the Greeks), the authors found that current treatments such as venesection were “resurrected” in the 20th century, once “more credible ideas and methods” replaced the humoral theory, a theory that had “thrived for two millennia”, acquiring “mythical proportions”.

Denic, Souid, and Nicholls state that the history of these treatments is a confirmation of Lindy’s Law, but we can conclude that is also a useful example of Feyerabend’s framework: ancient, pre-scientific methods which were retained for centuries because of traditions become suddenly obsolete—only to be back in vogue once new theories are developed and found consistent with those treatments.

This contribution shows how Lindy’s Law can help describe Feyerabend’s retention of ideas: an ancient method or theory that is still relevant today may have gone through many iterations and may have been brought back at one point (through the “backwards movements” Feyerabend describes). Thus, longevity may be a result of an active process of retention and resurgence of ideas throughout their lifespan.

Whether longevity tells us something about the future lifespan of an idea is something that would go against Feyerabend’s framework, which precisely refused to rank theories according to specific criteria. In his Critique of Mimesis (2022) [26], Emanuele Antonelli has brought forward this idea through the analysis of René Girard’s theories around anthropology and myth. A full description of Girard’s main theories is outside the scope of this work, but it is worth mentioning that the analysis of myths is important to Girard’s anthropological framework because—according to him—they provide valuable insights into the origins of human culture.

Antonelli describes how Girard (1989) believed that ancient stories “tell us the way archaic communities dealt with a very specific situation”. Girard noted that these myths are usually set in moments of crisis (e.g., plagues, wars) and highlighted that the stories themselves identified the causes of the crisis and punished those whom society deemed as culprits (“scapegoating mechanism”). Girard’s thesis is that these “sacrificial crises” helped shaping the society through renewing its social order. Antonelli points out that this process of investigation of causes (that Girard attributes to myths) is a common definition of science4 [27].

Antonelli echoes Feyerabend in stating that these stories “are true in a different but no lesser way than what we now call science” and even goes a step further, mentioning directly Lindy’s Law and stating that ideas that “have been around for a much longer time and are still relevant to our endeavour are more true”. His conclusion is probably the strongest interpretation of Lindy’s Law as an epistemological principle: if the longevity of an idea is meant to have a direct positive relationship with its future life expectancy, it should have a direct relationship with its expected lifespan as a prevailing theory5 [28].

The qualification of ideas—that have to be “still relevant to our endeavour”—makes Antonelli’s conclusions certainly less disruptive (without that, he would be claiming simply that old ideas are truer), but they are against Feyerabend’s key principle of refusing to accept the existence of “prevailing” theories. Going beyond Feyerabend, Antonelli (through Girard) suggests that—precisely because some of these ideas have survived for so long as a prevailing theory—they should be considered as more robust, i.e., more difficult to refute.

In other words, longevity is indeed a useful indicator in the estimation of the endurance of scientific theories: an old theory that is prevailing today has a high probability of remaining the dominant one, and a higher probability of remaining dominant compared to a “prevailing-yet-new” theory.

The reader would appreciate that history of science provides examples in which this principle seems to be violated; every time an old (expressed in centuries or millennia), established prevailing theory has been abandoned in favor of a new one (e.g., heliocentrism, general relativity), we can assume we found ourselves with low-probability events, which can help explain their impact and scope in the history of science.

Lindy’s Law is in this case a precautionary principle: the probability of a new, disruptive theory replacing “old and dominant” one is extremely low, and scientific revolutions are rare6 [28].

5. Conclusions

The purpose of this work is to summarize the history of Lindy’s Law (or Lindy Effect) and document its transition from mathematical paradox to philosophical principle.

The evolution of this principle has opened interesting questions surrounding the role of the longevity of ideas in the evolution of scientific theories. Through the analysis of his main body of work, it can be found that Paul Feyerabend’s framework is compatible with Lindy’s Law and that his principle of retention of ideas can be identified as the underlying process behind the observation of Lindy’s Law for scientific theories and methods.

The work also analyses the instances where Lindy’s Law was applied to the history of science, finding that longevity is starting to be considered in the development of frameworks aimed at describing the process of retention of scientific theories.