A brief history of long memory: Hurst, Mandelbrot and the road to ARFIMA

Long memory plays an important role in many fields by determining the behaviour and predictability of systems; for instance, climate, hydrology, finance, networks and DNA sequencing. In particular, it is important to test if a process is exhibiting long memory since that impacts the accuracy and confidence with which one may predict future events on the basis of a small amount of historical data. A major force in the development and study of long memory was the late Benoit B. Mandelbrot. Here we discuss the original motivation of the development of long memory and Mandelbrot's influence on this fascinating field. We will also elucidate the sometimes contrasting approaches to long memory in different scientific communities


Introduction
In many fields, there is strong evidence that a phenomenon called "long memory" plays a significant role, with implications for forecast skill, low frequency variations, and trends. In a stationary time series the term "long memory"-sometimes "long range dependence" (LRD) or "long term persistence"-implies that there is non-negligible dependence between the present and all points in the past. To dispense quickly with some technicalities, we clarify here that our presentation follows the usual convention (Beran, 1994;Beran et al., 2013) and define a stationary finite variance process to have long memory, when its autocorrelation function (ACF) diverges: ∞ k=−∞ ρ(k) = ∞. This is equivalent to its power spectrum having a pole at zero frequency (Beran, 1994;Beran et al., 2013). In practice, this means the ACF and the power spectrum both follow a power-law, because the underlying process does not have any characteristic decay timescale. This is in striking contrast to many standard (stationary) stochastic processes where the effect of each data point decays so fast that it rapidly becomes indistinguishable from noise.
The study of long memory processes is important because they exhibit nonintuitive properties where many familiar mathematical results fail to hold, and because of the numerous datasets (Beran, 1994;Beran et al., 2013) where evidence for long memory has been found. In this paper we will give a historical account of four key aspects of long memory: • The environmetric observations in the 1950s which first sparked interest: the anomalous growth of range in hydrological time series, later known as the "Hurst" phenomenon.
• After more than a decade of controversy, the introduction by Mandelbrot of the first stationary model-fractional Gaussian noise (FGN)-which could explain the Hurst phenomenon. This was in itself controversial because it explicitly exhibited LRD (which he dubbed "the Joseph effect").
• The incorporation of LRD, via a fractional differencing parameter d, into the more traditional ARMA(p, q) models, through Hosking and Granger's ARFIMA(p, d, q) model.
• The currently active topic of the incorporation of modern statistical inference, and in particular Bayesian methods, into the maturing subject of LRD.
The development of the concept of long memory, both as a physical notion and a formal mathematical construction, should be of significant interest in the light of controversial application areas like the study of bubbles and business cycles in financial markets (Sornette, 2004), and the quantification of climate trends (Franzke, 2012;Bunde et al., 2014). Yet no article about long memory covers the history in much detail. Instead most introduce the concept with passing reference to its historical significance; even books on LRD tend to have only a brief history. 1 This lack of historical context is important not just because a knowledge of the history is both intrinsically rewarding and enlightening, but also because it can help to avoid pitfalls in novel methodological development in modern time series analysis, and in application by practitioners. Here we attempt to bridge that gap in a way that is both entertaining and accessible to a wide statistical and scientific audience. We assume no mathematical details beyond those given in an ordinary time series textbook (e.g., Brockwell and Davis, 1991), and any additional notation and concepts will be kept to a minimum.
The key questions that we seek to answer are "Who first considered long memory processes in time series analysis, and why?" and "How did these early studies evolve into the modern-day subject?" 2 As we shall see, this evolution took less than three decades across the middle of the twentieth century. During this period there was significant debate about the mathematical, physical, and philosophical interpretations of long memory. It is both the evolution of this concept, and the accompanying debate (from which we shall often directly quote), in which we are mostly interested. The kind of memory that concerns us here was a conceptually new idea in science, and rather different, for example, from that embodied in the laws of motion developed by Newton and Kepler. Rather than Markov processes where the current state of a system now is enough to determine its immediate future, the fractional Gaussian noise model requires information about the complete past history of the system.
As will become evident, the late Benoît B. Mandelbrot was a key figure in the development of long memory. Nowadays most famous for coining the term and concept 'fractal', Mandelbrot's output crossed a wide variety of subjects from hydrology to economics as well as pure and applied mathematics. 3 During the 1960s he worked on the theory of stochastic processes exhibiting heavy tails and long memory, and was the first to distinguish between these effects. Because of the diversity of the communities in which he made contributions, it sometimes seems that Mandelbrot's role in statistical modelling is perhaps underappreciated (in contrast, say, with the physics and geoscience communities (Aharony and Feder, 1990;Turcotte, 1997)). It certainly seemed this way to him: Of those three [i.e. economics, engineering, mathematics], nothing beats my impact on finance and mathematics. Physics -which I fear was least affectedrewarded my work most handsomely. (Mandelbrot, 2013) A significant portion of this paper is devoted to his work. We do not, however, intend to convey in any sense his 'ownership' of the LRD concept, and indeed much of the modern progress concerning long memory has diverged substantially from Mandelbrot's work so much that he no longer agreed with some of it.
Mandelbrot's motivation in developing an interest in long memory processes stemmed from an intriguing study in hydrology by Harold Hurst (1951). Before we proceed to discuss this important work it is necessary to give a brief history of hydrological modelling, Hurst's contributions, and the reactions to him from other authors in that area in Section 2. Then we discuss Mandelbrot's initial musings, his later refinements, and the reactions from the hydrological community in Section 3. In Section 4 we discuss the modern development of fractionally differenced models culminating from this sequence of thought. Finally, we conclude in Section 5 with a discussion of some new developments in the field, including Bayesian model averaging to combine short memory, long memory, and heavy tailed processes as a means of accomodating the demands of scientific questions of the present day.
2 For specificity, we clarify here that our interpretation of 'modern-day subject' comprises of the definitions of long memory given above, including footnoted alternatives, together with the ARFIMA(p, d, q) processes defined through the backshift operator B as Φ(B(1 − B) d X t = Θ(B)ε t where Φ and Θ are autoregressive and moving average polynomials, and ε t is white noise. For more details, see any modern time series text (e.g., Brockwell and Davis, 1991).
3 He famously once said "Very often when I listen to the list of my previous jobs I wonder if I exist. The intersection of such sets is surely empty" (Gleick, 1987).

Hurst, and a brief history of hydrology models
Water is essential for society to flourish since it is required for drinking, washing, irrigation and for fuelling industry. For thousands of years going back to the dawn of settled agricultural communities, humans have sought methods to regulate the natural flow of water. They tried to control nature's randomness by building reservoirs to store water in times of plenty, so that lean times are survivable. The combined factors of the nineteenth century Industrial Revolution, such as fast urban population growth, the requirement of mass agricultural production, and increased energy requirements, led to a need to build large scale reservoirs formed by the damming of river valleys. When determining the capacity of the reservoir, or equivalently the height of the required dam, the natural solution is the 'ideal dam': [An 'ideal dam' for a given time period is such that] (a) the outflow is uniform, (b) the reservoir ends the period as full as it began, (c) the dam never overflows, and (d) the capacity is the smallest compatible with (a), (b) and (c). (Mandelbrot and Wallis, 1969a) 4 From a civil engineer's perspective, given the parameters of demand (i.e. required outflow) and time horizon, how should one determine the optimal height of the dam? To answer this question we clearly need an input, i.e. river flows. It is not hard to imagine that for a given set of inputs it would, in principle, be possible to mathematically solve this problem. A compelling solution was first considered by Rippl (1883) "whose publication can . . . be identified with the beginning of a rigorous theory of storage reservoirs" (Klemeš, 1987).
Despite solving the problem, Rippl's method was clearly compromised by its requirement to know, or at least assume, the future variability of the river flows. A common method was to use the observed history at the site as a proxy; however records were rarely as long as the desired time horizon. Clearly a stochastic approach was required, involving a simulation of the future using a stochastic process known to have similar statistical properties to the observed past. This crucial breakthrough, heralding the birth of stochastic hydrology, was made by Hazen (1914) who used the simplest possible model; an iid Gaussian process.
In practice, just one sample path would be of little use so, in principle, many different sample paths could be generated, all of which could be analysed using Rippl's method to produce a distribution of 'ideal heights'. This idea of generating repeated samples was pursued by Sudler (1927), however the stochastic approach to reservoir design was not generally accepted in the West until the work of Soviet engineers was discovered in the 1950s. The important works by Moran (1959) and Lloyd (1967) are jointly considered to be the foundations of modern reservoir design, and helped establish this approach as best practice.

Hurst's paper
Harold Edwin Hurst had spent a long career in Egypt (ultimately spanning 1906-68) eventually becoming Director-General of the Physical Department where he was responsible for, amongst other things, the study of the hydrological properties of the Nile basin (Sutcliffe, 1979). For thousands of years the Nile had helped sustain civilisations in an otherwise barren desert, yet its regular floods and irregular flows were a severe impediment to development. Early attempts at controlling the flow by damming at Aswan were only partially successful. Hurst and his department were tasked with devising a method of water control by taking an holistic view of the Nile basin, from its sources in the African Great Lakes and Ethiopian plains, to the grand delta on the Mediterranean.
In his studies of river flows, Hurst (1951) used a method similar to Rippl's in which he analysed a particular statistic of the cumulative flows of rivers over time called the 'adjusted range', R. Let {X k } be a sequence of random variables, not necessarily independent, with some non-degenerate distribution. We define the nth partial sum Y n =: X 1 + · · · + X n . Feller (1951) then defines the Adjusted Range, R(n), as: Hurst referred to this as simply the 'range' which is now more commonly used for the simpler statistic R * (n) = max 1≤k≤n {Y k }−min 1≤k≤n {Y k }. Moreover he normalised the adjusted range by the sample standard deviation to obtain what is now called the Rescaled Adjusted Range statistic, denoted R/S(n): Hurst's reasons for this normalisation are unclear but, as we shall see later, proved remarkably fortuitous. The attraction of using R/S is that, for a given time period of say n years, R/S(n) is a proxy for the ideal dam height over that time period. Hurst (1951) then examined 690 different time series, covering 75 different geophysical phenomena spanning such varied quantities as river levels, rainfall, temperature, atmospheric pressure, tree rings, mud sediment thickness, and sunspots. He found that in each case, the statistic behaved as R/S(n) ∝ n k for some k. He estimated k using a statistic he called K, and found that K was approximately normally distributed with mean 0.72 and standard deviation 0.006. He actually acknowledged that "K does vary slightly with different phenomena", and that the range (0.46 − 0.96) was large for a Gaussian fit, however to a first approximation it appeared that the mean value of 0.72 might hold some global significance.
At this point it is worth highlighting an aspect of Hurst's work which often gets overlooked. As we shall see, the R/S statistic has enjoyed great use over the past fifty years. However the modern method of estimating the exponent k is not that originally used by Hurst. His estimate K was obtained by assuming a known constant of proportionality: specifically he assumed the asymptotic (i.e. for large n) law that R/S(n) = (n/2) k . A doubly logarithmic plot of values of R/S(n) against n/2 should produce a straight line, the slope of which is taken as K. By assuming a known constant of proportionality, Hurst was effectively performing a one parameter log-regression to obtain his estimate of k.
His reason for choosing this approach was that it implies R/S(2) = 1 exactly 5 , and consequently this 'computable value' could be used in the estimation procedure. This methodology would nowadays be correctly regarded as highly dubious because it involves fitting an asymptotic relationship but using a small fixed point. This logical flaw was immediately remarked upon in the same journal issue by Chow (1951). As we will see, Mandelbrot later introduced the now-standard method of estimation by dropping this fixed point and performing a two-parameter log-regression to obtain the slope. Hurst's original method was forgotten and most authors are unaware that it was not the same as the modern method; indeed many cite Hurst's result of 0.72 unaware that is was obtained using an inappropriate analysis.
Notwithstanding these shortcomings, Hurst's key result that estimates of k were about 0.72 would likely not have been either noteworthy or controversial in itself had he not shown that, using contemporary stochastic models, this behaviour could not be explained.
In the early 1950s, stochastic modelling of river flows was immature and so the only model that Hurst could consider was the iid Gaussian model of Hazen (1914) and Sudler (1927). Rigorously deriving the distribution of the range under this model was beyond Hurst's mathematical skills, but by considering the asymptotics of a coin tossing game and appealing to the central limit theorem, he did produce an extraordinarily good heuristic solution. His work showed that, under the independent Gaussian assumption, the exponent k should equal 0.5. In other words Hurst had shown that contemporary hydrological models fundamentally did not agree with empirical evidence. This discrepancy between the theory and practice became known as the 'Hurst phenomenon'. 6 Hurst's observation sparked a series of investigations that ultimately led to the formal development of long memory. Hurst himself offered no specific explanation for the effect although he clearly suspected the root of cause might lie in the independence assumption: Although in random events groups of high or low values do occur, their tendency to occur in natural events is greater. ... There is no obvious periodicity, but there are long stretches when the floods are generally high, and others when they are generally low. These stretches occur without any regularity either in their time of occurrence or duration (Hurst, 1951, §6).
Despite several follow-up publications (Hurst, 1956a,b;Hurst et al., 1965), Hurst himself played no direct part in the further development of long memory. The specific purpose of his research was to design a system to control the Nile with a series of small dams and reservoirs. 5 it actually equals 1/ √ 2 but Hurst was calculating standard deviations by dividing by n rather than n − 1 6 It is worth clarifying a potential ambiguity here: since the phrase was coined, the 'Hurst Phenomenon' has been attributed to various aspects of time series and/or stochastic processes. For clarity, we will use the term to mean "the statistic R/S(n) empirically grows faster than n 1/2 ". These plans were later turned into the Aswan High Dam with Hurst still acting as scientific consultant into his eighties (Sutcliffe, 1979;Mandelbrot and Hudson, 2008).

Reactions to the Hurst phenomenon
Hurst's finding took the hydrological community by surprise, not only because of the intrinsic puzzle, but because of its potential importance. As previously mentioned, the R/S(n) statistic is a proxy for the ideal dam height over n years. If Hurst's finding was to be believed, and that the statistic increases faster than n 1/2 , there would be potentially major implications for dam design. In other words, dams designed for long time horizons might be too low, with water shortages as one potential consequence.
Although the debate over Hurst's findings, which subsequently evolved into the debate about long memory, was initially largely confined to the hydrological community, fortuitously it also passed into more mainstream mathematical literature -a fact which undoubtedly helped to lend it credence in later years. Despite Hurst's non-rigorous approach, and an unclear mathematical appeal for what was essentially a niche subject, the eminent probabilist Willy Feller (1951) contributed greatly by publishing a short paper. By appealing to Brownian motion theory, he proved rigorously that Hurst was correct; for sequences of standardised iid random variables with finite variance, the asymptotic distribution of the adjusted range, R(n), should obey the n 1/2 law: E[R(n)] ∼ π 2 1/2 n 1/2 . It should be emphasised that Feller was studying the distribution of the adjusted range, R(n), not the rescaled adjusted range R/S(n). The importance of dividing by the standard deviation was not appreciated until Mandelbrot, however Feller's results would later be shown to hold for this statistic as well.
By proving and expanding (since the Gaussianity assumption could be weakened) Hurst's result, Feller succeeded in both confirming that there was a phenomenon of interest, and also that it should interest mathematicians as well as hydrologists. Over the course of the 1950s more precise results were obtained although attention was unfortunately deflected consideration the simple range (e.g. Anis and Lloyd, 1953;Anis, 1955Anis, , 1956) as opposed to R/S. The exact distribution of R(n) was found to be, in general, intractable; a notable exception being that for the simplest iid Gaussian case, where (Solari and Anis, 1957) Having conclusively shown that Hurst's findings were indeed worthy of investigation, several different possible explanations of the eponymous phenomenon were put forward. It was assumed that the effect was caused by (at least) one of the following properties of the process: a) an 'unusual' marginal distribution, b) non-stationarity, c) transience (i.e pre-asymptotic behaviour), or d) short-term auto-correlation effects. For Hurst's original data, the first of these proposed solutions was not relevant because much of his data were clearly Gaussian. Moran (1964) claimed that the effect could be explained by using a sequence of iid random variables with a particular moment condition on the distribution. Although this case had been shown by Feller (1951) to still asymptotically produce the n 1/2 law, Moran showed that in such cases the transient (or the pre-asymptotic) phase exhibiting the Hurst phenomenon could be extended arbitrarily. 7 Furthermore, Moran pointed out that if the finite variance assumption was dropped altogether, and instead a symmetric α-stable distribution was assumed, the Hurst phenomenon could apparently be explained: E[R(n)] ∼ kn 1/α , for 1 < α ≤ 2. and some known (computable) k. However, as Mandelbrot later showed, the division by the standard deviation is indeed crucial. In other words, whilst Moran's arguments were correct, they were irrelevant because the object of real interest was the rescaled adjusted range. Several Monte Carlo studies, notably those by Mandelbrot and Wallis (1969b), confirmed that for iid random variables, regardless of marginal distribution, R/S(n) asymptotically follows the n 1/2 law. That Mandelbrot (1975) and Mandelbrot and Taqqu (1979) were finally able to prove this result remains of the principal reasons why the R/S statistic has remained popular to the present day.
The second potential explanation of the Hurst phenomenon, non-stationarity, is harder to discount and is more of a philosophical (and physical) question than a mathematical one.
Is it meaningful to talk of a time-invariant mean over thousands of years? If long enough realizations of such time series were available would they in fact be stationary? (O'Connell, 1971, §3.

2)
Once we assume the possibility of non-stationarity, it is not hard to imagine that this could lead to an explanation of the phenomenon. Indeed Hurst (1957) himself suggested that non-stationarity might be an explanation; however his heuristics involving a pack of playing cards were far from being mathematically formalisable. Klemeš (1974) and Potter (1976) later provided more evidence, however the first rigorous viable mathematical model was that by Bhattacharya et al. (1983), in which the authors showed that a short-memory process perturbed by a non-linear monotonic trend can be made to exhibit the Hurst phenomenon. 8 In his influential paper, Klemeš (1974) not only showed that the Hurst phenomenon could be explained by non-stationarity, but argued that assuming stationarity may be mis-founded: The question of whether natural processes are stationary or not is likely a philosophical one. . . . there is probably not a single historic time series of which mathematics can tell with certainty whether it is stationary or not . . . Traditionally, it has been assumed that, in general, the geophysical, biological, economical, and other natural processes are nonstationary but within relatively short time spans can be well approximated by stationary models. (Klemeš, 1974) As an example, Klemeš suggested that a major earthquake might drastically affect a river basin so much as to induce a regime-change (i.e. an element of non-stationarirty). However on a larger (spatial and temporal) scale, the earthquake and its local deformation of the Earth may be seen as part of an overall stationary 'Earth model'. Thus choosing between the two forms is, to some extent, a matter of personal belief. As we will see in the next section, Mandelbrot did in fact consider other models with a particular type of nonstationarity himself while formulating his stationary FGN model, but unfortunately Klemeš was unaware of that work, about which a more fruitful discussion might perhaps have occurred.
If we discount this explanation and assume stationarity, we must turn to the third and fourth possible explanations, namely transience (i.e. pre-asymptotic behaviour) and/or the lack of independence. These two effects are related: short-term auto-correlation effects are likely to introduce significant pre-asymptotic behaviours. As mentioned earlier, Hurst himself suggested some kind of serial dependence might explain the effect, and Feller suggested: It is conceivable that the [Hurst] phenomenon can be explained probabilistically, starting from the assumption that the variables {X k } are not independent . . . Mathematically this would require treating the variables {X k } as a Markov process. (Feller, 1951) Soon however, Barnard (1956) claimed to have shown that Markovian models still led to the n 1/2 law and it would be shown later (Mandelbrot and Wallis, 1968;Mandelbrot, 1975) that any then-known form of auto-correlation must asymptotically lead to the same result. The required condition on the auto-correlation function turned out to be that it is summable, whereby for iid random variables with ACF ρ(·) (Siddiqui, 1976): Even before this was formally proved, it was generally known that some complicated autocorrelation structure would be necessary to explain the Hurst phenomenon: It has been suggested that serial correlation or dependence [could cause the Hurst phenomenon]. This, however, cannot be true unless the serial dependence is of a very peculiar kind, for with all plausible models of serial dependence the series of values is always approximated by a [Brownian motion] when the time-scale is sufficiently large. A more plausible theory is that the experimental series used by Hurst are, as a result of serial correlation, not long enough for the asymptotic formula to become valid. (Moran, 1959) Thus Moran was arguing that, since no 'reasonable' auto-correlation structure could account for the Hurst phenomenon, it should be assumed that the observed effect was caused by preasymptotic behaviour, the extent of which was influenced by some form of local dependence. In other words, he was suggesting that a short-memory process could account for the Hurst phenomenon over observed time scales. This issue has both a practical and philosophical importance. It would later be argued by some that, regardless of the 'true' model, any process that could exhibit the Hurst phenomenon over the observed (or required) time scales would suffice for practical purposes. Using such processes requires a choice. One might accept the Hurst phenomenon as genuine and acknowledge that, although theoretically incorrect, such a model is good enough for the desired purpose. Alternatively, one might reject the Hurst phenomenon as simply a pre-asymptotic transient effect, and therefore any model which replicates the effect over observed ranges of n is potentially valid. 9 Mandelbrot, for one, was highly critical of those who followed the latter approach: So far, such a convergence [to the n 1/2 law] has never been observed in hydrology. Thus, those who consider Hurst's effect to be transient implicitly attach an undeserved importance to the value of [the sample size] . . . These scholars condemn themselves to never witness the full asymptotic development of the models they postulate. (Mandelbrot and Wallis, 1968) Despite this, the concept of short-memory-induced transience was explored both before and after Mandelbrot's work. Matalas and Huzzen (1967) performed a rigorous Monte Carlo analysis of the AR(1) model and demonstrated that for medium n and heavy lag-one serial correlation, the Hurst phenomenon could be induced (albeit Matalas and Huzzen were actually using Hurst's original erroneous K estimate). Fiering (1967) succeeding in building a more sophisticated model; however he found he needed to use an AR(20) process to induce the effect -an unrealistically large number of lags to be useful for modelling.
To summarise, more than a decade on from Hurst's original discoveries, no satisfactory explanation for the Hurst phenomenon had been found. To quote (Klemeš, 1974) again: Ever since Hurst published his famous plots for some geophysical time series . . . the by now classical Hurst phenomenon has continued to haunt statisticians and hydrologists. To some it has become a puzzle to be explained, to others a feature to be reproduced by their models, and to others still, a ghost to be conjured away.
It was at this point that Benoît Mandelbrot heard of the phenomenon.

Mandelbrot's fractional models
In the early 1960s, Mandelbrot had worked intensively on the burgeoning subject of mathematical finance which is concerned with modelling economic indices such as share prices. Central to this subject was the 'Random Walk Hypothesis' which provided for Brownian motion models. This was first implicitly proposed in the seminal (yet long undiscovered) doctoral thesis by Bachelier (1900). The detailed development of this topic is also interesting but beyond the scope of this paper. It suffices to say here that, although Bachelier's model was recognised as an adequate working model which seemed to conform to both intuition and the data, it could also benefit from refinements. Various modifications were proposed but one common feature they all shared was the underlying Gaussian assumption.
In a ground-breaking paper, Mandelbrot (1963) proposed dropping the Gaussianity assumption and instead assuming a heavy tailed distribution, specifically the symmetric αstable distribution (e.g., Samorodnitsky and Taqqu, 1994, §1.1). In short, this notion was highly controversial; for example see Cootner (1964a). But the paper was significant for two reasons. Firstly it helped to give credibility to the growing study of heavy tailed distributions and stochastic processes. Secondly it was indicative of Mandelbrot's fascination with mathematical scaling. The α-stable distributions have the attractive property that an appropriately re-weighted sum of such random variables is itself an α-stable random variable. This passion for scaling would remain with Mandelbrot throughout his life, and is epitomised by his famous fractal geometry.
Returning to Hurst's results, Mandelbrot's familiarity with scaling helped him immediately recognise the Hurst phenomenon as symptomatic of this, and, as he later recounted (Mandelbrot and Hudson, 2008), he assumed that it could be explained by heavy tail processes. He was therefore surprised when he realised that, not only were Hurst's data essentially Gaussian, but as discussed previously, the rescaled adjusted range is not sensitive to the marginal distribution. Instead, he realised that a new approach would be required. In keeping with the idea of scaling, he introduced the term 'self-similar', 1 and formally introduced the concept in its modern form: Let Y (t) be a continuous-time stochastic process. Then Y (t) is said to be self-similar, with self-similarity parameter J, if for all positive c, Y (ct) d = c J Y (t). Using this concept, Mandelbrot (1965b), laid the foundations for the processes which would initially become the paradigmatic models in the field of long memory, the self-similar fractional Brownian motion (FBM) model and its increments, the long range dependent fractional Gaussian noise (FGN) model.
At this point it is necessary to informally describe FBM. It is a continuous-time Gaussian process, and is a generalisation of ordinary Brownian motion, with an additional parameter h. 2 This parameter can range between zero and one (non-inclusive to avoid pathologies) with different values providing qualitatively different types of behaviour. The case h = 1/2 corresponds to standard Brownian motion.
Fractional Brownian motion can be thought of in several different and equivalent ways, for example as a fractional derivative of standard Brownian motion, or as stochastic integral. These details need not concern us here; the most important fact is that FBM is exactly self-similar, which means that a 'slowed-down' version of a process will, after a suitable spatial re-scaling, look statistically identical to the original, i.e. they will have the same finite dimensional distributions. In this sense FBM, like standard Brownian motion (which of course is just a special case of FBM), has no characteristic time-scale, or 'tick'. In practical applications it is necessary to use a modification of FBM because it is (like standard Brownian motion) a continuous time process and non-stationary. Thus the increments of FBM are considered; these form a discrete process which can be studied using conventional time series analysis tools. These increments, called fractional Gaussian noise (FGN), can be considered to be the discrete approximation to the 'derivative' of FBM. Note that in the case of h = 1/2, FGN is simply the increments of standard Brownian motion, i.e. white noise. Mandelbrot and Van Ness (1968, corollary 3.6) showed that this process is stationary, but most importantly (for its relevance here), it exhibits the Hurst phenomenon: for some c > 0, R/S(n) ∼ cn h . This result was immensely significant; it was the first time since Hurst had first identified the phenomenon, that anyone had been able to exhibit a stationary, Gaussian stochastic process capable of reproducing the effect. The mystery had been partially solved; there was such a process, and for over a decade it remained the only model known to be able to fully explain the Hurst phenomenon. Mandelbrot (1965b) then proceeded to show that such a process must have a spectral density function that blows up at the origin. By proposing such a model, he realised he was attempting to explain with one parameter h both low-and high-frequency effects, i.e. he was ". . . postulating the same mechanism for the slow variations of climate and for the rapid variations of precipitation". He also recognised that the auto-correlation function of the increments would decay slower than exponential, and (for 1/2 < h < 1) would not be summable. This correlation structure, which is now often taken to be the definition of long memory itself, horrified some. Concurrently, the simplicity of FGN, possessing only one parameter h, concerned others. We shall consider these issues in depth later, but the key point was that although Mandelbrot had 'conquered' the problem, to many it was somewhat of a Pyrrhic victory (Klemeš, 1974).

Initial studies of Mandelbrot's model
Mandelbrot immediately attempted to expand on the subject although his papers took time to get accepted. He ultimately published a series of five papers in 1968-9 through collaborations with the mathematician John Van Ness and the hydrologist James Wallis. Taken as a whole, these papers offered a comprehensive study of long memory and fractional Brownian motion. They helped publicise the subject within the scientific community and started the debates about the existence of long memory and the practicality of FGN, which have continued until the present day.
The first of these papers (Mandelbrot and Van Ness, 1968) formally introduced FBM and FGN and derived many of their properties and representations. The aim of this paper was simply to introduce these processes and to demonstrate that they could provide an explanation for the Hurst phenomenon; this was succinctly stated: We believe FBM's do provide useful models for a host of natural time series and wish therefore to present their curious properties to scientists, engineers and statisticians.
Mandelbrot and Van Ness argued that all processes thus far considered have "the property that sufficiently distant samples of these functions are independent, or nearly so", yet in contrast, they pointed out that FGN has the property "that the 'span of interdependence' between their increments can be said to be infinite". This was a qualitative statement of the difference between short and long memory and soon led to the formal definition of long memory. As motivation for their work, they cited various examples of observed time series which appeared to possess this property: in economics (Adelman, 1965;Granger, 1966), '1/f noises' in the fluctuations of solids (Mandelbrot, 1967a), and hydrology (Hurst, 1951).
Intriguingly, and undoubtedly linked to Mandelbrot's original interest in heavy-tailed processes, Mandelbrot and Van Ness (1968, §3.2) noted: If the requirement of continuity is abandoned, many other interesting selfsimilar processes suggest themselves. One may for example replace [the Brownian motion] by a non-Gaussian process whose increments are [α-] stable . . . Such increments necessarily have an infinite variance. 'Fractional Lévy-stable random functions' have moreover an infinite span of interdependence.
In other words the authors postulated a heavy-tailed, long memory process. It would be over a decade before such processes were properly considered due to difficulties arising from the lack of formal correlation structure in the presence of infinite variance. 3 One key point which is often overlooked is that Mandelbrot and Van Ness did not claim that FGN is necessary to explain the Hurst phenomenon: ". . . we selected FBM so as to be able to derive the results of practical interest with a minimum of mathematical difficulty". Often Mandelbrot was incorrectly portrayed as insisting that his, and only his, model solved the problem. Indeed Mandelbrot himself took an interest in alternative models, although as we will later see, he essentially rejected Granger and Hosking's ARFIMA which was to become the standard replacement of FGN in statistics and econometrics literatures.
Furthermore, neither did the authors claim that they were the first to discover FBM. They acknowledged that others (e.g. Kolmogorov, 1940) had implicitly studied it; however Mandelbrot and Van Ness were undoubtedly the first to attempt to use it in a practical way.
Having 'solved' Hurst's riddle with his stationary fractional Gaussian model, Mandelbrot determined to get FGN and FBM studied and accepted, in particular by the community which had most interest in the phenomenon, hydrology. Therefore his remaining four important papers were published in the leading hydrological journal Water Resources Research. These papers represented a comprehensive study of FBM in an applied setting, and were bold; they called for little short of a revolution in stochastic modelling: ... current models of statistical hydrology cannot account for either [Noah or Joseph] effect and must therefore be superseded. As a replacement, the 'selfsimilar' models that we propose appear very promising (Mandelbrot and Wallis, 1968) As its title suggests, Mandelbrot and Wallis (1968) introduced the colourful terms 'Noah Effect' and 'Joseph Effect' for heavy tails and long memory respectively; both labels referencing key events of the Biblical characters' lives. Ironically, the river level data were in fact close enough to Gaussian to dispense with the 'Noah Effect' so the actual content of the paper was largely concerned with the 'Joseph Effect', but rainfall itself provides a rich source of heavy tailed, "Noah" datasets. However Mandelbrot preferred treating these two effects together as different forms of scaling; spatial in the former and temporal in the latter. Mandelbrot and Wallis (1968) defined the 'Brownian domain of attraction' (BDoA) and showed that such models cannot account for either effect and should therefore be discarded. The BDoA was (rather loosely) defined as the set of discrete-time stochastic processes which obey three conditions; namely, the Law of Large Numbers, the Central Limit Theorem, and asymptotic independence of past and future partial sums. Alternatively the BDoA is the set of processes which are either asymptotically Brownian, or can be well-approximated by Brownian motion. A process in the BDoA is in some sense 'nice', i.e. it is Gaussian or Gaussian-like and has short memory, and was given the term 'smooth'. Processes outside of the BDoA were labelled 'erratic' 4 . This 'erratic' behaviour could be caused by one, or both, of the Joseph and Noah effects. Mandelbrot and Wallis showed that processes lying within the BDoA will, after an initial transient behaviour, obey the n 1/2 law. They rejected, on philosophical grounds, that the Hurst phenomenon might be caused by transient effects.
Mandelbrot proceeded to provide more evidence in support of his model. Mandelbrot and Wallis (1969a) included several sample graphs of simulated realisations of FGN with varying h. The explicit aim was to "encourage comparison of [the] artificial series with the natural record with which the reader is concerned". These simulations were performed by using one of two methods developed by the authors which were different types of truncated approximations. 5 Mandelbrot and Wallis wanted to subject these simulations to R/S analysis but they recognised the previously mentioned logical flaw in Hurst's approach. They therefore developed 4 Mandelbrot later preferred the terms 'mild' to "nice", and subdivided "erratic" into heavy tailed 'wild' and strongly dependent 'slow". We stick with his original terminology. 5 As documented by Mandelbrot (1971) it was soon found that one of these approximations was far from adequate because it failed to accurately reproduce the desired effects. The algorithms were also slow to implement; a significant practical problem when computer time was expensive and processing power limited. Mandelbrot (1971) therefore introduced a more efficient algorithm. Later an exact algorithm would be created by Hipel and McLeod (1978a,b) which forms the basis for modern algorithms (Davies and Harte, 1987;Beran, 1994) which use the Fast Fourier Transform . a systematic two-parameter log-regression to obtain an estimate of h which we will denote H. This approach to R/S analysis has since become the standard method.
The simulated sample paths were subjected to both R/S and spectral analysis, and for both cases it was found that the simulated paths largely agreed with the theory, i.e. the sample paths seemed sufficiently good representations of the theoretical processes. For the R/S analysis, it was found, as expected, that there existed three distinct regions: transient behaviour, 'Hurst' behaviour, and asymptotic '1/2' behaviour. This last region was caused entirely because the simulations were essentially short memory approximations to the long memory processes; infinite moving averages were truncated to finite ones. Thus this third region could be eliminated by careful synthesis, i.e. by making the running averages much longer than the ensemble length. Furthermore the transient region was later shown (Taqqu, 1970) to be largely a feature of a programming error. Mandelbrot and Wallis (1969c) applied their R/S method to many of the same data types as Hurst (1951Hurst ( , 1956a and Hurst et al. (1965), and similarly found significant evidence in favour of the long memory hypothesis. In a comparison of Hurst's K with their H, Mandelbrot and Wallis pointed out that K will tend to under-estimate h when h > 0.72 but over -estimate when h < 0.72. So Hurst's celebrated finding of a global average of 0.72 was heavily influenced by his poor method, and his estimated standard deviation about this mean was underestimated. This important point, that the original empirical findings which helped spawn the subject of long memory were systematically flawed, has long been forgotten.
Next, Mandelbrot and Wallis (1969b) undertook a detailed Monte Carlo study of the robustness to non-Gaussianity of their R/S method. As previously mentioned, in general R/S was shown to be very robust. The different distributions studied were Gaussian, lognormal, 'hyperbolic' (a skewed heavy-tailed distribution -not α-stable but attracted to that law), and truncated Gaussian (to achieve kurtosis lower than Gaussian). The distribution of the un-normalised adjusted range, R(n), was shown to be highly dependent on the distribution, however the division by S(n) corrected for this. For any sequence of iid random variables, their estimate H was always (close to) 1/2. When studying dependent cases, they considered various non-linear transformations (such as polynomial or exponential transforms) and found that robustness still held. However R/S was shown to be susceptible in the presence of strong periodicities; a fact rather optimistically dismissed: "Sharp cyclic components rarely occur in natural records. One is more likely to find mixtures of waves that have slightly different lengths . . . ".
Finally, Mandelbrot and Wallis (1969b) intriguingly replaced the Gaussian variates in their FGN simulator with 'hyperbolic' variates. Although now known have drawbacks, this was for a long time the only attempt at simulating a heavy-tailed long memory process.

Reactions to Mandelbrot's model
By proposing heavy tailed models to economists, Mandelbrot had had a tough time advocating against orthodoxy (Mandelbrot, 2013). Because his fractional models were similarly unorthodox, he learned from his previous experience, and was more careful about introducing them to hydrologists. By producing several detailed papers covering different aspects of FBM he had covered himself against charges of inadequate exposition. Unsurprisingly however, many hydrologists were unwilling to accept the full implications of his papers.
Firstly, Mandelbrot's insistence on self-similar models seemed somewhat implausible and restrictive, and seemed to totally ignore short-term effects. Secondly, Mandelbrot's model was continuous-time which, although necessary to cope with self-similarity, were only useful in a theoretical context because we live in a digital world; data are discrete and so are computers. As soon as his models were applied to the real world they became compromised 6 : The theory of fractional noise is complicated by the motivating assumptions being in continuous time and the realizable version being needed in discrete time. (Lawrance and Kottegoda, 1977, §6.2) In one major respect Mandelbrot was simply unlucky with timing. Soon after his papers about FBM were published, the hugely influential book by Box and Jenkins (1970) was published, revolutionising the modelling of discrete time series in many subject areas.
Prior to 1970, multiple-lag auto-regressive or moving average models had been used (and as previously mentioned had failed to adequately replicate the Hurst phenomenon), but the Box-Jenkins models combined these concepts, together with an integer differencing parameter d, to produce the very flexible class of ARIMA(p, d, q) models. As in other scientific fields, many hydrologists were attracted to these models, and sought to explore the possibility of using them to replicate the Hurst phenomenon.
It is important to note that ARIMA models cannot genuinely reproduce the asymptotic Hurst phenomenon since all ARIMA models either have short memory, or are non-stationary. However by choosing parameters carefully, it can be shown that it is possible to replicate the observed Hurst phenomenon over a large range of n. O'Connell (1971) was an early exponent of this idea; specifically he used an ARMA(1, 1) model which could (roughly) preserve a given first-lag auto-correlation as well as h. 7 To summarise, in the early 1970s there were two distinct approaches to modelling hydrological processes. One could use traditional AR processes (or their more advanced ARMA cousins) which, although able to partially replicate the Hurst phenomenon, were essentially short memory models. Alternatively one could use Mandelbrot's FGN process in order to replicate the Hurst phenomenon accurately. Unfortunately this dichotomy was strong and the choice of approach largely came down to whether accounting for low-or high-frequency effects was the principal aim for the modeller. Mandelbrot himself was well aware (c.f. Mandelbrot and Wallis, 1968, p911) that he was suggesting switching the usual order of priority when modelling stochastic processes. Many were uncomfortable with this approach 6 Mandelbrot was primarily interested in FBM; he saw the necessary discretisation, FGN, as its derivative, both literally and metaphorically.
7 For completeness, we mention that other modelling approaches were investigated to try and replicate the Hurst phenomenon. One such model was the so-called 'broken-line' process detailed by Rodríguez-Iturbe et al. (1972), Garcia et al. (1972), and Mejia et al. (1972Mejia et al. ( , 1974 which sought to preserve a twice differentiable spectrum. This was criticised by Mandelbrot (1972a) and did not prosper. because, whereas the ARMA models could be coerced into replicating the Hurst phenomenon, FGN was completely uncustomisable with regards to high frequencies.
It remains for the hydrologist to decide which type of behaviour [low-or highfrequency] is the more important to reproduce for any particular problem. No doubt derivations of FGN's preserving both high and low frequency effects will eventually emerge and such a choice will not be necessary. (O'Connell, 1971, §2.3) Further studies involving ARMA processes were undertaken by Wallis and O'Connell (1973), Lettenmaier and Burges (1977) (who proposed a mixture of an ARMA(1,1) model with an independent AR(1) model), and the set of papers by McLeod and Hipel (1978) and Hipel and McLeod (1978a,b). These latter authors were the first to apply to long memory processes the full Box-Jenkins philosophy of time series estimation: model identification, parameter estimation, and model-checking. To compare models, they were the first to use formal procedures such as information criteria, and formally test residuals for whiteness. With this setup they fitted models to six long-run geophysical time series suspected of possessing long memory, and found that in each case the best fitting ARMA models were chosen in preference to FGN. They also fitted more complex ARMA models (than ARMA(1,1)) and showed again that the observed Hurst statistic can be maintained over the length of series used. 8 Along with their practical difficulty, another ground for rejecting Mandelbrot's models was his sweeping assertions about their physical interpretation. Slightly paraphrasing, he claimed that, since long memory was the only explanation for the Hurst phenomenon, the underlying physical processes must possess long memory. This approach of inferring physics from an empirical model was generally rejected. For a start, many were reluctant to drop the natural Markovian assumption about nature: The past influences the future only through its effect on the present, and thus once a certain state of the process has been reached, it matters little for the future development how it was arrived at. (Klemeš, 1974) Indeed the renown hydrologist Vit Klemeš was a leading opponent of Mandelbrot's interpretation. As indicated earlier, he personally suspected non-stationarity might be the true cause for the observed Hurst phenomenon. Whilst he was convinced of the importance of the Hurst effect, and accepted FGN as an empirical model (he used the phrase 'operational model') he strongly rejected using it to gain an understanding of physics: An ability to simulate, and even successfully predict, a specific phenomenon does not necessarily imply an ability to explain it correctly. A highly successful operational model may turn out to be totally unacceptable from the physical point of view. (Klemeš, 1974) He likened the apparent success of Mandelbrot's FGN in explaining the Hurst phenomenon to the detrimental effect that the Ptolemaic planetary model had on the development of astronomy. Klemeš had strong reservations about the concept of long memory, asking: By what sort of physical mechanism can the influence of, say, the mean temperature of this year at a particular geographic location be transmitted over decades and centuries? What kind of a mechanism is it that has carried the impact of the economic crisis of the 1930s through World War II and the boom of the 1950s all the way into our times and will carry it far beyond? though he conceded that there were in fact possible mechanisms in the man-made world, although not, in his view, in the physical one. Klemeš (1974) also showed how at least two other classes of model would exhibit the Hurst effect, i) integrated processes, such as random walks or AR(1) with a φ parameter close to 1, and ii) a model with heavy-tailed distributions of the times between changes in the mean. Ironically, this was very similar in spirit to a model that Mandelbrot had discussed almost 10 years earlier, as we will now explain.
Because of the attention received by FBM it is often not realised that Mandelbrot proposed and published another class of models for 1/f noise in parallel with FBM (Mandelbrot, 1965a(Mandelbrot, , 1967a)! These models were, however, very different from the stationary FGN, being, in an important sense, nonstationary. Mandelbrot (1965aMandelbrot ( , 1967a) discussed a simple example, though unfortunately in a characteristically complex way. He described a stochastic process intended to model intermittency, in which "off" periods of no activity were interrupted by jumps to a negative or positive "on" active state. Rather than the more familiar short-tailed distributions of the waiting times between transitions between on and off states, long tailed ones were described. His key finding was that traditional measures such as the periodogram and autocorrelation function would return a "1/f " spectrum for this model, but that the power spectrum S(f ) could be decomposed into a stationary part, and one depending on the time series' length. Rather than representing a true singularity in power at the lowest frequencies, in this model he described the infrared catastrophe in the PSD as a "mirage" resulting from the fact that the moments of the model varied in time in a step-like fashion, a property he called "conditional covariance stationarity": In other words, the existence of f θ−2 noises challenges the mathematician to reinterpret spectral measurements otherwise than in "Wiener-Khinchin" terms.
... operations meant to measure the Wiener-Khinchin spectrum may unvoluntarily measure something else, to be referred to as the "conditional spectrum" of a "conditionally covariance stationary" random function. (Mandelbrot, 1967b) Mandelbrot drew a clear contrastbetween his conditionally stationary, non-Gaussian 1/f model and his stationary Gaussian (FGN) model: Section VI [... of this paper... ] showed that some f θ−2 L(f ) noises have a very erratic sampling behavior. Some other f θ−2 noises are Gaussian and, therefore, perfectly "well-behaved;" an example is provided by the "fractional white noise" [i.e. FGN] which is the formal derivative of the process of Mandelbrot and Van Ness [i.e. FBM] (Mandelbrot, 1967b) Unfortunately Mandelbrot (1967a,b) received far less contemporary attention than did his papers on heavy tails in finance in the early 1960s or the series with Van Ness and Wallis in 1968-69 on stationary fractional Gaussian models for LRD, gaining only about 20 citations in its first 20 years. It was for example apparently unknown to his critic Vit Klemeš, and seems to be still relatively little known, being for example not cited by Beran et al. (2013), and while listed in the citations of Beran (1994) we can find no mention in the text. Although he revisited the paper with a new commentary in the volume of his Selecta (Mandelbrot, 1999) that deals with multifractals and "1/f " noise, Mandelbrot himself neglected to mention it explicitly in his popular and historical accounts of the genesis of LRD such as (Mandelbrot and Hudson, 2008), and clearly saw it as a different strand of his work to that on fractional Brownian motion collected in another Selecta volume (Mandelbrot, 2002c). Fractal point process models closely related to those of Mandelbrot (1967a) were, however, subsequently studied extensively by others (e.g. Lowen and Teich (2005)), particularly with applications to internet traffic, and are now being actively considered as solutions to the "1/f paradox", e.g. by Niemann et al. (2013) in the context of blinking quantum dots. In this latter context the property of time dependence in "1/f " spectra is receiving renewed attention, being related to the physical phenomenon of weak ergodicity breaking.
Sadly Klemeš and Mandelbrot seem not to have debated nonstationary approaches on an equal footing with the stationary Gaussian FGN, as with the advantage of historical distance one can see the importance of both. One may view this kind of structured nonstationarity as a mechanism for 1/f noise as being an intermediate step between stationary LRD, and more unstructured and arbitrary forms of nonstationarity, although space prevents further discussion here.
Klemeš was not alone in his concern over the interpretation of Mandelbrot's models: Using self-similarity (with h = 1/2) to extrapolate the correlated behaviour from a finite time span to an asymptotically infinite one is physically completely unjustified. Furthermore, using self-similarity to intrapolate [sic] to a very short time span . . . is physically absurd. (Scheidegger, 1970) Interestingly, in his reply, Mandelbrot (1970) somewhat missed the point: [The] self-similar model is the only model that predicts for the rescaled range statistic . . . precisely the same behaviour as Harold Edwin Hurst has observed empirically. To achieve the same agreement with other models, large numbers of ad hoc parameters are required. Thus the model's justification is empirical, as is ultimately the case for any model of nature.
Yet another argument used to oppose the use of long memory models arose from a debate about their practical value. By not incorporating long memory into models, at how much of a disadvantage was the modeller? Clearly, this is a context-specific question, but the pertinent question in hydrology is: by how much does incorporating long memory into the stochastic model change the ideal dam height? One view, shared by Mandelbrot: The preservation within synthetic sequences . . . [of h] is of prime importance to engineers since it characterizes long term storage behaviour. The use of synthetic sequences which fail to preserve this parameter usually leads to underestimation of long term storage requirements. (O'Connell, 1971).
By ignoring the Hurst phenomenon, we would generally expect to underestimate the ideal dam height but how quantifiable is the effect? Wallis and Matalas (1972) were the first to demonstrate explicitly that the choice of model did indeed affect the outcome: by comparing AR(1) and FGN using the Sequential Peak algorithm -a deterministic method of assessing storage requirements based on the work of Rippl (1883) and further developed in the 1960s. Wallis and Matalas showed that the height depends on both the short and long memory behaviours, and in general, FGN models require larger storage requirements, as expected. Lettenmaier and Burges (1977) went into more detail by looking at the distribution of the ideal dam height (rather than simply the mean value) and found it followed extreme value theory distributions. Lettenmaier and Burges also showed that long memory inputs required slightly more storage, thus confirming the perception that long memory models need to be used to guard against 'failure'.
However Klemeš et al. (1981) argued against using this philosophy; instead suggesting that 'failure' is not an absolute term. In the context of hydrology, 'failure' would mean being unable to provide a large enough water supply; yet clearly a minimal deficit over a few days is a different severity to a substantial drought over many years. Any 'reasonable' economic analysis should take this into account. Klemeš et al. claimed that the incorporation of long memory into models used to derive the optimum storage height is essentially a 'safety factor', increasing the height by a few percent, however ". . . in most practical cases this factor will be much smaller than the accuracy with which the performance reliability can be assessed." In summary therefore, Mandelbrot's work was controversial because, although it provided an explanation of Hurst's observations, the physical interpretation of the solution was unpalatable. There was no consensus regarding the whole philosophy of hydrological modelling; should the Hurst phenomenon be accounted for, and if so implicitly or explicitly? Moreover, the new concept of long memory, borne out of the solution to the riddle, was both non-intuitive and mathematically unappealing.
Much of the debate outlined above was confined to the hydrological community, in particular the pages of Water Resources Research. With the exception of articles appearing in probability journals concerning the distributions of various quantities related to the rescaled adjusted range, little else was known about long memory by statisticians. This was rectified by a major review paper by Lawrance and Kottegoda (1977) which helped bring the attention of the Hurst phenomenon to the wider statistical community.
One of those non-hydrologists who took up the Hurst 'riddle' was the eminent econometrician Clive Granger. In an almost-throwaway comment at the end of a paper, Granger (1978) floated the idea of 'fractionally differencing' a time series, whose spectrum has a pole at the origin. 9 Granger's observation was followed up by both himself, and independently by the hydrologist Hosking (1981), who between them laid the foundations for a different class of long memory model. This class of ARFIMA models are the most commonly used long memory models of the present day. If the empirical findings of Hurst helped to stimulate the field, and the models of Mandelbrot helped to revolutionise the field, the class of ARFIMA models can be said to have made the field accessible to all.

Fractionally differenced models
One of the objections to Mandelbrot's fractional Gaussian noise was that it was a discrete approximation to a continuous process. Hosking (1981) explained how FGN can be roughly thought of as the discrete version of a fractional derivative of Brownian motion. In other words, FGN is obtained by fractionally differentiating, then discretising. Hosking proposed to reverse this order of operations, i.e discretising first, then fractionally differencing.
The advantage of this approach is that the discrete version of Brownian motion has an intuitive interpretation; it is the simple random walk, or ARIMA(0, 1, 0) model. We may fractionally difference this using the well-defined 'fractional differencing operator of order d' to obtain the ARFIMA(0, d, 0) process, which for 0 < d < 1/2 is stationary and possesses long memory. From this loose derivation, we immediately see a clear advantage of this process: it is formalisable as a simple extension to the classical Box-Jenkins ARIMA models. Granger and Joyeux (1980) arrived at a similar conclusion noticing that it was both possible to fractionally difference a process and, in order not to over-or under difference data, it may be desirable to do so. Direct motivation was provided by Granger (1980) who showed that such processes could arise as an aggregation of independent AR(1) processes, where the Auto-Regressive parameters were distributed according to a Beta distribution (this aggregation of micro-economic variables was a genuine motivation, rather than a contrived example). Furthermore, Granger and Joyeux pointed out that in long-term forecasts it is the low frequency component that is of most importance. 10 Both Granger and Joyeux (1980) and Hosking (1981) acknowledged that their model was based on different underlying assumptions to Mandelbrot's models. They also recognised the extreme usefulness of introducing long memory to the Box-Jenkins framework. By considering their fractionally differenced model as an ARIMA(0, d, 0) process, it was an obvious leap to include the parameters p, q in order to model short-term effects; thence the full ARFIMA(p, d, q) model. By developing a process which could model both the short and long memory properties, the authors had removed the forced dichotomy between ARMA and FGN models. By being able to model both types of memory simultaneously, ARFIMA models immediately resolved the main practical objection to Mandelbrot's FGN model. Depending on the individual context and viewpoint, ARFIMA models can either be seen as pure short memory models adjusted to induce long memory behaviour, or pure long memory models adjusted to account for short-term behaviour. ARFIMA models are more often introduced using the former of these interpretations -presumably because most practitioners encounter the elementary Box-Jenkins models before long memory -however it is arguably more useful to consider the latter interpretation.
Although slow to take off, the increased flexibility of ARFIMA models, and their general ease of use compared to Mandelbrot's FGN, meant that they gradually became the long memory model of choice in many areas including hydrology and econometrics, although we have found them still to be less well known in physics than FGN. Apart from their discreteness (which may, or may not be a disadvantage depending on the point of view) the only disadvantage that ARFIMA models have is that they are no longer completely self-similar. The re-scaled partial sums of a 'pure' ARFIMA(0, d, 0) model converge in distribution to FBM (see e.g. Taqqu, 2003, §6), so, in some sense, the process can be seen as the increments of an asymptotically self-similar process. However any non-trivial short memory component introduces a temporal 'tick' and destroys this self-similarity.
Perhaps inevitably given his original motivation for introducing self-similarity as an explanation for the Hurst phenomenon, and his further development of the whole concept of scaling into fractal theory, Mandelbrot was not attracted to ARFIMA models. Decades after their introduction, and despite their popularity, Mandelbrot would state: [Granger] prefers a discrete-time version of FBM that differs a bit from the Type I and Type II algorithm in Mandelbrot and Wallis (1969a). Discretization is usually motivated by unquestionable convenience, but I view it as more than a detail. I favor very heavily the models that posses properties of time-invariance or scaling. In these models, no time interval is privileged by being intrinsic. In discrete-time models, to the contrary, a privileged time interval is imposed nonintrinsically. (Mandelbrot, 2002b) Convenience would seem to rule the roost in statistics, however, as ARFIMA-based inference is applied in practice far more often than FBM/FGN. Many practitioners would argue that sample path of a given process and is therefore stochastic in nature. Given the same model and parameters, two practitioners will produce different sample paths (assuming their random number generator seeds are not initiated to the same value). However their sequences will have the same statistical properties.
it is not hard to justify use of a "privileged time interval" in a true data analysis context: the interval at which the data are sampled and/or at which decisions based on such data would typically be made, will always enjoy privilege in modeling and inference.
As we saw above, the introduction of the LRD concept into science came with Mandelbrot's application of the fractional Brownian models of Kolmogorov to an environmetric observation -Hurst's effect in hydrology. Nowadays, an important new environmetric application for LRD is to climate research. Here, the fractionally differenced ARFIMA model now widespread (e.g., Percival and Mofjeld, 2001;Gil-Alana, 2005;Franzke, 2010Franzke, , 2012Imbers et al., 2014). In that context ARFIMA plays an important role in understanding long-term climate variability and in trend estimation, but remains less well known in some user communities compared to, for example, SRD models of the Box-Jenkins type, of which AR(1) is still the most frequently applied. Conversely, in many branches of physics the fractional α-stable family of models including FBM remain rather better known than ARFIMA. The process of codifying the reasons for the similarities and differences between these models, and also the closely related anomalous diffusion models such as the Continuous Time Random Walk, in a way accessible to users, is underway -e.g. the short review of (Watkins, 2013) -but much more remains to be done here, particularly on the "physics of FARIMA".

Conclusion
We have attempted to demonstrate the original motivation behind long memory processes, and trace the early evolution of the concept. Debates over the nature of such processes, and their applicability or appropriateness to real life, are still ongoing. Importantly, the physical meaning of FBM has been clarified by studies which show how it plays the role of the noise term in the generalised Langevin equation when a particular ("1/f ") choice of heat bath spectral density has been made, see for example Kupferman (2004). Rather than draw our own conclusions, we rather intended to illuminate the story of this fascinating area of science, and in particular the role played by Benoit Mandelbrot, who died in 2010. The facet of Mandelbrot's genius on show here was to use his strongly geometrical mathematical imagination to link some very arcane aspects of the theory of stochastic processes to the needs of operational environmetric statistics. Quite how remarkable this was can only be fully appreciated when one reminds oneself of the available data and computational resources of the early 1960s, even at IBM. The wider story (Mandelbrot and Hudson, 2008;Mandelbrot, 2013) in which this paper's theme is embedded, of how he developed and applied in sequence, first the α-stable model in economics, followed by the fractional renewal model in 1/f noise, and then FBM, and a fractional hyperbolic precursor to LFSN, and finally a multifractal model, in the space of about 10 years, shows both mathematical creativity and a real willingness to listen to what the data was telling him. The fact the he (and his critics) were perhaps less willing to listen to each other is a human trait whose effects on this story (we trust) will become less significant over time.
A new and exciting thread to emerge quite recently involves Bayesian inference and model averaging. Until recently, classical methods have dominated the literature. The earliest Bayesian works include the parametric models of Koop et al. (1997), Pai and Ravishanker (1998), and Hsu and Breidt (2003), however all were fraught with computational drawbacks for longer time series. The Bayesian paradigm could potentially offer advantages over classical methods including flexibility in specification of priors (i.e., physical expertise could be used to elicit an informative prior) if computational hurdles could be surmounted. Even the Bayesian community began to doubt potential in certain contexts for computational reasons. Liseo et al. (2001) and Holan et al. (2009) argued, echoing a sentiment in the classical literature, that full parametric long memory models (like ARFIMA) are 'too hard' to work with using Bayesian machinery. Furthermore, often long memory parameters (e.g., d) are the only object of real interest, and consideration of a single class of models, such as ARFIMA blending long and (a specific form of) short memory, is too restrictive, motivating methods which have similarities to classical periodograms.
However, recent methods (e.g, Graves et al., 2014) have demonstrated that ARFIMA presents a convenient framework for efficient Bayesian model averaging in order to marginalise out short memory, seasonal effects and missing observations, so that statements about long memory effects can be made unconditionally. Graves et al. provide a trans-dimensional MCMC scheme (Green, 1995) that modernizes the approach of Ehlers andBrooks (2006, 2008) in the style of Pai and Ravishanker (1998), blending a new approximate likelihood with long and short memory elements, all of which can be integrated within a single, coherent, Bayesian inferential framework. The same framework can be extended to handle heavy-tailed, and even infinite variance innovations, blending both Noah and Joseph effects into a single model (Graves, 2013).