Share with your friends










Submit

Analytics Magazine

The Hurst Exponent: Predictability of Time Series

July/August 2012

A statistical measure used to classify time series and infer the level of difficulty in predicting and choosing an appropriate model for the series at hand.

Subir MansukhaniBy Subir Mansukhani

“Clouds are not spheres, mountains are not cones, coastlines are not circles, and bark is not smooth, nor does lightning travel in a straight line.”
– Benoit Mandelbrot

In 1951 the celebrated British hydrologist H.E. Hurst published a paper titled, “The Long-Term Storage Capacity of Reservoirs.” The paper dealt specifically with the modeling of reservoirs, but as it turned out, the results also held valid for a number of other natural systems. Hurst was looking for a way to model the levels of the river Nile so that architects could construct an appropriately sized reservoir system. While his recommendations were not implemented (the 1952 Egyptian revolutions saw to that), he gave life to a statistical methodology for distinguishing random from non-random systems and to identify the persistence of trends, a methodology known as Rescaled Range analysis or R/S analysis.

Many years later, while investigating the fractal nature of financial markets – specifically, the tendency of a time series to regress strongly to its mean or to cluster in a direction – noted mathematician Benoit Mandelbrot happened to stumble across Hurst’s work and, recognizing the potential therein, introduced to fractal geometry, in Hurst’s honor, the term Generalized Hurst Exponent. Put simply, the Hurst exponent is used as a measure of the long-term memory of a time series.

In addition to the Hurst exponent, Mandelbrot also coined two more terms useful in describing the long-term memory of a time series. He called the first one the Joseph Effect and the second one the Noah Effect. The Joseph Effect tells us whether movements in a time series are part of a long-term trend and refers to the Old Testament where Egypt would experience seven years of rich harvest followed by seven years of famine. The Noah Effect is the tendency of a time series to have abrupt changes and the name is derived from the biblical story of the Great Flood. Both of these effects in a time series can be inferred from the Hurst exponent.

Estimating the Hurst exponent

The Hurst exponent is not so much calculated as it is estimated. A variety of techniques exist for estimating the Hurst exponent (H) and the process detailed here is both simple and highly data intensive. To estimate the Hurst exponent one must regress the rescaled range on the time span of observations. To do this, a time series of full length is divided into a number of shorter time series and the rescaled range is calculated for each of the smaller time series. A minimum length of eight is usually chosen for the length of the smallest time series. So, for example, if a time series has 128 observations it is divided into:

  • two chunks of 64 observations each
  • four chunks of 32 observations each
  • eight chunks of 16 observations each
  • 16 chunks of eight observations each

Steps for estimating the Hurst exponent after breaking the time series into chunks:

For each chunk of observations, compute:

  • the mean of the time series,
  • a mean centered series by subtracting the mean from the series,
  • the cumulative deviation of the series from the mean by summing up the mean centered values,
  • the Range (R), which is the difference between the maximum value of the cumulative deviation and the minimum value of the cumulative deviation,
  • the standard deviation (S) of the mean centered values, and
  • the rescaled range by dividing the Range by the standard deviation.

Finally, average the rescaled range over all the chunks.

The rescaled range and chunk size follows a power law, and the Hurst exponent is given by the exponent of this power law. When the frequency of an event varies as the power of some quantity associated with the event, it is said to follow a power law. A wide variety of natural and manmade phenomena follow a power law. For example, the 80/20 rule (20 percent of the population holds 80 percent of wealth), the winner-take-all phenomenon, friend connections in a social network and forest fires all follow power laws.

Interpreting the Hurst Exponent

Using the Hurst exponent we can classify time series into types and gain some insight into their dynamics. Here are some types of time series and the Hurst exponents associated with each of them.

A Brownian time series: In a Brownian time series (also known as a random walk or a drunkard’s walk) there is no correlation between the observations and a future observation; being higher or lower than the current observation are equally likely. Series of this kind are hard to predict. Figure 1 provides an example of a Brownian time series and its estimated Hurst exponent. The Hurst exponent for the data plotted above was estimated to be 0.53 – a Hurst exponent close to 0.5 is indicative of a Brownian time series.

Figure 1: A Brownian time series (H = 0.53).
Figure 1: A Brownian time series (H = 0.53).

An anti-persistent time series: In an anti-persistent time series (also known as a mean-reverting series) an increase will most likely be followed by a decrease or vice-versa (i.e., values will tend to revert to a mean). This means that future values have a tendency to return to a long-term mean. Figure 2 provides an example of an anti-persistent time series and its estimated Hurst exponent.

Figure 2: An anti persistent time series (H = 0.043).
Figure 2: An anti persistent time series (H = 0.043).

The Hurst exponent for the data plotted above was estimated to be 0.043.A Hurst exponent value between 0 and 0.5 is indicative of anti-persistent behavior and the closer the value is to 0, the stronger is the tendency for the time series to revert to its long-term means value.

A persistent time series: In a persistent time series an increase in values will most likely be followed by an increase in the short term and a decrease in values will most likely be followed by another decrease in the short term. Figure 3 provides an example of a persistent time series and its estimated Hurst exponent.

Figure 3: A persistent time series (H = 0.95).
Figure 3: A persistent time series (H = 0.95).

The plot shows the intra-day tick level data for an NYSE traded fund. The Hurst exponent was estimated to be 0.95, which indicates a persistent time series. A Hurst exponent value between 0.5 and 1.0 indicates persistent behavior; the larger the H value the stronger the trend.

Conclusion

The Hurst exponent is a useful statistical method for inferring the properties of a time series without making assumptions about stationarity. It is most useful when used in conjunction with other techniques, and has been applied in a wide range of industries. For example the Hurst exponent is paired with technical indicators to make decisions about trading securities in financial markets; and it is used extensively in the healthcare industry, where it is paired with machine-learning techniques to monitor EEG signals. The Hurst exponent can even be applied in ecology, where it is used to model populations.

Subir Mansukhani is an innovation lead analyst with Mu Sigma.

REFERENCES

  1. H.E. Hurst, 1951, “Long-term storage of reservoirs: an experimental study,” Transactions of the American Society of Civil Engineers, Vol. 116, pp. 770-799.
  2. Bo Qian, Khaled Rasheed, 2004, “Hurst Exponent and financial market predictability,” IASTED conference on “Financial Engineering and Applications”(FEA 2004), pp. 203-209,
  3. Mandelbrot, Benoit B., 2004, “The (Mis)Behavior of Markets, A Fractal View of Risk, Ruin and Reward,” Basic Books, 2004.

business analytics news and articles



Headlines

Report: One in five cloud-based user accounts may be fake

According to the Q2 2018 DataVisor Fraud Index Report, more than one in five user accounts set up through cloud service providers may be fraudulent. The report, based on information gathered between April and June, analyzes 1.1 billion active user accounts, 1.5 million email domains, 231,000 device types and 562 cloud hosting providers and data centers, among other indicators. Read more →

When managers respond to online critics, more negative reviews ensue

A new study in the INFORMS journal Marketing Science found that when managers respond to online reviews it’s possible that those responses could actually stimulate additional reviewing activity and an increased number of negative reviews. The study, “Channels of Impact: User Reviews When Quality is Dynamic and Managers Respond,” is authored by Judith Chevalier of the Yale School of Management and NBER, Yaniv Dover of the Hebrew University of Jerusalem and Dina Mayzlin of the Marshal School of Business at the University of Southern California. Read more →

IE student designs software to optimize snow removal at Penn State

It is well known among the State College and Penn State communities that it takes a lot for university officials to shut the campus down after a major snowfall. In fact, since 2010, the University Park campus has been shut down just three full days due to snowfall. Much to the chagrin of students – and faculty and staff – the snow day at Penn State may just have become even more elusive, thanks to software developed by recent industrial engineering graduate Achal Goel. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden

INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.

OTHER EVENTS

Applied AI & Machine Learning | Comprehensive
Dec. 3, 2018 (live online)


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.