Share with your friends










Submit

Analytics Magazine

Executive Edge: Executive briefing on simulation in strategic forecasting

May/June 2011

CLICK HERE TO GO TO THE DIGITAL VERSION OF THIS ARTICLE

Simulation forecasts have several important advantages over single point estimates.

Michael KubicaBy Michael Kubica

Any strategic forecast is by definition a representation of the future. Understanding that a single estimate of the future is not truly representative, Monte Carlo simulation is a powerful alternative to the “best estimate” forecast. Business leaders are becoming increasingly aware of the deficiencies inherent in traditional forecasting methods. And the past two decades have ushered in an explosion of tools to facilitate novice and expert alike in applying Monte Carlo simulation.

Though the growth and accessibility of these tools has been staggering, less prolific has been the adoption of the methodologies these powerful tools enable. Why? Part of the answer lies in a lack of understanding of what a simulation forecast is, what the relative merits and limitations are, and when it is most appropriate to consider using it. In this article I address these questions.

What is a Simulation Forecast?

In a traditional forecast, input assumptions are mathematically related to each other in a model. Based on these defined mathematical relationships, model outputs are calculated, such as market units sold, market share and revenue. The model may be simple or very elaborate. The defining characteristic is that inputs are defined as single point values, or “best estimates.”

This type of model has been the staple of business for many years. They can answer questions such as, “If all of our assumptions are perfectly accurate, we can expect …” However, experience has shown that all of the assumptions are not perfectly accurate. We are forecasters after all, not fortunetellers!

Simulation can remedy this problem. Instead of defining input variables as single point estimates, we define them as probability distributions representing the range of uncertainty associated with the variable being defined. These “ranged” variables are fed into the exact same forecast model. When the simulation model is run, we sample each input variable’s distribution thousands of times and relate each instance of the distribution samples within the forecast model structure. Because we have defined the inputs as uncertainties, the outputs represent all of these uncertainties in a simulation forecast. Instead of a single line on the graph, we may represent an infinite number of lines, bounded by the possibilities constrained by the input distributions. Of course, we summarize these probabilistic outputs according to the confidence intervals relevant to the decision at hand.

The Relative Merits and Limitations

Simulation forecasts have several important advantages over single point estimates. First, assuming that the input variables are representative of the full range of possible values along with best estimates where available, we have what may be referred to as a representative forecast. A representative forecast incorporates all currently available information, including uncertainty about future values. In this sense it is a truthful forecast.

A simulation forecast allows for an examination of both what is possible and how likely each of those possibilities are. We can examine the best estimate forecast in the context of the full range of possibility and discern true upside and downside risk. We gain these advantages without losing the ability to do specific scenario analyses. But now we can peer into the risk associated with achieving any defined scenario.

Simulation modeling does come at some cost, though. Rather than having a single input per assumption, you will define anywhere from one to four inputs, depending on the type of distribution being represented. This is because, in order to create a probability distribution to represent your uncertainty regarding the assumption, you will need to define the bounds of possibility for that variable (minimum, maximum) in addition to a best estimate, and potentially a “peakedness” variable.

This makes the model appear more complex and can make it seem more daunting to users (the truth is, the model itself has not changed from the point estimate, given that it was appropriately specified to begin with). This appearance of increased complexity can contribute to a “black box” perception among model consumers. Avoiding this issue is often as simple as explaining that the expanded input set is nothing more than representing the diligence that (hopefully) is going into formulating the best estimate in a traditional forecast, and leveraging all of this additional information to improve understanding and decision-making.

Simulation outputs cannot always be interpreted the same way as traditional forecast outputs. It is therefore prudent to hold an orientation meeting with model consumers to discuss how to interpret results and to address common misapplications of simulation outputs. While it is not necessary for users to understand the theory per se, it is important to avoid having them multiply percentiles together or misinterpreting what the probabilistic outputs mean. A small investment here can go a very long way in creating value from the forecasting process.

When Should Simulation be Considered as the Methodology of Choice?

I once attended a pharmaceutical portfolio management conference where I heard one of the speakers say (in the context of creating forecasts to drive portfolio analysis): “Simulation is OK, but you better be really sure you are right about your assumptions if you are going to use it!” I was astounded that such misinformation was coming from someone forwarded as an expert. Nothing could be further from the truth. The less certain you are about the assumptions and the more there is at stake based on the decisions being made from the model, the more appropriate and important simulation forecasting is. This is especially true if the cost associated with being wrong significantly exceeds the cost of the incremental resources to define that uncertainty.

In summary, simulation forecasting is a powerful methodology for understanding not just the possible future outcomes, but establishing a truthful representation of how likely any single scenario is within the range of possibilities. Because strategic (long-term) forecasting is inherently risky and driven by many uncertain variables, adding Monte Carlo simulation to your forecasting tool chest can create enormous value.

Michael Kubica is president of Applied Quantitative Sciences, Inc. Please send comments to mkubica@aqs-us.com.

CLICK HERE TO GO TO THE DIGITAL VERSION OF THIS ARTICLE

Analytics Blog

Electoral College put to the math test


With the campaign two months behind us and the inauguration of Donald Trump two days away, isn’t it time to put the 2016 U.S. presidential election to bed and focus on issues that have yet to be decided? Of course not.


Save

Headlines

Gartner: AI technologies to be pervasive in new software products

Market hype and growing interest in artificial intelligence (AI) are pushing established software vendors to introduce AI into their product strategy, creating considerable confusion in the process, according to Gartner, Inc. Analysts predict that by 2020, AI technologies will be virtually pervasive in almost every new software product and service. Read more →

Drone delivery: Professor develops solution to minimize delays in operations

When delivery companies like FedEx, Amazon and UPS launch drones to deliver packages in the near future, one Kennesaw State University computer science professor may be at the crux of solving one of its most complicated problems. Donghyun (David) Kim, assistant professor of computer science and an expert in computer algorithm optimization, is designing a fast-running algorithm to tackle simultaneous coordination problems among multiple delivery trucks and the drones launched from them. Read more →

Tech spending growth limited to about 5 percent through 2018

Forrester predicts U.S. business and government tech spending will continue to grow by 4.8 percent through 2017 and increase to 5.2 percent in 2018. While these forecasts are higher than Forrester’s projections following the 2016 presidential election, they are lower than the expected numbers from a year ago. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

Essential Practice Skills for High-Impact Analytics Projects
Sept. 26-27, Executive Conference Center, Arlington, Va.

Foundations of Modern Predictive Analytics
Oct. 2-3, VT Executive Briefing Center, Arlington, Va.

2017 INFORMS Annual Meeting
October 22-25, 2017, Houston

2017 Winter Simulation Conference (WSC 2017)
Dec. 3-6, 2017, Las Vegas

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.