Share with your friends


Analytics Magazine

Software Survey: Decision Analysis

November/December 2010

Software Survey: Decision Analysis

Visualization and options continue to imp[rove along with Web-based integration, but a wish list of “wants and needs” remain unmet.

By Don Buckshaw

Analytics Magazine D.A. directory and product listings
Analytics Magazine For a directory of the decision analysis software products referenced in this article and a side-by-side comparison of their capabilities, go to:
Analytics Magazine Analytics Magazine

In introducing the first decision analysis software survey for OR/MS Today ( in 1993, Dennis Buede wrote, “Decision analysis is the discipline of evaluating complex alternatives in the light of uncertainty, value preferences and risk preference” [1]. Decision analysis software has certainly changed since then. Computational horsepower and memory were critical concerns in 1993. Today, vendors are producing a richer set of analysis tools and better visualization and analysis options. The functionality of tools seems to increase each year, thanks to the integration of optimization, simulation, forecasting and data analysis techniques. Opportunities for Web-based collaboration and integration into business analytics have never been better.

The survey

OR/MS Today, the membership magazine of the Institute for Operations and the Management Sciences (INFORMS, www.informs/org), recently published its 10th decision analysis software survey, and, as before, it features some exciting new software packages along with some old favorites.

As in years past, vendors were directed to an online questionnaire to fill out and return. There are no judgments on the quality of the software – this is not a contest. Instead it is an opportunity for a company to describe its DA software to interested users in a way that is helpful for comparisons. The reader can quickly see ( surveys/das/das.html) a list of the capabilities of each package as reported by the vendor. Interested vendors who did not respond by the deadline have the opportunity to add their software package to the online version of the survey.

This year’s survey initially featured 36 decision analysis products from 24 vendors. For comparison purposes, there were 30 packages from 21 vendors in the 1994 survey. Six vendors or software packages from the 1994 survey appear in this year’s survey, including Logical Decisions, Lumina Decision Systems, Palisade, Banxia Software and Catalyze. One software package from 1994, DPL, is still around today, but is owned by a different company – Syncopation Software.

The responses

The software listed in this year’s survey seem to fall into four basic types with quite a bit of overlap. The first is software designed for problem structuring and brainstorming, often with no quantitative component. This includes scenario planning and mind mapping software. The second type of software provides a prioritized list or portfolio of options from a group and typically implements a decision method such as multiple objective decision analysis, multiple criteria decision-making, the analytic hierarchy process and expert systems. The third type of software analyzes single attribute decision-making with uncertainty and often involves influence diagrams and decision trees. The last type of software concentrates on uncertainties and probabilistic analysis and includes Monte-Carlo simulation and Bayesian belief nets. Some forecasting, optimization and general operations research analysis packages are also included. Of course, several packages combine these four ideas in varying degrees of detail.

Prices for software range from free to $10,000 for a professional version. Some Web-based decision analysis software not included in this survey can cost far in excess $10,000 for enterprise versions.

All of the software in the survey runs on the Windows operating system, and several also run on the Macintosh and Unix operating systems. Ten of the packages are Web implementations, up from six packages two years ago. This trend will likely continue. Almost all packages account for uncertainty and have methods to import and export the models; most can export their data to Microsoft Excel.

What I would like to see in 2012

Looking back over time, one feature of decision analysis software hasn’t evolved as fast as the analytic side – the support for the new analyst to properly structure and build models that make analytic sense. Back in 1994, Buede encouraged software developers “to explore structuring support because these tools will not be truly valuable until users are able to structure valid representations of their decision problem for analysis” [2]. In the 2002 survey, another survey author, Dan Maxwell, said, “All of the modeling techniques encoded in software have their own underlying axioms, assumptions  and limiting conditions. And, the risk of unwittingly creating an apparently elegant and informative model that violates these conditions — potentially misinforming decision-makers — has never been higher” [3]. It seems that software has not replaced the decision analyst, but it has simplified and enhanced the trained analyst’s capabilities to quickly and easily analyze very complex systems. Decision analysis software, however, does little to guide the new analyst along the path to success. The field of decision analysis is possibly unique in that the mathematical underpinnings are very simple, but the underlying assumptions and axioms that cause the model to have real meaning are often hard to understand. There has been some advancement in this area, especially in the areas of brainstorming value hierarchies and eliciting probabilities, but there is still much room for improvement.

As a decision analysis practitioner, I have the opportunity to review or participate in many different decision analysis projects. Recently, I observed a large organization struggle with a very difficult problem. This problem showed many of the traits of a complex decision problem: multiple stakeholders, multiple and conflicting objectives, uncertainty, portfolio of options, sequential decisions and interdependencies. In addition, the organization displayed a systemic inability to develop a process to solve this annual problem.

This year the organization implemented a new solution. Did they decide to take a value-focused modeling approach, create a decision-making process that accounted for conflicting objectives and enforced clarity of decisions? No. Instead they bought a Web-based software tool. Instead of discussing why stakeholders might have greatly different rationale for the weights of the decision variables, they can now quickly get consensus by averaging the opinions of many people over the Web. The result is that rather than having many confused and frustrated people in a room slowly making poor decisions, they now have many confused people in many different rooms making poor decisions at a faster pace than ever before! As Douglas Hubbard said, “An exercise that builds consensus to go down a completely disastrous path probably ensures only that the organization goes down the wrong path even faster” [4].

The lesson is obvious to decision analysis experts: decision analysis software is a tool that should be used to support smart analysis, not to replace it. Over the last few years, INFORMS introduced a Soft Skills Workshop at its annual business practice conference. The purpose of the workshop is to pass on the modeling tricks of the trade that practitioners have developed over the years to simplify and increase the accuracy of their models through smart, structured interactions with customers.

I would like to see software developers design their products in a way that can position both new and old analysts for success. Let’s automate some of these tricks of the trade so that the wisdom that has come so hard over the many years can be made available to another generation of decision analysts. Here are some ideas:

  1. Integrate brainstorming techniques, such as a virtual affinity diagramming process, with decision analytic tools. Include helpful hints for structuring that pop up to ensure that “ends,” not “means,” are being modeled, and that the objectives are mutually exclusive and collectively exhaustive.
  2. Ask for user input in a way that is easy to answer and helps calibrate the decision model with decisions that a person would realistically make. For elicitation, research has shown that humans are good at making comparative assessments (e.g., “I think A is better than B”) but poor at making ratio judgments (e.g., “I think A is five times better than B”) [5].
  3. Provide helpful hints while creating scales to remind the user that the low end of the scale does not include infeasible options (which would render an additive model useless), and remind the user to allow for feasible growth in the upper part of the measure to fully allow for the principles of value-focused thinking to be implemented.
  4. Create new ways to elicit swing weights that automatically form easy to answer preference questions based on the high and low ends of the scales. Also, integrate ways to display and explain weights, such as the swing weight matrix [6], so that weights have both a mathematical and a managerial justification.
  5. Create better displays to visually score alternatives that help ensure consistency across responses and avoid scale compression.
  6. Provide structured interview questions for probability distribution elicitation designed to identify and avoid decision biases, such as the anchoring effect, by first considering the highs and lows of the distribution.
  7. Encourage the use of value-focused thinking by identifying value before alternatives, structuring alternative generation around the value hierarchy, and determining ways to create better alternatives analytically or by creating hybrid alternatives.
  8. Provide better aggregation of group input to highlight disagreement among stakeholders, and if the model is sensitive to these disagreements, alert the user and recommend options (e.g., different preference sets, better definitions, stating assumptions or simply displaying the differences as value ranges).


The decision analysis software field is going strong with some old favorites and interesting new additions. Memory and computation needs are no longer a concern, and more distributed, Web-based implementations are coming to the fore. However, I would like to see more of an effort for software vendors to build in some form of coaching into their products so that even a novice can be confident that their models are producing sensible results. Finally, if you know of any software not represented in this survey, tell the vendor go to, fill out the online survey and the software will be added to the online version of the survey.

Don Buckshaw ( is a senior principal analyst with Innovative Decisions, Inc. a management consulting firm serving business and government clients and specializing in the disciplines of decision and risk analysis, operations research and systems engineering. He has more than 20 years of experience as an intelligence and operations research analyst.


  1. Buede, Dennis, “Decision Analysis Software: Aiding the Development of Insight,” OR/MS Today, April 1993.
  2. Buede, Dennis, “Decision Analysis Software: Aiding Insight II,” OR/MS Today, April 1994.
  3. Maxwell, Daniel, “Decision Analysis Software: Aiding Insight VI,” OR/MS Today, June 2002.
  4. Hubbard, D., “The Failure of Risk Management,” John Wiley and Sons, Hoboken New Jersey, 2009.
  5. Lai, S., “An Empirical Study of Equivalence Judgments vs. Ratio Judgments in Decision Analysis,” Decision Sciences, Vol. 32, No 2, Spring 2001, pp. 277-302.
  6. Parnell, G., Driscoll, P., & Henderson, D., editors, “Decision making for systems engineering and management,” Wiley Series in Systems Engineering. Wiley & Sons, 2008.



Former INFORMS President Cook named to U.S. Census committee

Tom Cook, a former president of INFORMS, a founding partner of Decision Analytics International and a member of the National Academy of Engineering, was recently named one of five new members of the U.S. Census Bureau’s Census Scientific Advisory Committee (CSAC). The committee meets twice a year to address policy, research and technical issues relating to a full range of Census Bureau programs and activities, including census tests, policies and operations. The CSAC will meet for its fall 2018 meeting at Census Bureau headquarters in Suitland, Md., Sept. 13-14. Read more →

Gartner identifies six barriers to becoming a digital business

As organizations continue to embrace digital transformation, they are finding that digital business is not as simple as buying the latest technology – it requires significant changes to culture and systems. A recent Gartner, Inc. survey found that only a small number of organizations have been able to successfully scale their digital initiatives beyond the experimentation and piloting stages. “The reality is that digital business demands different skills, working practices, organizational models and even cultures,” says Marcus Blosch, research vice president at Gartner. Read more →

Innovation and speculation drive stock market bubble activity

A group of data scientists conducted an in-depth analysis of major innovations and stock market bubbles from 1825 through 2000 and came away with novel takeaways of their own as they found some very distinctive patterns in the occurrence of bubbles over 175 years. The study authors detected bubbles in approximately 73 percent of the innovations they studied, revealing the close relationship between innovation and stock market bubbles. Read more →



INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden


Applied AI & Machine Learning | Comprehensive
Sept. 10-13, 17-20 and 24-25

Advancing the Analytics-Driven Organization
Sept. 17-20, 12-5 p.m. LIVE Online

The Analytics Clinic: Ensemble Models: Worth the Gains?
Sept. 20, 11 a.m.-12:30 p.m.

Predictive Analytics: Failure to Launch Webinar
Oct. 3, 11 a.m.

Advancing the Analytics-Driven Organization
Oct. 1-4, 12 p.m.-5 p.m.

Applied AI & Machine Learning | Comprehensive
Oct. 15-19, Washington, D.C.

Making Data Science Pay
Oct. 29 -30, 12 p.m.-5 p.m.


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to