Software Survey: Decision Analysis
Decision tools continue to evolve, providing analysts with more horsepower to transform increasingly vast amounts of information into decision advantage.
By William M. Patchak
As the U.S. Men’s National Soccer Team’s dreams for World Cup victory ended this summer in a close loss to Belgium, pundits debated what areas of improvement were most needed for the team’s future success. “They need more possession time with the ball,” one stated. “Yes, but it has to be possession with purpose,” another countered. In fact, this statement probably rings true for many of today’s decision professionals.
After all, the world is full of information. An individual’s access to information continues to increase beyond what few could have ever imagined. Already, there is great anticipation for wearable digital devices capable of streaming it directly to our wrists or eyeglasses. At the same time, headlines read of fierce debates over the benefits vs. privacy risks of companies collecting it on their customers. Yet for all of the attention paid to devices and methods for acquiring “big data,” so little focus is placed on the most important challenge – the seam between data and the decisions it can enable.
How does one go beyond simply “possessing” the data? How does one leverage the vast and varied amount available to provide meaningful decision support? In the last decision analysis survey in 2012, I touched on how decision analysts have a wide range of software tools available to support their mission “[to evaluate] complex alternatives in the light of uncertainty, value preferences and risk preferences,” as Dennis Buede defined it in the first survey 21 years ago.  This year’s list of decision analysis software packages reinforces how decision tools continue to evolve and provide analysts with more and more horsepower to transform the increasingly vast amounts of available information into decision advantage.
In terms of its approach and collection methods, this year’s survey did not stray from previous iterations. An online questionnaire was provided to vendors based on previous participation or the staff’s knowledge of a new product. Vendors who did not receive the original questionnaire can still provide their software’s information to the survey results online by contacting Patton McGinley (email@example.com). Just as in 2012 and previous years, this publication provides the vendor responses verbatim and does not intend for the results to imply quality or cost effectiveness. Rather, the list serves to raise awareness of the variety of tools available.
In all, this year’s survey features 38 software packages from a total of 21 vendors, with some vendors listing multiple tools or multiple versions of the same tool. Eleven vendors from 2012 did not participate this year, but seven new vendors have joined the response list. And while some software packages are listed for the first time, many from the 2012 survey have returned, albeit with some new features.
As was the case with previous editions of the survey, this year’s results (see sidebar) reflect a diverse group of vendors and prices. Along with the United States, companies from the United Kingdom, Sweden, Belgium, Finland and Canada are represented. Meanwhile, prices for the software packages range from under $20 to several thousand dollars, depending on the type of license and the nature of the package. And as in previous survey editions, use examples range throughout commercial and government industries to include energy, finance, healthcare and defense.
Focusing on updated features from 2012, many vendors report improvements to user interfaces in addition to new technical functions such as additional probability distributions and interfaces with Microsoft products (e.g., Excel). Regarding a topic highlighted in the 2012 introductory article, this year’s list features six new Web implementations: three as new features from returning packages and three from packages submitting to the survey for the first time. While not all software tools will (or should) offer Web implementations, the change is worth noting because it may indicate the presence of a trend likely to continue in the future.
In both 2010 and 2012, the topics of “built-in coaching” and classroom vs. online training were discussed. This year’s proportion of software packages offering online training increased by 10 percent (from 45 percent to 55 percent) with a corresponding 13 percent decrease in classroom training. While some of this change is due to certain vendors not returning from 2012, several packages now claim to offer online training for the first time. Indeed, the decision analysis community may soon find the will and capability to provide what Don Buckshaw wondered would be possible in his analysis of the 2010 survey: built-in coaching that allows “a novice [to be] confident that their models are producing sensible results.” 
Beyond such noticeable swings as training options, the small number of entries and changing group of respondents over the years make it impossible to perform any kind of reliable statistical analysis on the data set. More importantly, to attempt such would be bad science in a survey and article highlighting the need for good science. However, certain high-level observations can be made of this year’s respondent group compared to their predecessors. While the total number of software packages did decrease, there were several features that now make up a larger percentage of the respondent pool than before. They include multiple stakeholder collaboration (71 percent in 2014), risk preference (66 percent in 2014) and selecting a best option using multiple competing objectives (89 percent in 2014). These three attributes are worth flagging because, being associated largely with the “soft skills” of eliciting and prioritizing objectives and risk preferences, they can be overlooked in a world where data itself is often viewed as the end solution.
Bridging the Gap between Data and Decisions
Indeed, these three decision support features – stakeholder group collaboration, risk preferences and multiple objective analysis – are just some of the techniques that help to bridge the gap between information (i.e., data) and informed decision-making. A quick survey of recent literature indicates that the need to do so is readily apparent. According to research by the Economist Intelligence Unit (EIU) and PricewaterhouseCoopers (PWC), “experience and intuition, and data and analysis, are not mutually exclusive. The challenge for business is how best to marry the two … even the largest data set cannot be relied upon to make an effective big decision without human involvement.”  The same study also found that executives were skeptical of how data and analytics can assist big decisions, especially with regard to emerging markets.
In fact, these “big decisions” (i.e., more strategic level problems) are where decision-makers themselves are often unclear of their risk preferences and where data insights alone may not lead to a clear choice of alternatives for meeting their objectives. As opposed to operational level decisions that can be informed more directly by descriptive types of data analytics, these strategic problems often require decision professionals to meet their customers halfway between the data and the decision. They require an analyst not only to accurately interpret the data available, but to also demonstrate how it can illuminate a customer’s understanding of their own preferences and objectives, which until that point were not readily apparent.
So what is the decision analysis community to do in challenging environments of strategic decision-making? This survey’s list of software products provides a great starting point. Ultimately, however, software cannot do it alone – decision professionals bear the ultimate responsibility. As strategic consultant Dhiraj Rajaram explained in his October 2013 article in Analytics magazine:
“Leveraging data effectively to enable better decisions requires more than just data sciences… In the real world, however, not all business problems are clearly defined. Many of these problems start off muddy. To help solve them, one needs to understand and appreciate the business context. It requires an interdisciplinary approach consisting of several different skills: business, applied math, technology and behavioral sciences.” 
Rajaram outlines a scope of skills that decision professionals must rely upon in knowing how to utilize available software tools, knowledge of a customer’s industry and preferences, and interaction with the customers themselves. Bringing this vast and varied array of attributes to the problem, analysts can provide true added-value in helping customers find new ways to leverage available data.
Until Next Time
Decision-makers today continue to face a range of decision types: from operational to strategic, from evidence-driven to those that require the marriage of evidence with possibilities-based analysis. As in previous years, the software packages listed in the 2014 survey largely reflect this range in uses and offer decision professionals a true spectrum of toolsets to provide their clients with decision advantage. In both those occasions where the analysis of data itself can lead to decision insight and those where input from decision-makers themselves must play a role, software continues to evolve to meet the needs of decision professionals.
Already there are indications that data-driven decision support has made inroads, albeit slowly. According to Harvard Business Review in December 2013, “those that consistently use data to guide their decision making are few and far between. The exceptions, companies …[with] a culture of evidence-based decision making.” 
The challenge remains for decision professionals to expand the culture of evidence-based decision-making to more strategic applications, and to help bring in the mind-sets and preferences of key decision-makers to meet the data. Where hard problems persist, this year’s list of software packages provides a sample of tools available to help decision professionals bridge that continuing gap between data and insightful decisions.
William M. Patchak (firstname.lastname@example.org) is an analyst with Analytic Solutions Group, a management consulting firm specializing in data analytics and visualization, systems architecture and systems engineering, decision analysis and operations research, and modeling and simulation. He is a member of INFORMS.
- Buede, Dennis, “Decision Analysis Software: Aiding the Development of Insight,” OR/MS Today, April 1993.
- Buckshaw, Don, “Decision Analysis Software Survey,” OR/MS Today, October 2010.
- Economist Intelligence Unit and PricewaterhouseCoopers, “Gut & gigabytes: Capitalising on the art & science in decision-making,” PricewaterhouseCoopers, September 2014.
- Rajaram, Dhiraj, “Why some data scientists should really be called decision scientists,” Analytics magazine, October 2013.
- Ross, Jeanne W., Cynthia M. Beath, and Anne Quaadgras, “You May Not Need Big Data After All,” Harvard Business Review, December 2013.
Survey directory & data
To view the directory of decision analysis software vendors, software products and survey results, click here.