Decision Analysis Survey
Past, present and future of dynamic software emphasizes continuous improvement of vital analytics tool.
By Samantha Oleson
Fifty years have elapsed since the founding of the field of decision analysis by Howard Raiffa  and Ron Howard . 2016 is not only a milestone year due to the anniversary but also because it marks the passing of one of the founders, Howard Raiffa. Such significant events make this year a time for reflection. Seasoned decision analysts can celebrate their contributions to the advancement of the field, while young operations research (O.R.) and analytics professionals should take this time to imagine how their current and future contributions will shape the next 50 years of the field.
In a 1988 paper on the state of the field of decision analysis, Ron Howard wrote, “the accomplishments and promise of the field are impressive” and improvements to the “procedures for formulating, eliciting, evaluating, and appraising the decision problem” occur every day. He further explained that despite the improvements, decision analysis has “not [yet] become commonplace even in very important decisions.” However, he believed that as of 1988, “decision analysis [was] poised for a breakthrough in its usefulness to human beings” that would be achieved in part using “[computer-based] intelligent decision systems … to provide the benefits of decision analysis on a broader scale than ever before” .
Now, 30 years after the 1988 article, we can agree with Ron Howard that decision analysis has accomplished much since its founding. He was also correct in predicting that technology would play an ever-increasing role in the decision analysis field and that more contributions are still to come. Although a very difficult task, it is possible to identify several of these contributions as defining achievements for the first 50 years of the field of decision analysis. These include:
- establishment of university programs and classes focused on decision analysis,
- adoption of decision analysis techniques and principles in everyday business practices, and
- integration of computing methods and tools.
During a 2011 meeting of members of the Decision Analysis Society of INFORMS, leading decision analysts contemplated the future of the field. Their conclusions mirrored Ron Howard’s 1988 predictions in that they identified two factors that would characterize the future of decision analysis: 1) collaboration with other fields of study and, 2) advances in technology . Eric Horvitz, managing director of Microsoft Research’s Redmond lab, cited the role of “decision-theoretic ideas” in the development of artificial intelligence as a prime example of the significant contributions the field of decision analysis has made in the past and will make in the future. He went on to conclude that, “There is great opportunity for more interaction between the communities … to date the surface has only been scratched” . Many attendees agreed that future work in decision analysis will not only prove the practice to be indispensable to the field of computer science but also to “such realms as healthcare, climate change, energy and national security” .
Now, standing on the threshold of the next 50 years of decision analysis we must ask ourselves, “How can we make the visions of our founders and current leaders a reality?” Helping to tackle the present and future challenges of inter-field collaboration and integration of advanced computing methods are the vendors that develop the decision analysis tools used by the O.R. community today.
This year’s software survey seeks to catalog these tools and the features they offer to practitioners. The goal is to help readers to use these tools to continue to further spread the use of the field of decision analysis.
This 2016 software survey assists readers in evaluating the featured decision analysis software products in three categories: 1) decision analysis applications, 2) usability features, and 3) licensing and training options. Decision analysis applications examine the analytical uses of the tools, as well as features offered for elicitation of decision problem components. Usability features highlight available features that make the tools user-friendly, compatible with other software and operating systems, and useful in communicating results. Since the 2014 survey, we added questions to this category to more closely examine this area. The final category, licensing and training options, asks for the options provided by the vendors for purchasing and achieving proficiency with the products.
The approach and collection method of this year’s survey did not differ from previous years. Vendor representatives completed an online questionnaire consisting of approximately 60 questions. Vendors who completed the 2016 survey include those who participated in previous surveys or those who recently came to the attention of Lionheart Publishing staff. Vendors who have not yet participated in the 2016 decision analysis software survey may submit details of their product to the online version of the survey by filling out the questionnaire available at http://lionhrtpub.com/ancill/dassurvey.shtml.
Results of the 2016 software survey (URL) are presented verbatim. The results do not necessarily imply quality or cost effectiveness, but rather to provide a detailed catalogue of possible decision analysis tools available on the market today.
The 2016 software survey features 29 software packages from 19 vendors. Companies from the United States, United Kingdom, Belgium, Finland, Canada, New Zealand and Sweden are represented in this year’s survey. These companies provide their decision analysis packages to a variety of industries, with the healthcare, defense, energy and mining industries reported most frequently. While not all of the vendors who participated in 2014 responded this year, first-time responders submitted information for eight software packages. Three of these new responses were for products just launched in 2016. The following sections provide a brief overview of the results of each of the three survey categories:
Decision analysis applications: Vendors most frequently reported that their products could be applied to decision problems involving multiple competing objectives and uncertainty. A smaller number of products offered features for risk tolerance, sequential decision-making and evidential reasoning. Other features reported by vendors that were not specifically queried in the survey included Monte Carlo simulation, trade-off analysis, group value elicitation, game theory and decision framing. Vendors reported that more than 75 percent of tools could be used for elicitation of value functions/scores and model structure. Decision analysis features less frequently offered by the products included elicitation of probabilities, criteria/attribute weighting and the value of imperfect information.
Usability features: Not surprisingly, most of the products provide their users with common usability features such as the ability to import and export components, document model structure and/or judgments with text and display analytical results graphically. Additionally, vendor responses show that nearly 90 percent of the decision analysis tools featured are capable of interfacing with other software, and approximately 50 percent have XML and/or API features. Thirteen of the 29 packages are offered as web implementations. Of the 13, three are veteran products providing web-based access as a new feature and two are software just released in 2016.
Licensing and training: Most of the responding vendors reported that their products can be purchased for either educational or commercial use. Nearly 35 percent of the products offer an enhanced or high-performance version of the tool. Many of the products are offered free or at a discounted rate for educational use, while the price of enhanced or high-performance and commercial versions of the tools vary depending on the licensing plan. All but five vendors reported that training programs are available to users either through the vendor itself or through a third party. Fifteen of these vendors offer web-based training, two of which are returning products that did not previously offer web-based training.
Current and Future Trends
Although it is difficult to conduct detailed statistical analysis of the data set due to the small sample size and changes in the list of participants year to year, it is possible to identify several trends:
Collaboration: With each year of the survey, vendors report the expanded capability of their tools to provide collaborative decision analysis solutions. Today’s tools not only feature group collaboration capabilities such as group elicitation, and simultaneous data input and viewing, but also features that lay the groundwork for inter-field collaboration. Compatibility with other software and/or XML and API features make decision analysis techniques more readily accessible to other industries and fields of study. The forward-thinking development and continuous improvement of decision analysis tools will help users to realize the collaborative future envisioned by both Ron Howard in 1988 and by the 2011 workshop attendees.
Web implementation and cloud computing: Continuing the trend from 2014, a growing number of vendors offer web-based implementations of their products and web-based training and support. In fact, two of the respondents this year specifically stated that their packages are cloud-based products. With the increasing use of “the cloud,” it is likely that products will be offered in this environment more and more frequently.
Visualization: Improvements in computing technology have made it easier than ever to interact with the decision space through the data and values that define it. With this in mind, this year’s survey introduced new questions that examine the communication and visualization features offered by the various products. Vendors reported that about 75 percent of tools have customizable visualization features that allow users to perform actions such as drag-and-drop chart development and manipulation, or color and formatting selection. As for decision space exploration using interactive graphics, displays, or interfaces, about 70 percent of the products surveyed had this capability. While visualization and communication may not be the primary focus of tools used for decision analysis, the number of vendors reporting significant updates to the reporting, graphing and formatting features of their tools suggest they understand that there will be a strong need for these features in years to come.
In 2016, we not only celebrate the accomplishments of Ron Howard and Howard Raiffa but also the accomplishments of the many decision analysts who used their founding concepts to expand the field even further. As a young decision analyst, it is difficult to imagine that the founders left much room for further innovation. How are we supposed to continue to advance the field over the next 50 years?
Leading voices in decision analysis say there is room to expand through collaboration with other fields and through the application of advanced computing methods, so we must focus our energies in these areas. There is much uncertainty associated with the roles we will play in the future of decision analysis, however, this is the type of problem we have been well trained to overcome. It’s time to leap into the next 50 years.
Samantha N. Oleson (firstname.lastname@example.org) is an analyst with Innovative Decisions, Inc., an analytics consulting firm specializing in decision and risk analysis, operations research and systems engineering.
A version of this article appeared in OR/MS Today, the membership magazine of the Institute for Operations Research and the Management Sciences (INFORMS).
1. Gavel, Doug, 2016, “Harvard Remembers Howard Raiffa,” Harvard Gazette, News.harvard.edu, July 11, 2016. Web access: Sept. 7, 2016.
2. Stanford University, 2016, “Profiles: Ronald Howard,” accessed Sept. 9, 2016.
3. Howard, Ronald A., 1988, “Decision analysis: practice and promise,” Management Science, Vol. 34, No. 6, pp. 679-695.
4. Abbas, Ali, 2012, “Decision analysis: past, present and future,” OR/MS Today, February 2012.
5. Horvitz, Eric, Abbas, Ali, 2012, “Decision Analysis Workshop.” Procedures of Decision Analysis Workshop, Palo Alto, Calif., OR/MS Today, February 2012 (print version).
Survey data & directory
- 54FEATURES Welcome to ‘worksocial’ world By Samir Gulati New approach, technology blends data, process and collaboration for better, faster decision-making. How to pick a business partner By David Zakkam and Deepinder Singh Dhingra Ten things to consider when evaluating analytics and decision sciences partners. Big data, analytics and elections By…
- 51We were repeatedly reminded several times last year that variability can confound statistical predictions and unlikely events do occur. Upsets in sports and politics are always news, since having the underdog beat the “sure thing” is surprising and noteworthy. What is exciting in sports is unexpected in politics, since we…
- 44NEW: Statistical 2/9/17 Decision Analysis - 10/10/16 Forecasting - 6/9/16 Vehicle Routing - 2/12/16 Simulation - 10/5/15 Linear Programming - 6/8/15 Spreadsheet Add-ins - 8/13/10 Supply Chain Management - 6/14/03 Nonlinear Programming - 6/2/98 Lionheart Publishing, Inc., publishes a number of software surveys each year in conjunction with one of…
- 39FEATURES Why large, once-successful companies fail By Gary Cokins The reasons are numerous, but the world’s problems are more complex than ever, creating the need for analytics. Actionable Analytics By Krishna Rupanagunta, Meena Anantha Padmanabhan and Vinay Mony How to convert thought into action and bridge the gap between analytics…
- 32Definitions are useful to the extent they serve a purpose. So, is defining “analytics” important? Yes and no. It’s not likely that we’ll ever arrive at a conclusive definition of analytics, but discussing what is and isn’t within its realm provides cohesive boundaries; boundaries that can help analytics emerge as…