Share with your friends










Submit

Analytics Magazine

Software Survey: Decision analysis

Biennial survey demonstrates continuous advancement of vital tools for decision-makers, managers and analysts.

Justin AmoyalBy Justin Amoyal

In 2018, the introduction and rapid adoption of once futuristic technologies has become the norm, unfortunately and frequently leaving the numerous hours of design, testing, fabrication and analysis – most importantly decision analysis – without much or any consideration. Prior to becoming realities, these technologies were items of imagination, so much so that they were utilized in movies to add to the “futuristic” element. Going back to “Space Odyssey” (2001), Facetime-like video and Siri-like artificial intelligence were introduced. Facebook took a page out of “The Minority Report” (2002) by using individual’s interests and history to drive personalized advertisements. Perhaps the most important technological advancement of our time, predicted in “The Net” (1995), was online pizza ordering and delivery [1].

These technologies have driven the development of new fields and domains to study, and implicitly with them, experts, models, data and analytics to continuously advance them. Yet, it is rare that a product or system today solely requires the expertise of only one individual or domain. The Encyclopedia of Information Science and Technology expands on this by explaining a core problem is bringing multiple professional communities together: “More specifically, to design effective serious games, the expertise of educators, content experts, and learning experts will need to be utilized on top of all other experts who are typically involved in game design” [2].The complexity inherent with the desired performance of today’s systems requires the combined expertise of many, thus continuously growing the need for collaboration amongst multiple experts when attempting to make decisions.

Project teams frequently struggle with optimally combining the inputs from stakeholders, experts and models to reach decisions. It is typical for the model builders to side with the model results, the experts to believe in their research, and the stakeholders to strive to advance their portion of the project or their leadership’s interests. What is the remedy to effectively managing these conflicting objectives? In addition to the complexities associated with the problem itself, “an effective decision analyst must understand the challenges of decision-making in organizations, the mathematical foundations of decision analysis, and the soft skills required to work with the decision makers, stakeholders, and experts” [3].

Project teams frequently struggle with optimally combining the inputs from stakeholders, experts and models to reach decisions. Source. ThinkStock

Project teams frequently struggle with optimally combining the inputs from stakeholders, experts and models to reach decisions.
Source. ThinkStock

What tools can we use as thought leaders, decision-makers, managers and analysts to define these decision frameworks in our own projects? The results of this year’s biennial decision analysis software package survey are meant to assist in providing the tools and numerous capabilities available to us.

The Survey

In line with previous years, the 2018 survey was made available via an online questionnaire sent to vendors who previously participated or those recommended by staff. Vendors who did not respond by the deadline can still provide their software’s information by submitting a questionnaire and it will be added to the online version of the survey. Just as in previous years, vendor responses are provided verbatim and do not imply quality or cost effectiveness. Rather, the list serves to raise awareness of the variety of tools available. Approximately 60 multiple choice and short answer questions made up this year’s questionnaire. The primary focus was to define the current capabilities and any updates or advancements associated with each software package, enabling a high-level perspective on decision analysis software capabilities to our audience.

The primary goal of the software survey was to ask questions focused on the decision analysis capabilities of each tool. These include but are not limited to multiple competing objectives, probabilistic dependencies, stakeholder collaboration, value functions, strategy tables, elicitation and decision algorithms. Moving forward, vendors were asked questions to define the usability of their applications. To do so, they provided feedback on their software interfaces, visualization capabilities, model and data protection, user interfaces, and other usability related features. Types of training offered, as well as the recognized certifications associated with each software tool, were also investigated in the 2018 survey.

2018 Results

This year’s survey includes 28 software packages from 19 different vendors, domestic and international, who responded by the deadline. Pricing for licenses varied, ranging from free products to a max of $100,000, depending on the user/license type and elements of the software package. Specific industries and markets for these software applications are widespread, including defense (military, aerospace and shipbuilding) healthcare (pharma, hospitals and clinics), energy-related domains (petroleum and nuclear energy), as well as many others.

Decision analysis capabilities: Each of the decision analysis-focused capabilities proved to be available by at least one of the software tools reviewed. Uncertainty existed in every tool, yet implementation likely differed within the applications, and it is therefore worth investigating to find the one best fit for an individual project. Unfortunately, there were two minimally available features, an evidential reasoning (e.g., Bayesian belief networks) capability and portfolio decision-making, which do not exist in 79 percent and 48 percent, respectively, of the tools reviewed. Value functions/scores and risk tolerance existed in 75 percent of the tools, while strategy tables were available in 43 percent of them. Seven of the tools offered AHP, while 13 offered MODA/MAUT.

Usability: Usability features were fairly consistent among the vendor responses. XML integration and API (embedded decision support system) proved to be the capabilities most limited. Yet the inclusion of an API has increased since 2016 from 50 percent of the tools to 68 percent this year. Only eight of the tools offered simultaneous data input and viewing.

When considering data visualization, it is difficult to rely solely on the decision analysis software tool. Inherent in the name, these are decision analysis tools and therefore they provide the ability to output decision analysis-related visualizations such as graphical sensitivity analysis, expected value tornado diagrams, decision trees and influence diagrams, but stakeholders and decision-makers are usually not decision analysts. Thus, it is important to consider the exporting capabilities of these tools, particularly related to the export of data which existed in 86 percent of the tools. Exporting the data provides the ability to develop advanced visualizations, customizable to the desires and experience of project stakeholders, which may better convey results than the standard visualization existing within the software tools.

Licensing and training: The majority of the vendors (80 percent) reported that a limited use (run-time) or demo version of the tool is available. Nearly all of the products are available for purchase for educational (96 percent) and/or commercial (100 percent) use. As expected, many of these tools are offered free or at a discounted rate for educational use. The number of tools offering enhanced/high performance capabilities with these software packages showed a significant increase since 2016, from 35 percent to 75 percent this year.

Pricing for these advanced features varies significantly, typically based on the client’s needs and options selected. Multiple options for training were offered in relation to the software tools, mostly offered by the vendors themselves (79 percent). Online resources including training, tutorials and open source collaboration was offered for 75 percent of the tools, a nearly 25 percent increase since 2016.

Conclusion

This year’s decision analysis software survey demonstrates that software capabilities continue to advance, enabling users to conduct decision analysis regardless of industry, interfacing and elicitation needs, and desired visualizations or operating systems. When solving difficult problems, it is important to understand the resources available, interfacing requirements, desired decision analysis techniques to be utilized, required outputs and other problem-specific elements to select the tool that will best fit the analysis.

It is no surprise that every tool reviewed in this survey will not be the “perfect fit” for each decision analysis problem. Thus, the percentages mentioned earlier associated with nonexistent features is not detrimental to a tool’s potential performance for a specific project. A project on a tight timeline may not have the opportunity for iterative analysis; therefore, a tool offering this capability would provide no additional value.

In closing, this year’s software survey includes an impressive list of applications available to decision professionals, and the survey results can assist professionals in selecting the best decision analysis package for their particular needs.

Justin Amoyal (jamoyal@innovativedecisions.com) is an analyst with Innovative Decisions, Inc., and a member of INFORMS. A version of this article appeared in OR/MS Today, the membership magazine of INFORMS.

References

  1. Pahle, Rebecca, 2016, “15 Movies That Predicted the Future,” Mental Floss, Sept. 15, mentalfloss.com/article/86080/15-movies-predicted-future.
  2. “Gaming,” 2015, Encyclopedia of Information Science and Technology, Information Science Reference, pp. 3,291-3,291.
  3. Parnell, Gregory S., and Bresnick, Terry A, 2013, “Introduction to Decision Analytics” in “Handbook of Decision Analysis” (by Parnell, Bresnick, Steven N. Tani and Eric R. Johnson), John Wiley & Sons.

Survey directory and results

For a directory of decision analysis software vendors who participated in this year’s survey and the complete survey results, click here.

Analytics data science news articles

Related Posts

  • 69
    Fifty years have elapsed since the founding of the field of decision analysis by Howard Raiffa and Ron Howard. 2016 is not only a milestone year due to the anniversary but also because it marks the passing of one of the founders, Howard Raiffa. Such significant events make this year…
    Tags: decision, analysis, survey, tools, software
  • 39
    January/February Cybersecurity: new threats, new solutions The IOT and related, hidden security risks Can analytics save U.S. healthcare system? March/April Supply chain advances and solutions Software survey: vehicle routing Capitalizing on AI & machine learning May/June Social media, marketing & analytics Real-time customer personalization Next generation revenue management July/August Software…
    Tags: software, survey, decision, analysis
  • 39
     NEW: Forecasting 6/9/16   Vehicle Routing - 2/17/18 Simulation - 10/8/17 Linear Programming - 6/13/17 Statistical - 2/9/17 Decision Analysis - 10/10/16 Spreadsheet Add-ins - 8/13/10 Supply Chain Management - 6/14/03 Nonlinear Programming - 6/2/98 INFORMS publishes a number of software surveys each year in conjunction with one of its…
    Tags: software, decision, analysis, year
  • 39
    FEATURES Welcome to ‘worksocial’ world By Samir Gulati New approach, technology blends data, process and collaboration for better, faster decision-making. How to pick a business partner By David Zakkam and Deepinder Singh Dhingra Ten things to consider when evaluating analytics and decision sciences partners. Big data, analytics and elections By…
    Tags: analysis, decision, software, survey, tool
  • 32
    We were repeatedly reminded several times last year that variability can confound statistical predictions and unlikely events do occur. Upsets in sports and politics are always news, since having the underdog beat the “sure thing” is surprising and noteworthy. What is exciting in sports is unexpected in politics, since we…
    Tags: software, analysis, survey

Headlines

Using machine learning and optimization to improve refugee integration

Andrew C. Trapp, a professor at the Foisie Business School at Worcester Polytechnic Institute (WPI), received a $320,000 National Science Foundation (NSF) grant to develop a computational tool to help humanitarian aid organizations significantly improve refugees’ chances of successfully resettling and integrating into a new country. Built upon ongoing work with an international team of computer scientists and economists, the tool integrates machine learning and optimization algorithms, along with complex computation of data, to match refugees to communities where they will find appropriate resources, including employment opportunities. Read more →

Gartner releases Healthcare Supply Chain Top 25 rankings

Gartner, Inc. has released its 10th annual Healthcare Supply Chain Top 25 ranking. The rankings recognize organizations across the healthcare value chain that demonstrate leadership in improving human life at sustainable costs. “Healthcare supply chains today face a multitude of challenges: increasing cost pressures and patient expectations, as well as the need to keep up with rapid technology advancement, to name just a few,” says Stephen Meyer, senior director at Gartner. Read more →

Meet CIMON, the first AI-powered astronaut assistant

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, made its debut aboard the International Space Station. The ISS’s newest crew member, developed and built in Germany, was called into action on Nov. 15 with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.

OTHER EVENTS

Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.