So You Want To Be a Decision Analyst?
All it takes is the right mixture of analytic skill and insight, a big kit of tools and techniques, interpersonal skills and a little bit of luck.
By Daniel T. Maxwell
I have had the privilege of produstry for almost 20 years. Over that time I have found three things to be almost universally true when I walk in the door to meet with either a senior leader or their staff at the beginning of a project. First, the decision situation is important to the organization and is likely something that is very different from their prior experiences. Otherwise they would not be spending the money. Second, there are multiple stakeholders involved with many different perspectives, including competing demands for limited resources and differing areas of expertise relating to the decision situation they face. Third, the goals they are attempting to achieve, the information they are using to inform their decisions and the possible courses of action are: 1) poorly organized, 2) ambiguously defined, 3) uncertain, and 4) often conflicting.
Contributing to the solution rather than becoming part of the problem in situations like these requires a mixture of analytic skill and insight, some interpersonal skill and a little bit of luck. While the importance of luck cannot be underemphasized, a big kit of tools and techniques, some polite persistence and a lot of common sense go a long way in these applied settings. Fortunately, these things are exactly what a practically minded decision analyst or decision analytic perspective can bring to a project.
Keys To Success
Achieving success as a decision analyst is really quite simple. As the analyst you are responsible for developing a model and/or providing an analysis result that does four things. First, it clearly lays out the goals that the organization is attempting to achieve, and ideally it refines the goals into a set of quantitative measures. Second, the possible courses of action or decisions available for achieving those goals must be unambiguously specified. Third, the relevant facts and uncertainties or situational variables, which will influence the success or failure of a course of action during the execution, should be identified. Finally, the relationships between the goals, alternatives and situational variables should be identified, explored and communicated to the stakeholders in the organization(s) in a clear and understandable manner. Of course these simple parts don’t make a decision analysis project easy, and it does not mean that there isn’t tremendous complexity buried in the details associated with developing each of the parts. But, decision analysis as a discipline provides a process and a whole set of computational tools for illuminating all of these things for decision-makers.
|DECISION ANALYSIS: FOUR “DELIVERABLES”|
|• Focused set of goals and objectives
• Clearly defined alternatives
• Identification of key variables
• Illumination of relationships among variables
Generally, as decision analysts we help decision-makers by bringing some structure and proven methods to the de-cision-making process. There are many subtly different views of this process, like the one in Figure 1 , almost all of which lay out an iterative process that starts by clarifying the decision situation to be addressed, formulating a model of some sort, and then solving and exercising that model to provide insight for a decision-maker and stakeholders. We, in the decision analysis community, have many spirited debates about the nuances of process and alternative tools that might be applied. While these are all important for the evolution of our profession, they are rarely the issues that influence the success or failure of an applied decision analysis project.
The keys to success have much more to do with getting people to understand and communicate clearly about the substance of the situation they face. Decision analytic models and processes live in the shadows of successful applied projects. In fact, my biggest successes in terms of organizational contribution and impact are supported by models that are extremely simple or sometimes even set aside when it comes time for the executive decision session. Some analysts view these kinds of outcomes as failures, but there are often good reasons for setting the model aside at the end. There are important factors that cannot or should not be included in a model. Success is about informing and improving the organizational decision process — not replacing it.
Even with a structured process, in practice providing insight to decision-makers is an extremely challenging undertaking. Challenging, however, does not mean impossible. Many of the things decision analysts do in executing a project and the decision analysis process are very compatible with “best practices” for leading and managing large organizations.
First steps are extremely important to overall project success because they provide the foundation for the remainder of the effort and the analyst’s relationship with the decision-makers and stakeholders. Of course it is important to make good first impressions and listen carefully during the initial meetings. An analyst must listen to what is said, and pay attention to what isn’t said. Often, it is what isn’t said that is most illuminating for developing an understanding of the decision situation, especially the interpersonal and inter-organizational factors. Be careful to communicate only what the client needs to know to facilitate your participation in solving their problem effectively, not what you know about modeling and decision analysis.
Credibility with the client is usually more effectively built by demonstrating you understand and care about their problem, rather than pontificating about abstract decision analysis ideas. I recall one of my first decision analytic projects. I was fresh out of graduate school and certain that multi-objective decision analysis (MODA) models were going to save the world. I was called in to support a U.S. Army general (I was in the Army at the time) who I thought wanted some help understanding an analytic hierarchy process model that was providing some counterintuitive results. I thought I was providing real value when I described how rank reversal occurs and why MODA was a superior approach. Unfortunately, the person who built that AHP model was present and started a technical debate right in front of the general. We were both expeditiously dismissed as useless to the task at hand. It was a very humbling and important professional lesson. I now occasionally use AHP software if it is already in use and will speed the process of focusing everyone on penetrating the problem. All of the decision analytic techniques have strengths and weaknesses. The successful analyst knows what the weaknesses are and helps the client avoid them.
Another key to success is to involve the right people. Successful decision analysts recognize that solving complex problems is a team sport. Considering the complexity of the modern environment and technology, it is highly unlikely that any one person or a homogeneous team will possess the depth and breadth of knowledge and experience to develop a balanced understanding of a complex decision situation. Involving affected and knowledgeable people is also critical to getting “buy in” and energetic execution of the course of action that is recommended.
In constructing analysis project teams there are three types of talent the decision analyst should be looking for: 1) domain expertise, 2) scientific and academic expertise, and 3) applied science and engineering skills. Figure 2 provides some examples of the kinds of people to consider. Of course, if you are a senior leader or manager using these principles in your organization you can directly influence the team’s structure. If you are a staff analyst or consultant you will need to “sell” the benefits of the multi-disciplinary team, likely a little bit at a time. Finally, your team skill assessment should include a self-appraisal to identify your own weaknesses and an effort to obtain that talent to compensate for them.
Another extremely important team-building activity throughout the project is to identify and involve stakeholders in the decision situation. The reality of the modern government and business environment is that there is rarely a single “decision-maker,” especially for the big decisions. Different parts of an organization will have varying responsibilities. Often these differences result in competing goals and organizational tension. The successful decision analyst recognizes that this is expected, natural, and must be addressed as an integral part of the decision analysis process. Well-executed facilitation diffuses initially emotional differences and channels that energy into constructive dialogue about legitimate differences of perspective and knowledge. “Facilitation” in this case isn’t limited to hosting decision conference-style meetings. The appropriately immersed analyst should be engaged in e-mail dialogue, making conversation and productive time in the cafeteria.
It is also likely that new stakeholders will emerge as the project progresses. As much as possible the process should be inclusive. There will be an investment of time and resources as people and perspectives are added to the effort. There will probably be pressure not to slow down to pick up the stragglers. Inclusion, however, is the most reliable way to ensure that the creative, high-value course of action that is selected at the end of your study will be implemented and supported across the organization. The decision theoretic literature has fleshed out the idea of “flat maxima” . The practical implication of this idea is that the analysis is likely to identify a few very promising alternatives, and consensus coupled with energetic execution of any of those alternatives will be a bigger factor influencing outcomes than finding an “optimal” solution. So, work hard to be inclusive and obtain broad support.
Four Parts of Analysts
Now that we understand the critical importance of people, we need briefly revisit the four parts of the analysis. Of course first, we need to understand and develop a set of goals. Refining and articulating goals and objectives for a difficult decision is often hard. If you are told at the first meeting that the measures of success have been developed and “it was easy,” beware, as it is likely a poor model. In decision analysis values are operationalized into a set of fundamental objectives . Often a well-executed effort at identifying and refining the fundamental objectives will occupy a significant amount of time and effort. It will also identify new high value alternatives, which usually help the stakeholders develop a common frame of reference, and definitely provides a foundation upon which the remainder of the modeling and analysis efforts can be built upon. Keeney  presents an excellent discussion of fundamental objectives, desirable properties of objectives and techniques for their development. On the practical side, strategic plans, requirements documents, operational plans, existing repositories of metrics and prior studies are all sources that should be studied and used to stimulate discussion and build your organizational knowledge.
Successfully identifying the available courses of action is the next requirement. Decision situations fall into two general categories: individual decisions and portfolios. For the single decision we are attempting to identify the best single alternative and in the portfolio we are attempting to identify the highest valued set of actions or investments from a bigger list of possibilities. Parnell  provides an excellent discussion of the two classes of decision models along with some valuable modeling considerations. To say we are identifying a “course of action” significantly understates what actually happens in a good decision analysis project. Well-executed decision analysis creates courses of action for consideration throughout the process. They could be the result of collaborative brainstorming exercises, modification of known possible courses of action, or maybe just a great idea over lunch. There are nuances to single alternatives and portfolio decisions that can be important to capture during model development. For example, the course of action may actually be a set of sequential decisions or a more complicated decision strategy with multiple dependencies. There are many tools and techniques for representing and analyzing all classes of decisions; influence diagrams are a great way to model sequential decisions. Additionally, don’t be afraid to draw on other disciplines for useful techniques. The morphological box, from systems engineering is a very effective way to simplify representation of complicated multi-dimensional alternatives .
The third thing to provide is an illumination of the key situational variables. There are two classes of variables: 1) the inputs and logical consequences of a decision (e.g., resources consumed), and 2) key uncertainties that could impact the success of a course of action. (e.g. what actions might a competitor initiate?). The first type of variables can be very reliably predicted once they are identified and placed in context. The second type is more challenging because these are variables that will likely remain uncertain, possibly until the outcomes are determined or even longer. Doing this effectively requires the energetic and effective participation of those stakeholders with different perspectives. Accomplishing this as an analyst is largely a facilitation exercise. There are some great references from the facilitation community to draw on for ideas on how to approach these challenges .
The fourth and final part of the analysis is to help the decision-maker(s) and stakeholders to understand the relationships among the goals and objectives, alternative courses of action, and situational variables. This includes exploring those relationships for and with the client. The key to success here is to keep it simple. Techniques from the Systems Thinking community have proven very helpful for helping to structure the relationships and provide a foundation for quantitative modeling .
Once this is completed, all of the parts are in place to begin some serious quantitative modeling of the decision situation. In some cases a simple multi-attribute decision model is sufficient; in other cases where the situation has lots of complicated uncertainty, an influence diagram, a Bayes Net or even a simulation may be appropriate. Be especially sensitive at this stage to the client’s proclivities. You will likely be many steps in front of them about what is possible analytically, and too much too fast could undermine both near-term success and the opportunity for longer-term contribution.
Applied decision analysis can be an especially challenging and rewarding undertaking. It can test the limits of your analysis talents, your interpersonal skills and your persistence. It is truly where the rubber meets the road in operations research and analytics.
Dan Maxwell is the president of KadSci, LLC and the chief scientist for DeepMile Networks, LLC. In these roles he leads numerous decision analytic research efforts and provides applied decision analysis support to many senior government and corporate leaders and directs the design and development of analysis software. Maxwell holds a Ph.D. in Information Technology from George Mason University. He is an advisory member of the Military Operations Research Society (MORS) Board of Directors and is on the Editorial Board of CRC Press. He can be reached via e-mail at: email@example.com
1. Maxwell, D., 2006, “Improving Hard Decisions,” OR/MS Today, Vol.33, No. 6, December 2006.
2. Von Winterfeldt, D. & Edwards, W., 1986, “Decision Analysis and Behavioral Research,” Cambridge University Press, Cambridge.
3. Keeney, Ralph, 1992, “Value Focused Thinking,” Harvard University Press, Cambridge.
4. Parnell, G., 2007, Chapter 19, “Value-Focused Thinking Using Multiple Objective Decision Analysis,” in “Methods for Conducting Military Operational Analysis: Best Practices in Use Throughout the Department of Defense,” Military Operations Research Society, editors Andrew Loerch and Larry Rainey.
5. Buede, D., 1998, “The Engineering Design of Systems: Models and Methods,” Wiley-Interscience, New York.
6. For example, Wilkinson, M., 2004, “The Secrets of Facilitation,” Wiley, New York.
7. See Senge, P., 2006, “The Fifth Discipline: The Art and Practice of the Learning Organization,” Doubleday, New York.