Decision-making: Taming the data tidal wave
AHP combines business intelligence and data mining technologies with social aspects of human decision-making.
By Kevin Connor
Nothing changes the course of world history, public policy and business results like decision-making. Good decisions are based on an in-depth understanding of the benefits, opportunities, costs and risks inherent in our choices. To make the best choices, decision-makers need information. Those decisions are hampered by the growing tidal wave of undifferentiated information that is the sign of our times.
By some estimates, the amount of digital information is increasing tenfold every five years. We have reached an amazing point in human history: We are collecting so much information that it is impossible to know what we know. Hidden among the bits and bytes, encoded as zeros and ones, are trends, connections and insights that both define and shape our world.
How can businesses and government institutions find meaningful and actionable insights to inform decisions and improve performance?
This article looks at one way to uncover those trends, connections and insights: a mathematical principle called the Analytic Hierarchy Process (AHP). Developed in the 1970s, AHP is now being integrated into complicated decision-making processes by both public sector and private industry.
The Information Tidal Wave: Scope Creates Uncertainty
Some recent studies estimate that the average knowledge worker now spends two hours a day searching for information later deemed useless, while an estimated 80 percent of this wealth of digital information is unstructured. Couple these ideas with studies that suggest one-third of our decisions aren’t implemented and that half of those that are don’t meet their objectives.
We’re capturing more information than ever before, but we’re not really using it to our advantage. All of this leads to choices that seem more uncertain, increasingly complex and risky. With all this information at our disposal, we feel we should “know,” yet we don’t.
Group decision-making is difficult. Advances in technology to improve our use of data are evolving rapidly, but integration of data with the human and social elements of decision-making have continued to prove challenging. How can organizations bring together people, establish processes with the appropriate tools and technologies to facilitate efficient and effective decision-making in the world of voluminous data – all while honoring the intuition and judgment of their best minds?
Dealing with the cresting wave of information requires that we take a broader view of decision-making that brings together both the best information, subject to the best minds, through a process that integrates them. By blending the most relevant information with the wisdom and intuition of experts, our ability to tip the scales of uncertainty and risks toward benefit and opportunity can be greatly increased.
So, how might we find such a process that allows us to utilize the appropriate content we have and give context from our expertise and knowledge? Some of the best minds in the public sector and private industry depend on the AHP for surprising insights.
Before the Analytic Hierarchy Process: The Volatility of Group Decisions
Dr. Thomas Saaty developed the AHP at the Wharton School at the University of Pennsylvania in the 1970s, while working on nuclear non-proliferation negotiation strategies for the State Department. In aligning the perspectives of some of the world’s brightest economists, utility theorists, game theorists, scientists and lawyers, Dr. Saaty came to a realization: Although this group of brilliant people had tremendous intellectual gifts and knowledge, their challenge finding a coherent and aligned point of view was a social one.
The fear of not being heard or understanding how our contribution influences a choice can bring out the worst sides of our nature when interacting collaboratively to choose a course of action. Battling it out in decision-making often reduces collective discourse in organizations and societies from a fair, rational, respectful and transparent debate about what is best for “us” to an advocacy based, power driven, contemptuous and in the worst cases dishonest grab for what is best for “me.”
Whether we are buying a home or a car with our spouse and family, attempting to understand the most probable risks to national security, or choosing the next blockbuster drug to improve the health and well being of society, subtle and potentially corrosive threads can infect any decision. Our inability to see what others see and why, or that others’ perspectives may be valid, tends to more deeply entrench our own biases, and forces us to vehemently defend our position. Along the way, this approach destroys value and compromises our ability to achieve our goals.
How can any group, with its disparate and diverse views of the world, express what was important to each member in addressing the challenge at hand, and be able to see their collective and desired course of action?
AHP is a collaborative decision-making methodology that is used to structure and analyze complex and potentially volatile decisions. Based in fundamental principles of mathematics and psychology, it was developed for the purpose of synthesizing the judgments of a group of decision-makers and providing them a rational, transparent and collaborative way to express themselves and understand the totality of their interests, preferences and priorities.
The Inner Workings of AHP: Merging Data and Expert Judgment
The application of AHP follows a structured format. Let’s outline the key steps in the process:
Define relevant criteria: Decision-makers develop a hierarchy or tree of the relevant criteria, objectives or goals for their decision. The criteria are grouped in clusters from high-level categories at the top level to more specific sub-criteria that define those categories at the bottom.
Figure 1: Defining relevant criteria, objectives or goals.
These lowest level criteria are then described with ratings and measures that will be used to differentiate the value of options against these criteria. This step creates the scorecard that captures data and evaluations. These criteria can range from the purely quantitative factors such as net present value or miles per hour, to more subjective and qualitative factors such as strategic fit or risk.
Establish the weight of each criterion: It is important to establish the relative priorities of the criteria to be used for assessing the options. Having been identified by the group of decision-makers, the criteria are all at least relevant and of interest but clearly not equally important.
A group of decision-makers must determine which factors more strongly predict a valuable outcome in their decision given the context of their choices. By comparing each the criteria against each other using a pairwise comparison approach, decision-makers are forced to value the decision criteria head-to-head to deal only with the difference in each paired combination:
- Which is more important?
- By how much?
- A or B?, B or C?
And so on.
Mathematics can then determine how all the criteria relate amongst each other. By stating their preference openly, decision-makers can identify points of agreement and disagreement and have a framework for expressing information that only they may have that is influencing their position on some matter.
Figure 2: Sensitivity analysis clarifies decision alternatives.
Evaluate options based on these ratings: Decision-makers (or the subject matter experts that they entrust) must rate or evaluate the options using the quantitative or qualitative rating scales that were derived to measure how well the options reflect the priorities expressed in the criteria. This results in a relative prioritization or score for the various options that now includes the blended priorities of the decision-makers and reflects the impact of the known measures or metrics – as well as the judgment of the experts that are party to the decision.
Consider alternatives: The group may perform “sensitivity analysis” on the prioritization of the decision alternatives – for example, “what if criterion A was twice as important?” or “what if we didn’t consider criterion B?” This allows decision-makers to determine the robustness of a given course of action and understand the key drivers of the decision. Once this has been accomplished, the cost (dollars, time or even people) of each option can be added and any constraint in resources. The result is an optimized allocation of resources among the options using a cost/benefit analysis.
By bringing together the available data along with the judgments of decision-makers in a structured way, AHP helps decision-makers see an otherwise elusive, holistic picture of how their goals and understanding of the problem fit their choices.
Using the principles of comparative judgment, AHP allows each decision-maker to express his or her individual priorities amongst the key criteria that form the mental model for their decision and combines each decision maker’s judgments with those of others. AHP facilitates a greater understanding, within a sense of fair process and participation with greater transparency and traceability of the preferences of groups faced with complex decisions.
Case Study: A Comparison of Financial Based Capital Allocation Decisions with AHP
No matter how diligently organizations attempt to risk adjust the value of their potential projects or programs, the social dynamics of decision-making can make estimates optimistic. In one study of 1,800 organizations, 80 percent couldn’t reach 5.5 percent annual revenue growth, falling far short of their strategic plan goals.
While financial forecasting methods attempt to compensate for these effects with risk adjustments, the inputs to financial models are often limited to optimistic estimates provided by project champions or sponsors influenced by the optimistic biases. Often, the need or desire for projects may cause a systemic bias that inflates the estimates of champions and sponsors.
These blind spots are seemingly inevitable. Yet, if we are to believe what research tells us, some of this systemic bias can be offset by crowd-sourcing the evaluations of experts across an organization using AHP as a framework for synthesizing their insights.
A surprising insight came to light in work done by a company in the pharmaceutical and life sciences industry. While this organization was regularly relying upon risk-adjusted revenues for their product lines, these risk adjustments were provided to the finance department by sponsor and champion organizations proposing their projects.
Figure 3: Lower risk, greater percentage of return.
The organization was able to classify projects using a risk scorecard that looked at how new elements of both the supply chain and technology competencies of the organization fit the project. They found strong correlations between the newness of the project offerings and how likely they were to be completed. They also found correlations between these new, different projects and how likely they would be to meet risk adjusted revenue forecasts.
It turned out that a secondary risk adjustment would be needed to further adjust expectations based on these “probability of success” assessments. However, getting these into the financial analysis proved challenging.
AHP was used to blend together both the quantitative strength of the risk adjusted revenue forecast with the qualitative probability of success criteria. The organization was able to compare expected revenues using the purely financial approach vs. the AHP approach.
Figure 4: Forecasted revenue portfolio vs. AHP recommended prioritization.
One hundred thousand simulations comparing the largest forecasted revenue portfolio against the AHP recommended prioritization showed that organizations can consistently expect a greater return on sales using a crowd-sourced, collaborative decision-making approach over purely financial methods, which have hidden assumptions and biases.
Using the same set of assumptions for the likelihood of completion and revenue generation for projects, two portfolios were selected. One portfolio was chosen from a list of largest-to-smallest revenue forecast projects, from the top down, until available resources to complete them ran out. The second portfolio was chosen using the cost/benefit approach of AHP, which included weighted criteria that combined financial metrics with qualitative probability of success estimates. The AHP-selected portfolio consistently returned as valuable a portfolio as the revenue forecasted approach – and 80 percent of the time outperformed it. When the AHP method did outperform the purely financial method, it did so by 20 percent.
The Analytic Hierarchy Process meets the challenge of bringing together business intelligence and data mining technologies with the social aspects of human decision-making. AHP creates a true picture of the landscape of a decision. It is a means to help decision-makers articulate important decision data points, and combines them with intangible factors that may be difficult to measure but can be expressed and categorized. By narrowing in on the measures of interest, decision-makers can eliminate the sense of data overload from today’s information tidal wave and acquire those pieces of information that are believed to be most relevant.
Group decision-making is difficult. Creating a framework and process to deal with it can make the process less error-prone. AHP assists decision-makers in creating a real context for their choices that adequately weights the dimensions they trust or believe influence their path to success. It is the only chance organizations have to safely navigate the rising tide of information.
Kevin Connor is vice president of Decision Lens’ Solutions Group. He can be reached at firstname.lastname@example.org.