Share with your friends










Submit

Analytics Magazine

Messy analytics? It’s OK. We’re housewives!

September/October 2013

Information systems and technology are messy, which makes applying analytics an arduous task.

Information systems and technology are messy, which makes applying analytics an arduous task

Gary Cokins

By Gary Cokins

Who is working with perfect systems? That is like asking if your house or apartment is always clean and tidy to be presentable to guests. In your organization, who cleans up messes? Who fixes things?

I am writing on this topic of messes based on a cafeteria-style breakfast I had in a hotel in Estonia where I was presenting a seminar. It was the peak morning rush, and all the tables were filled with diners. A couple that was seated next to me had just departed. Two women who I’d estimate were in their late 40s eyeballed the two vacant chairs. One made a silent inquiry, “Are these seats available?”

I promptly answered, “Yes, but it’s a mess” and pointed to the soiled dishes, coffee cups and silverware. One of the ladies immediately replied, “It’s OK. We’re housewives.” They then picked up the soiled dishes and carried them to a nearby cart.

After she replied, I had to laugh out loud. Her reply communicated so many things to me. It meant, “We are accustomed to cleaning up messes left behind by others.” It also meant, “Someone has to do it,” especially when it is urgent. It also meant, “If you have been cleaning up most of your life (and I suggest the reader replace “life” with “career”), then you are resigned to the reality that there will always be a mess or problem that will need to be addressed, cleaned or fixed.”

What Type of Mess are you Cleaning?

In information systems and technology (IS/IT) the messes are on three levels: technical, analytical and managerial. I will describe them soon, but first let’s note that there are two types of IS/IT cleaning and maintenance staff.

1. Hardware and infrastructure professionals “cleaners.” Candidly, I am not knowledgeable enough to understand what they do and what their jobs entail. I suspect it is complicated. To me it is like they are down in the boiler room of an ocean sailing ship. To oversimplify their job, it is to keep the lights on for the users.

2. Analyst “cleaners.” Their job is to convert data into information with the purpose to provide insight and foresight for better decisions and actions.
Both types are important. They serve different purposes.

When you read my classification of “mess” levels that follow, which of the three levels do you believe is the most damaging to an organization’s performance and success? Here they are:

• The technical mess. Technical messes involve disparate data sources, a patchwork of purchased hardware and software with predictable incompatibility problems and computing/storage capacity issues. Down in the IS/IT “boiler room” there are lots of hammers and wrenches to fix things.

The analytical mess. In contrast to the technical mess, which is typically complicated to deal with, the analytical mess is complex. (To learn the difference between complicated and complex in this context, see [1].) In summary, complicated systems have many moving parts, like a wristwatch with gears; but they operate in patterned ways. In contrast, complex systems have patterned ways, but the interactions – think variables – are continually changing. In the former, one can usually predict outcomes. The math may be easy with linear relationships. In the latter, such as air traffic control, weather and aircraft maintenance delays cause changes in the constant interactions with numerous variables.

Analyst messes are partly caused by technical messes. An example is defective and/or incomplete source data – the dirty data mess. But even if the analyst has the luxury of perfect input data quality (like a germ-free hospital surgery room or semiconductor chip clean room environment), they will still encounter a problem framing the mess. A key aspect of an analyst’s job is to correctly frame the problem they are trying to solve. Solutions to problems do not always require nails, so an analyst needs more in their tool belt than a hammer.

The good news for analysts is that high-performance analytics and visualization software is now leveraging massively high-speed computing and storage technology. The combination is like a powerful cleanser. With this advance in technology, instead of the analyst framing a problem with a carefully thoughtful hypothesis to test variables that used to require hours of computer time, today it can be run in seconds. This means that analysts can more quickly test and learn. Better yet, they can quickly test, fail and learn. Failing to validate a hypothesis no longer has the adverse consequences it once had in terms of causing delays. Analysts can now fail so quickly that no one sees the brief mess they made.

The managerial mess. Senior management can create a mess that is more difficult to clean compared to a technical or analytical mess. This is because the mess they create comes from their minds and attitudes.

The managerial mess has two broad categories: power and incompetence.

The mess caused by power typically involves such a high reliance on intuition, gut feel and past experience that managers believe they can get by without fact-based information and deep analysis. Confirmation bias further muddies the floor when executives nudge the analysts to twist the findings to satisfy the executive’s pre-conceived notion of what is the correct answer.

Messes caused by incompetence typically involve the inability to see or understand what the analyst’s findings imply for decisions and actions. For example, when a marketing analyst for a telecommunications company examines more than a thousand variables to determine what types of optimal deals might best be offered to maximize profits from differentiated types of customers, the findings may be counter-intuitive – yet valid. That is because analytics deals with complexity, not just complications.

Both managerial mess categories can be addressed, but it requires fortitude by employees to manage their managers; that is, to convince managers of the follies of their management style.

Cleaning Solutions for Messes

Technical messes can be cleaned with good, advanced capital investment planning for purchasing (or now with SaaS, renting) the correct hardware and software. Also educate, educate, educate; and train, train, train.

Analytical messes can be cleaned with good data governance practices (e.g., extraction, transform and load) and with high-performing analytics software with visualization to accelerate the analyst’s experimentation and investigation.

Managerial messes are more problematic. The stains in the carpet are deep.

After the long, cold winter, we all attempt to do spring cleaning. I submit that following the global economic collapse of 2008 many organizations are on the road to recovery. They experienced their winter of the economic cycle. It is time to clean up messes.

We need to behave like the two breakfast diners I met who said, “It’s OK. We’re housewives.” Get on with the task. Get your IT house in order. The popularity of analytics, operations research techniques and big data is quickly growing. Having messes will slow down your progress to leverage and deploy analytics.

Gary Cokins (gcokins@garycokins.com) is president of Analytics-Based Performance Management LLC in Cary, N.C. He is a member of INFORMS.

business analytics news and articles

Analytics Blog

Electoral College put to the math test


With the campaign two months behind us and the inauguration of Donald Trump two days away, isn’t it time to put the 2016 U.S. presidential election to bed and focus on issues that have yet to be decided? Of course not.


Headlines

Gaining distribution in small retail formats brings big payoffs

Small retail formats with limited assortments such as Save-A-Lot and Aldi and neighborhood stores like Target Express have been growing in popularity in the United States and around the world. For brands, the limited assortments mean greater competition for shelf-space, raising the question of whether it is worth expending marketing effort and slotting allowances to get on to their shelves. According to a forthcoming study in a leading INFORMS scholarly marketing journal, Marketing Science, the answer is “yes.” Read more →

Cognitive computing a disruptive force, but are CMOs ready?

While marketing and sales professionals increasingly find themselves drowning in data, a new IBM study finds that 64 percent of surveyed CMOs and sales leaders believe their industries will be ready to adopt cognitive technologies in the next three years. However, despite this stated readiness, the study finds that only 24 percent of those surveyed believe they have strategy in place to implement these technologies today. Read more →

How weather can impact consumer purchase response to mobile ads

Among the many factors that impact digital marketing and online advertising strategy, a new study in the INFORMS journal Marketing Science provides insight to a growing trend among firms and big brands: weather-based advertising. According to the study, certain weather conditions are more amenable for consumer responses to mobile marketing efforts, while the tone of the ad content can either help or hurt such response depending on the current local weather. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

Essential Practice Skills for High-Impact Analytics Projects
Sept. 26-27, Executive Conference Center, Arlington, Va.

Foundations of Modern Predictive Analytics
Oct. 2-3, VT Executive Briefing Center, Arlington, Va.

2017 INFORMS Annual Meeting
October 22-25, 2017, Houston

2017 Winter Simulation Conference (WSC 2017)
Dec. 3-6, 2017, Las Vegas

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.