Share with your friends


Analytics Magazine

Executive Edge: Good quant, bad quant

September/October 2013

How to tell accurate analytics methods from ‘quant quackery.’

Wes Nichols accurate analytics methodsBy Wes Nichols

Recently, a long-time MarketShare client joined a new Fortune 50 company as its CMO. When he inquired how the “marketing mix” models it was using were factoring in digital data, the answer shocked him: “Digital is out of scope.” Dumbfounded, he probed further and found that the model builders were unable to include online/offline effects, so simply left digital out. But that seems tenuous considering the company spends some 30 percent of its marketing budget on digital, and its products involve highly considered purchases with much digital activity.

It’s true, of course, that marketing and math haven’t always meshed. Marketing was considered a creative endeavor somehow divorced from the rigor and transparency of science-based business process. The very definition of analytics – “the scientific process of transforming data into insight for making better decisions” – caused tension in marketing-dom. For many, “marketing science” was an oxymoron.

But no more. Better analytic methods, cloud-based technology, new C-suite thinking and – yes – big data have changed all that. In our new data-driven marketing world, art and science not only coexist in marketing, they actually complement one another. [This Art vs. Science in Marketing Analytics video with former CMOs of P&G, Sony and a leading marketing scientist, makes the point.]

Good Math is the Analytics Cornerstone

The cornerstone of today’s successful marketing analytics technology – or what I call “Analytics 2.0” – is the math. If you don’t have the math right, by definition your attribution will be wrong, and by extension your allocations and attempts to optimize your investments will also be wrong. It’s critical to understand that effective marketing resource allocation depends on accurately attributing revenue to different marketing investments online as well as offline and at point of purchase.

The 1.0 version of marketing analytics includes traditional forms of measurement that we’ve had for decades, such as media mix models, agent-based models, digital attribution and simple correlations using Excel spreadsheets. Analytics 2.0 taps today’s perfect storm of big data, technology, predictive analytics and other marketing science to help companies reallocate billions of advertising dollars while realizing double-digit sales lifts with zero additional spend.

Advanced analytics in marketing can hone in on hundreds of a given company’s business drivers, from pricing, distribution and online reviews, to social media chatter, advertising and hard sales data to uncover critical insights about what’s really driving results, and what to do next in the real world. The allocation step is where you put what you’ve learned from attribution and testing into play. Then you can quickly measure outcomes, validate models by running real-time tests, and make course corrections to optimize allocations and results. (See my March 2013 Harvard Business Review cover story “Advertising Analytics 2.0” (abstract here).

Spotting ‘Quant Quackery’

Surprisingly, many big brands are still using largely discredited simple marketing mix econometric methods, or for online marketing “last click attribution.” Such models aren’t looking at the total ecosystem, nor are they measuring the precise impact of, say, TV on search, or search’s impact on retail sales. Simply plugging offline spend into digital marketing analytics models doesn’t achieve “cross-channel” analytics.

Using flawed models is like crediting a single movie theater for an Academy Award winning performance, or trying to win a football game with just eight players on the field. Unfortunately, while the buzzword “attribution” is everywhere these days, many of the solutions trying to solve for this challenge are sub-par. Yet models that play without all the pieces are little more than what we might call quantitative quackery.

For example, to get a true, holistic view of what’s going on and thus make better business decisions, you need all forms of digital (search, social and mobile) in the analytics. Without it you’re missing a rich vein of information about consumer behavior. And remember, even if you don’t spend much or even any money on digital doesn’t mean online behavior – the consumer’s “digital life” – isn’t influencing the decision-making process.

But that’s just one quant quackery component. Relying heavily on old-school data “samples” is another. Samples may still have a place, but part of big data’s beauty is the ability to use all the data from online and offline marketing and sales channels, plus external factors (such as the weather or unemployment), not just samples that are far more error-prone.

Still more quant quackery occurs when marketing analytics focus on attribution for only a small piece of the overall enterprise. The “better decision-making” that analytics promises must factor in enterprise-wide relationships. It’s an advanced form of an old connect-the-dots exercise, only in this version you have to include dots for activities and outcomes you might not be able to actually see, but which exist nonetheless in the data.

Other potential pitfalls lurk in new or unproven measurement approaches such as certain efforts to measure social stream ROI, agent-based models, machine learning and others. These methods are suspect since you can often make the models say whatever you want them to.

Quantifying Marketing’s Business Impact

Big Data without the right math-based analytics is a Big Problem. The “right” analytics are essential to bringing big data to life for marketing organizations, thus allowing for faster insights and better decision-making. This includes such things as:

  • quantifying the long-term impact of brand advertising (brand equity);
  • a holistic approach that includes all online and offline methods and channels;
  • deploying the latest technology, not simple regression models; and
  • transformational thinking that takes marketing analytics beyond simple “research project” status toward enterprise-wide adoption.

Consider one of our auto sector clients, a global manufacturer that’s a superstar in the world of marketing analytics. Their cross-functional analytics team has the daunting task of making sure the company spends its $1 billion marketing budget as effectively as possible while contributing to business goals, achieving the best ROI and increasing shareholder value.

They do it with advanced analytics that allow the company to run continuous marketing strategy simulations under a wide range of complex variations. These simulations employ cross-media attribution insights that help the company predict with greater accuracy than ever how changing the amount spent in one marketing area will likely impact the performance of advertising elsewhere, and what this all does for the bottom line.

Using advanced analytics, this Fortune 20 marketer has also been able to coordinate local and national marketing and dealer incentive budgets, and simply by shifting allocations generate tens of millions of dollars in new revenue from the same spending level.

Almost any company can deploy Analytics 2.0, focusing on marketing analytic methods that avoid the pitfalls noted above. But one thing is sure: Businesses that don’t will be left behind.

Wes Nichols is co-founder and CEO of MarketShare, which provides advanced marketing analytics technology for Global 1000 brands. For independent analysis of competitors in this space see the latest Forrester Wave Report, available here. In this video, Nichols offers a quick overview of Analytics 2.0.

business analytics news and articles



Using machine learning and optimization to improve refugee integration

Andrew C. Trapp, a professor at the Foisie Business School at Worcester Polytechnic Institute (WPI), received a $320,000 National Science Foundation (NSF) grant to develop a computational tool to help humanitarian aid organizations significantly improve refugees’ chances of successfully resettling and integrating into a new country. Built upon ongoing work with an international team of computer scientists and economists, the tool integrates machine learning and optimization algorithms, along with complex computation of data, to match refugees to communities where they will find appropriate resources, including employment opportunities. Read more →

Gartner releases Healthcare Supply Chain Top 25 rankings

Gartner, Inc. has released its 10th annual Healthcare Supply Chain Top 25 ranking. The rankings recognize organizations across the healthcare value chain that demonstrate leadership in improving human life at sustainable costs. “Healthcare supply chains today face a multitude of challenges: increasing cost pressures and patient expectations, as well as the need to keep up with rapid technology advancement, to name just a few,” says Stephen Meyer, senior director at Gartner. Read more →

Meet CIMON, the first AI-powered astronaut assistant

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, made its debut aboard the International Space Station. The ISS’s newest crew member, developed and built in Germany, was called into action on Nov. 15 with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8. Read more →



INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to