Big data, analytics and elections
By George Shen
The 2012 U.S. presidential election is over, and from a statistical viewpoint, the winner was a small group of people armed with analytics who out-predicted many so-called political experts (who relied mostly on gut instinct and experience). The election demonstrated that analytics fueled by big data and advancement in computing technology has become an integral part of the presidential campaign process. The real winner of the 2012 election is analytics.
While most people thought the election would be very close (as many politicians and pundits wanted us to believe), prior to the election, a number of quants and statisticians begged to differ and predicted it was anything but a “nail biter.” In the last few days before Election Day, their models and simulations predicted that Obama would prevail with close to 99 percent certainty based on aggregated poll data. For example, Nate Silver at FiveThirtyEight, a popular political blog published by The New York Times, predicted not only Obama that would win but by exactly how much. Simon Jackman, professor of political science at Stanford University, accurately predicted that Obama would win 332 electoral votes and that North Carolina and Indiana would be the only two states that Obama won in 2008 that would fall to Romney.
Others, including Drew Linzer (assistant professor of political science at Emory University), Sam Wang (a neuroscientist at Princeton University) and Josh Putnam (visiting assistant professor of political science at Davidson College) also correctly predicted the presidential race and many congressional races with great accuracy . It is worth noting that some of them had an outstanding track record in predicting the 2008 election results as well.
Most of these models were based on poll aggregation. Accurate predictions usually factored in the latest polls just before the election. However, Moody senior economist Cheng Xu took a different approach. His model, made in February 2012, used both state economic and political data and predicted Obama winning 303 electoral votes vs. Romney’s 235. It’s difficult to model nine months ahead of time, especially given the economic uncertainty in terms of the length and depth of the recession in every state. According to Xu, his model also took into account voter sentiments – “the grumpy voter effect” . Had Obama lost Florida, which has 29 electoral votes, Xu would have been spot on. (Obama won Florida by a razor-thin margin).
Of course, many journalists, pundits and politicos who are ill-equipped to interpret data were not short of opinions prior to the election. Some of these “political experts” disdained and ridiculed the analytics-driven predictions while others attacked the data scientists and statisticians. Geoffrey Norman at The Weekly Standard called Xu a “bad economist” and Joe Scarborough on MSNBC’s “Morning Joe” called Silver an “ideologue” and a “joke” (Scarborough later offered a post-election apology to Silver). In the end, data-driven analytics triumphed over hunches and experience. Vindication and respect are due for the quantitative minds.
Important Role for Analytics
Analytics played a bigger and more important role in the election than just predicting the outcome. Analytics was an integral part of the 2012 political campaign. In recent elections, Republican and Democratic campaigns have employed data-driven analytics and social-media data to stay ahead of the competition, but the Democrats clearly had the competitive advantage in the 2012 presidential. In June of last year, Politico reported that Obama had a data advantage and went on to say that the depth and breadth of the campaign’s digital operation, from political and demographic data mining to voter sentiment and behavioral analysis, reached beyond anything politics had ever seen . Obama’s 2012 data-crunching operation was far more sophisticated and more efficient at a large scale than its much-heralded 2008 social media juggernaut. (Note that Facebook was 10 times bigger in 2012 than it was in 2008).
During the six months leading up to the election, the Obama team launched a full-scale and all-front campaign, leveraging Web, mobile, TV, call, social media and analytics to directly micro-target potential voters and donors with tailored messages. Compared to previous presidential campaigns in 2004 and 2008, the 2012 campaign was going digital and analytical across all channels. The Obama campaign management hired a multi-disciplinary team of statisticians, predictive modelers, data-mining experts, mathematicians, software programmers and quantitative analysts. It eventually built an entire analytics department five times as large as that of its 2008 campaign.
In an interview with Time magazine, a group of Obama senior campaign advisers revealed an enormous data effort to support fundraising, micro-targeting TV ads and modeling of swing-state voters. They first went through a data integration process to consolidate many disparate databases and create a single, massive system that merged information collected from pollsters, fundraisers, field workers and consumer databases as well as social-media and mobile contacts with the Democratic voter files in the swing states . The advantage of the integrated system is that analytics could be performed effectively across multiple datasets from multiple channels – the ability to connect the digital dots. Furthermore, the information could be shared across the entire organization seamlessly, without multiple versions of the same data or potential data quality issues.
In addition to supporting campaign operations that simply pull data points, the mega database allows data scientists and number crunchers to build analytical models predicting swing voter segmentation with high “persuadability” based on demographic and socioeconomic data and voting record, incorporating the results from micro-targeting models that analyze hundreds of data points to generate “support scores” – a percentage probability that an individual would back the Democratic candidate . The advisers ran experimental campaigns, and analysts factored the results into the models to refine and improve them. The campaign rarely made assumptions without numbers to back them up, according to Obama’s campaign manager Jim Messina who had promised a totally different, metric-driven kind of campaign in which politics was the goal but political instincts might not be the means.
Big data and analytics played a critical role in fund raising too. Fund-raisers, such as George Clooney and Sarah Jessica Parker, were picked by number crunchers through data-mining discovery to match their appeals to certain donors and maximize the star powers. Fund-raising e-mail and text messages targeting certain demographics were tested first among supporters with different subject lines and contents on a small scale and subsequently achieved better results among potential voters on a larger scale. Fund-raising metrics were carefully gauged and analyzed between executions.
Big data and analytics also helped drive the campaign’s ad-buying decisions, which resulted in purchasing ads during unconventional programming and time slots. Here again the team relied on big data analytics rather than on outside media consultants and experts to decide where and when ads should run. Ultimately, this data-driven approach proved very successful in getting the messages out to the targeted viewers and driving the turnout in swing states.
Impact on the Election
Perhaps the 2012 election will be remembered as the first election where big data and analytics played a crucial role and had a tremendous impact on the outcome of the presidential election. Time will tell if it may have revolutionized the institution of politics, similarly to how Billy Beane of “Moneyball” fame and his data-driven approach changed the game of baseball and made a profound impact on the institution of professional sports. Nonetheless, the 2012 election will be a classic case of big data analytics and its applications for many years to come.
What analytics lessons can businesses draw from the 2012 election? The answer is plenty. First and foremost, businesses need to rely more on a data-driven approach and measured performance and less on gut instinct when data and analytics are available. It may require a cultural change and paradigm shift in some organizations. Second, understanding consumer behavior, sentiment and purchase pattern, predicting the next sales opportunity and most profitable customer, segmenting and micro-targeting the right population with tailored messages that resonate with customers are the challenges faced by almost every business. Businesses of all types and sizes should start building a solid, big data knowledge base and mastering the new social and digital intelligence across a variety of channels to identify, target and win customers similarly to how 2012 election was won on the digital front.
George Shen (firstname.lastname@example.org) is an information management specialist master with Deloitte Consulting. An information strategist and consultant with 17 years of experience advising, designing, and implementing business intelligence and data management solutions for many Fortune 500 clients in financial services, telecommunications and life sciences industry, Shen’s expertise spans across information strategy and architecture, business analytics, performance management and a variety of emerging technologies.
- Plumer, Brad, “Pundit Accountability: The Official 2012 Election Prediction Thread,” WONKBLOG, The Washington Post, Nov. 5, 2012.
- Cooper, Michael, “9 Swing States, Critical to Presidential Race, Are Mixed Lot,” The New York Times, May 5, 2012.
- Romano, Lois, “Obama’s Data Advantage,” Politico, June 9, 2012.
- Scherer, Michael, “Inside the Secret World of the Data Crunchers Who Helped Obama Win,” Time, Nov. 7, 2012.
- Issenberg, Sasha, “Obama Does It Better” (from “Victory Lab: The New Science of Winning Campaigns), Slate, Oct.29, 2012.
The opinions expressed here are the views of the author and do not necessarily reflect the views and opinions of Deloitte Consulting. Deloitte is not, by means of this article, rendering business, financial, investment or other professional advice or services. This article is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business.
- 55FEATURES ABM and predictive lead scoring Account-based marketing, and the related technology of predictive lead scoring, is dramatically changing the face of sales and marketing. By Megan Lueders Software survey: joys, perils of statistics Trends, developments and what the past year of sports and politics taught us about variability and…
- 55FEATURES ABM and predictive lead scoring Account-based marketing, and the related technology of predictive lead scoring, is dramatically changing the face of sales and marketing. By Megan Lueders Software Survey: Joys, perils of statistics Trends, developments and what the past year of sports and politics taught us about variability and…
- 53Organizations of all sizes and types are awash in data possibilities, yet most of them cannot capitalize on the potential for a variety of reasons. The good news, however, is that with the right decisions and focus, these possibilities can turn quickly into realized opportunities.
- 45November/December 2013 By Gary Cokins Allow me to take a contrarian view to the rapid interest in applying analytics. Have we seen this kind of story before? Is there a rush to judgment that analytics is the elusive cure-all magic potion or panacea that management has been seeking to achieve…
- 43July/August 2015 Creating value in an era of pervasive analytics and cheap computing. By Paddy Padmanabhan We are entering an era of democratization and crowdsourcing of analytics, driven by the acceleration of technological innovation and the open source movement, coupled with a global pool of highly qualified data scientists and…