Share with your friends


Analytics Magazine

Marketing: Leveraging Big Data around customers

September/October 2012

Opportunity to deliver real-time customer insights by harnessing the power of structured and unstructured data.

Rohit Tandon, Arnab Chakraborty and Ganga GanapathiBy Rohit Tandon, Arnab Chakraborty and Ganga Ganapathi (left to right)

The rise of the Internet has created terabytes of data or “Big Data” that is available to consumers and enterprises alike. This digitization is an emerging phenomenon and is pushing the boundaries of every industry, shifting consumer buying patterns. Marketers are leaving no stone unturned – the products that could once be picked up off the rack are now sold online.

To their advantage, the consumer is exercising freedom of choice regarding numerous such online services, making consumers far more empowered. Consumers research a prospective purchase offline as well as online, migrate between channels in the course of a single buying cycle, voice unsolicited opinions safe in the knowledge that these may be heard by organizations and influence buying behavior of others in their network, reiterating that “word-of-mouth’ has gone digital. The sophistication of buying behavior is at unprecedented levels, and it will only increase going forward.

Some concepts to keep in mind:

Data is divine: About 80 percent of data is unstructured. Not only does this data open new avenues of analysis, thus allowing companies to come closer to its customers in real time, it can also lead to premature death of companies if they are not receptive to the data to understand customer needs. By successfully understanding data, companies will be able to identify trends, create new products and services, and in the long run, make decisions and strategies that are backed by pure data and not sheer intuition. Equipped with large amounts of customer data – insurance companies, retailers, transportation companies and communications providers – have a unique opportunity to make this type of information services play to their advantage.

Sentiment-al: While companies don’t dispute the importance of data, they face maximum challenges in extracting and comprehending the data. Probing the unconscious mind of the consumer has tremendous value beyond advertising. It reveals that what consumers actually believe or think may contradict what they say when asked. Global e-commerce sales may soon reach $1 trillion. This trend is directly related to the number of people making transactions online, creating an opportunity for companies to use the online information to assess customers’ buying behavior and clout to make or mar a brand. Marketers can add immense value by making observations, listening and acting rather than focusing on invasive advertising. This will be the game changer.

Imagine, at a simplistic level the launch of a campaign for a new product or service. Paid advertising performs its job of awakening interest and creating desire for a hitherto unarticulated or unrecognized need. This might lead to a couple of searches, posts on social media platforms, and, if lucky, a visit to the company or brand’s Web site. The propensity to purchase is probably increasing or decreasing with every new byte of information consumed, and all this happens without even one traditionally defined “formal” interaction with the company. The evangelists or detractors impacting this decision are individuals who want to share their experience as a shopper and help a fellow shopper. Imagine the potential of growth or decline a single customer’s experience and comment online can create for a company.

Figure 1: Big Data is within grasp of organizations that can tame the data.
Figure 1: Big Data is within grasp of organizations that can tame the data.

Big Data is within our grasp: Therefore, all marketers require is a vision to tame data that millions of such interactions and transactions create. One also has to keep in mind the existing structured data, which together with unstructured data can create a new meaning to marketing. Companies that realize this early will be true pioneers of the Big Data era. Data points from multiple sources – viewership data, search, clickstream, social media metrics & sentiment, e-commerce, offline retail – if stitched together and analyzed appropriately – will reveal insights that can separate the marketing seers from the mere also-rans.

Seeing this deluge of data as a fountainhead of insights will help understand the present and foretell the future in a way that can create competitive advantage like never before. For example, it can improve access to articulated and unarticulated customer needs, make revelations of need gaps, identify influencers, tell the story behind trends, confirm early warning signals; in short, the power that marketers in turn can wield to create customer advantage has never been this obvious before.

Figure 2: Listening to the customer.
Figure 2: Listening to the customer.

Customer insights for ROI (It’s all about the money): With most transaction data, and even some forms of unstructured data being largely understood today, insights that will emerge from interactions that haven’t been documented in as much detail yet are next in line, such as shop floor data to discern insights from sales-rep interactions, emotion analysis during all human interactions, and other voice and video data for possibly unspoken revelations. These, when combined with better understood forms of data, will begin to reveal insights that can be applied across the marketing value chain.

Going forward, decisions on product roadmaps, marketing messages, sales levers and service interactions will become so steeped in insights that hitting the bull’s eye will cease to be matter of chance. Insights will have a direct impact on ROI. For example:

  • by uncovering new markets – demographic or psychographic customer segments for new and existing propositions;
  • by helping make sales cycles shorter and more effective, thus optimizing investments; and
  • by making service interactions effective, efficient and delightful for the customer, reducing dissonance and increasing loyalty. Driving the CI function to its rightful place under the sun.

Instances abound for the way Big Data can be applied across the marketing value chain.

The utility of social media analytics spanning the functions of insight generation, engagement, health metrics when combined with campaign information, macro-economic data, competitive data, sales insights, customer service interactions and much more on an on-going basis, can yield real-time, actionable insights that were previously difficult to sustain on an on going basis.

For example:

Product teams can get rich insights to plan with: Freeing themselves from the constraints of OEM blueprints and short-term demand plans, product teams can have access to the exciting insights they’ve always dreamed of basing their innovations on. While primary research and observations may continue to be the mainstay for some time to come, the scalability of listening in to social media platforms has brought the world closer than ever before. Emerging markets will never open up using the previous years’ trends from developed markets, and exciting idea for new categories itself may emerge from robust listening programs.

Marketing is as much about the relationship as it is about the transaction: Customers want to be recognized for the individuals they are and not treated as one among a hundred others. Having a Web page that knows that customers have returned for a second look, understands that desire is driven by an intimate understanding of the customer psyche and offers a replacement just when customers are contemplating a switch – these are the hotspots that turn customers on and buys their loyalty. In addition, the brand evangelists will become the trusted third-party influencer to others contemplating a relationship with the offering.

The most compelling argument, of course, is the opportunity to optimize marketing spends based on reading social signals correctly. Returns from “bought” and “owned” media begin to plateau or even show negative returns beyond a point, but juxtaposing this impact with the positive mileage that “earned” media can deliver recognizes the opportunity to save precious marketing dollars without compromising impact.

Figure 3: The insights data can provide are unlimited.
Figure 3: The insights data can provide are unlimited.

Spray & pray replaced by sure shot targeting: In this era of limited everything – investments, sales teams, time – the only things that need not be limited are the insights that data can provide. Combining hitherto disparate data sets – and analyzing them in ways that were previously not possible – to drive insights around sales personalization is possibly the lowest hanging fruit in an enterprise. The work is with largely recognized data sources, the returns are almost immediate, and the ROI delivered eases the next battle that will be fought to justify Big Data investments.

Real time insights for better customer service: Technology has enabled customer service at the touch of a button and is no longer dependant on the last mile interaction at a service center. Service interactions fit into the Big Data world in two wonderful ways: using insights generated from correctly assembled and analyzed Big Data, interactions can be personalized with a deep understanding of the customer and with a high chance of guaranteeing satisfaction. For instance, concerns that begin to trend in the social world soon enter call center interactions. Prepping agents or using the early warning signals to take decisive action are signs of an enterprise reacting in real time.

On the flip side, all interactions, whether on chat or voice, are rich fodder for enriching the database. For example, call centers can analyze text, sentiment and emotion combined with trends, demographics and psychographics to paint multi-dimensional pictures of the target audience, the competition and market behavior.

Real-Time Deployment of Customer Insights

The definition of Big Data is bound to vary between organizations and the quantum and variety of data that they have been used to in the past. For any enterprise CTO, the first implication of Big Data is the need for a strategy for dealing with large quantities of data. The term “Big Data” encompasses complex data sets so large and unwieldy that existing database management tools cannot aid in the collection, storage, analysis, sharing and visualization of the raw data and insights generated. The large data sets get generated because Big Data is a supposedly large, related data set, as opposed to the smaller component data sets that exist in isolation. This allows for co-relations to be found between data sets that could not previously talk to each other.

The technology stack has to be such that it allows for the collection and storage of data along the three dimensions of challenges and opportunities that it presents: the increasing and ever-growing volume of data, the velocity of data (in and out) and the variety across the types and sources of data.

Figure 4: The technology stack.
Figure 4: The technology stack.

Customer-related data for an enterprise may come from many sources: Web logs, social data, Internet search indexing, call center call and chat records, video archives, e-commerce, sales transactions, macro-economic data, related category market, competitive data, etc. By present standards, the volume of business data worldwide across all companies doubles every 1.2 years, so the CTO needs to have a vision that takes into account the growth of data that the enterprise will need to be prepared for. The key features that typically form the basis of a Big Data analytics platform may include;

  • real-time query and loading enables immediate access for rich analytics;
  • in-database analytics eliminates the need to extract;
  • database designer tools with the ability to work with structured and unstructured data, and enabling continual improvements while the system remains online;
  • data compression and columnar storage and execution for faster and efficient querying;
  • ability to scale out on demand without adding expensive infrastructure;
  • high availability;
  • optimized workload management so the users focus on execution; and
  • seamless integration with the growing ecosystem of analytics solutions such as Hadoop, MapReduce, ETL and Native BI.

Deployment on the cloud is also gaining acceptance, and this may help smaller companies with limitations on scalable infrastructure.

The techniques and technologies that need to be enabled to actually process Big Data are a combination of traditional and new-age methods: A/B testing, association rule learning, classification, cluster analysis, natural language processing, neural networks, pattern recognition, predictive modeling, regression, sentiment analysis, time series, etc.

The last mile, and often the most critical one, is possibly visualization. This is a challenge for Big Data analytics, and we’re seeing continued evolution in this space. Better visualization tools are critical because many cases of Big Data analytics deployment demand instant time to reaction, making the case for increased efficiency. Also, those who finally execute based on the insights may not necessarily be data scientists, and this makes it all the more important for findings and recommendations to be presented as clearly as possible. Needless to say, Web and mobile platforms are equally important to the final consumption of analytics.

The Future of Big Data

The future of customer insights has never been brighter, and it will only scale greater heights from where it is right now. In this era of limited everything – investments, sales teams and time – the only things that need not be limited are the insights that data can provide. Combining hitherto disparate data sets and analyzing them in ways that were previously not possible will drive insights around sales personalization, and it represents possibly the lowest hanging fruit in an enterprise. The returns are almost immediate and the ROI delivered eases the next battle that will be fought to justify Big Data investments.

While primary research and observations may continue to be the mainstay for some time to come, the scalability of listening in to social media platforms has brought the world closer than ever before. The utility of social media analytics spanning the functions of insight generation, engagement and health metrics, when combined with campaign information, macro-economic data, competitive data, sales insights, customer service interactions and much more on an on-going basis, can yield real-time, actionable insights that were previously difficult to sustain on an ongoing basis.

The digitization of life that we see around us provides millions of data points that can be stitched together to provide 360-degree views of the customer. Thinking ahead and being prepared to satisfy unarticulated needs through real-time insights and customized recommendations will help create customer delight like never before, as well as competitive advantage that can be quickly monetized. The overall story is a win-win situation for the both sides of the table, leaving us with the obvious answer to one last question: Has there ever been a better time to be a marketer?

Rohit Tandon, senior INFORMS member, is the vice president and worldwide head of Global Analytics at Hewlett-Packard Company, where he helps drive the analytics ecosystem to support HP’s vision and priorities through delivery of cutting edge analytical capabilities across sales, marketing, supply chain, finance and HR domains. Arnab Chakraborty (, senior INFORMS member,  is director of Global Analytics at H-P, where he is responsible for partnering with senior executive leadership teams within HP and drives the deployment of analytics solutions across the Americas, EMEA and APJ regions. Ganga Ganapathi ( is manager of Global Analytics at HP, focusing on marketing analytics. She partners with business teams to drive deployment of analytics solutions across the Americas, EMEA and APJ regions.

business analytics news and articles

Related Posts

  • 53
    Features Customer data: volume vs. value By Chirag Shivalker Companies invest millions of dollars in building and managing large data sets, but they don’t need more customer data; what they really need is the “right data.” Big data: products, services, operations By Dave Oswill What data scientists and engineers need…
    Tags: data, big, customer
  • 51
    International Data Corporation (IDC) recently released a worldwide Big Data technology and services forecast showing the market is expected to grow from $3.2 billion in 2010 to $16.9 billion in 2015. This represents a compound annual growth rate (CAGR) of 40 percent or about seven times that of the overall…
    Tags: data, big
  • 49
    Many organizations have noticed that the data they own and how they use it can make them different than others to innovate, to compete better and to stay in business. That’s why organizations try to collect and process as much data as possible, transform it into meaningful information with data-driven…
    Tags: data, big
  • 49
    May/June 2014 By (l-r) Pramod Singh, Ritin Mathur, Arindam Mondal and Shinjini Bhattacharya “There is a big data revolution. However, what is revolutionary is not the quantity of data alone. The big data revolution is that now we can do something with the data.” – Professor Gary King, Harvard University…
    Tags: data, insights, big
  • 47
    Features The direction of smart automation By Joseph Byrum Key technologies: What humanity optimized by artificial intelligence might look like: combining AI, big data, advanced analytics and autonomous systems. Turbocharging AI and ML projects By Shashank Dubey Three critical mindsets – business case, entrepreneur’s and adoption – close the gap…
    Tags: data, big, customer


Using machine learning and optimization to improve refugee integration

Andrew C. Trapp, a professor at the Foisie Business School at Worcester Polytechnic Institute (WPI), received a $320,000 National Science Foundation (NSF) grant to develop a computational tool to help humanitarian aid organizations significantly improve refugees’ chances of successfully resettling and integrating into a new country. Built upon ongoing work with an international team of computer scientists and economists, the tool integrates machine learning and optimization algorithms, along with complex computation of data, to match refugees to communities where they will find appropriate resources, including employment opportunities. Read more →

Gartner releases Healthcare Supply Chain Top 25 rankings

Gartner, Inc. has released its 10th annual Healthcare Supply Chain Top 25 ranking. The rankings recognize organizations across the healthcare value chain that demonstrate leadership in improving human life at sustainable costs. “Healthcare supply chains today face a multitude of challenges: increasing cost pressures and patient expectations, as well as the need to keep up with rapid technology advancement, to name just a few,” says Stephen Meyer, senior director at Gartner. Read more →

Meet CIMON, the first AI-powered astronaut assistant

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, made its debut aboard the International Space Station. The ISS’s newest crew member, developed and built in Germany, was called into action on Nov. 15 with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8. Read more →



INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to