Share with your friends


Analytics Magazine

Turn big data into information with high-performance analytics

September/October 2013

big data information high-performance analytics technology

Paul Kent (LEFT TO RIGHT), Radhika Kulkarni and Udo SglavoBy Paul Kent (LEFT TO RIGHT), Radhika Kulkarni and Udo Sglavo

We’re in the era of big data, but what do we mean by that? In our view, big data is a relative, not absolute, term. It means that the organization’s need to handle, store and analyze data (its volume, variety, velocity, variability and complexity) exceeds its current capacity and has moved beyond the IT comfort zone [1]. Big data is the classic dual-edged sword – both potential asset and possible curse. Most agree that there is significant, meaningful, proprietary value in that data. But few organizations relish the costs and challenges of simply collecting, storing and transferring that massive amount of data. And even fewer know how to tap into that value, to turn the data into information.

Is the enterprise IT department merely an episode of TV’s “Hoarders” waiting to happen – or will we actually find ways to locate the information of strategic value that is getting buried deeper and deeper in our mountains of data? Quite simply: What are we going to do with all of this data?

At its essence, high-performance analytics (HPA) offers a simple, but powerful, promise: Regardless of how you store data or how much of it there is, complex analytical procedures can still access that data, build powerful analytical models using that data, and provide answers quickly and accurately by using the full potential of the resources in your computing environment.

With high-performance analytics, we are no longer primarily concerned with where the data resides. Today, our ability to compute has far outstripped our ability to move massive amounts of data from disk to disk. Instead, we use a divide-and-conquer approach to cleverly send the processing out to where the data lives.

Ultimately, HPA is about the value of speed and its effect on business behavior. If the analytic infrastructure requires a day to deliver a single computational result, you’re likely to simply accept the answer it provides. But if you can use HPA to get an answer in one minute, your behavior changes. You ask more questions. You explore more alternatives. You run more scenarios. And you pursue better outcomes.

But how do we bring the power of high-performance analytics to data volumes of this scale? We believe there are three basic pillars – three innovative approaches – to bring HPA to dig data:

  • Grid computing: distribute the workload among several computing engines. Grid computing enables analysts to automatically use a centrally managed grid infrastructure that provides workload balancing, high availability and parallel processing for business analytics jobs and processes. With grid computing, it is easier and more cost-effective to accommodate compute-intensive applications and growing numbers of users appropriately across available hardware resources and ensure continuous high availability for business analytics applications. You can create a managed, shared environment to process large volumes of programs in an efficient manner.
  • In-database analytics: move the analytics process closer to the data. With in-database processing, analytic functions are executed within database engines using native database code. Traditional programming may include copying data to a secondary location, and the data is processed using the programming language outside the database. Benefits of in-database processing include reduced data movement, faster run-times, and the ability to leverage existing data warehousing investments.
  • In-memory analytics: distribute the workload and data alongside the database. In this approach, big data and intricate analytical computations are processed in-memory and distributed across a dedicated set of nodes to produce highly accurate insights to solve complex problems in near-real time. This is about applying high-end analytical techniques to solve these problems within the in-memory environment. For optimal performance, data is pulled and placed within the memory of a dedicated database appliance for analytic processing.

Keys to HPA Success

What does it take to succeed with high-performance analytics? HPA isn’t simply an incremental discipline. It involves innovative shifts in how we approach analytic problems. We view them differently and continue to find new ways to solve them. It’s more than simply taking a serial algorithm and breaking it into chunks. Success requires deeper, broader algorithms in multiple disciplines and the ability to rethink our business processes.

In our experience, HPA solutions to complex business problems require innovation along two different dimensions. First, algorithms and modeling techniques must be invented and built to exploit the power of massively parallel computational environments in three major areas:

  • Descriptive analytics. You can report and generate descriptive statistics of historical performance that help you see what has transpired far more clearly than ever before.
  • Predictive analytics. You can use data relationships to model, predict and forecast business results in impressive ways and predict future events and outcomes.
  • Prescriptive analytics. You can identify the relationships among variables to develop optimized recommendations that take advantage of your predictions and forecasts and foresee the likely implications of each decision option [2].

Second, HPA tools and products must be built, incorporating these high-performance analytics techniques, to enable the following:

  • visualization and exploration of massive volumes of data;
  • creation of analytical models that use multi-disciplinary approaches such as statistics, data mining, forecasting, text analytics and optimization; and
  • application of domain-specific solutions to complex problems that incorporate both specific analytical techniques as well as the business processes to support decision-making.

What makes HPA so compelling to businesses across the spectrum – and makes them willing to undertake this fundamental rethinking of analytics – is the ability to address and resolve transformational business problems that have the potential to fundamentally change the nature of the business itself. By processing billions of observations and thousands of variables in near-real time, HPA is unleashing power and capabilities that are without precedent. Your business could witness the same results, for example, by taking the following steps:

  • implementing a data mining tool that creates predictive and descriptive models on enormous data volumes;
  • using those variables to predict which customers might abandon an online application and offer them incentives to continue their session; and
  • comparing these incentives against one another and the budget, in real time, to identify the best offer for each customer.

That’s the kind of emphatic value that HPA can provide and why it’s continuing to garner the attention of many enterprises today.


Amazingly, the discipline of high-performance analytics continues to move forward at a rapid pace. As storage gets even more affordable and greater amounts of processing power become ever-cheaper, it’s easy for us to envision “analytical streaming” in real time where insights are not discrete events but are part of the minute-by-minute operation of the enterprise, woven into the fabric of every meaningful business process. Moving further down the cost curve will enable us to further democratize analytics and move it beyond the specialized analyst and into the hands of virtually every employee, increasing the breadth and depth of the value. By pushing out the power of this style of HPA, we have the opportunity to achieve exponentially outsized gains driven by new levels of rapid analysis.

Paul Kent is the vice president of Big Data at SAS. Radhika Kulkarni is vice president of Advanced Analytics R&D at SAS and a senior member of INFORMS. Udo Sglavo is principal analytical consultant at SAS. This article was excerpted and adapted from the chapter “Finding Big Value in Big Data: Unlocking the Power of High-Performance Analytics” by the authors in “Big Data and Business Analytics,” edited by Jay Liebowitz, ©2013 Taylor & Francis Group LLC. Reprinted with permission.


  1. For more information, visit:
  2. For more information, visit:

business analytics news and articles

Related Posts

  • 83
    July/August 2014 Key considerations for deep analytics on big data, learning and insights. By (l-r) Haluk Demirkan and Bulent Dal What is big data? Big data, which means many things to many people, is not a new technological fad. In addition to providing innovative solutions and operational insights to enduring…
    Tags: data, big, business, analytics, velocity, volume, variety, variability
  • 74
    November/December 2013 By Gary Cokins Allow me to take a contrarian view to the rapid interest in applying analytics. Have we seen this kind of story before? Is there a rush to judgment that analytics is the elusive cure-all magic potion or panacea that management has been seeking to achieve…
    Tags: analytics, big, business, data, velocity, variety, volume
  • 64
    July/August 2013 Four things to look for when building an analytics team. By Andrew Jennings Companies in every industry from retail to banking are leveraging big data to improve the customer experience and enhance their bottom lines. Big data – high volume, high velocity (real time) and high variety (structured…
    Tags: data, analytics, big, business, variety, volume, velocity
  • 53
    Organizations of all sizes and types are awash in data possibilities, yet most of them cannot capitalize on the potential for a variety of reasons. The good news, however, is that with the right decisions and focus, these possibilities can turn quickly into realized opportunities.
    Tags: data, analytics, business, big
  • 50
    From reporting traffic jams using the Waze app to a colony of ants working together to achieve extraordinary feats, examples of crowdsourcing and collective intelligence are all around us. As the problems that we face grow in complexity, the speed and reach of communications and data flow are vastly improving…
    Tags: analytics, data, volume, business, complexity, variety


Report: One in five cloud-based user accounts may be fake

According to the Q2 2018 DataVisor Fraud Index Report, more than one in five user accounts set up through cloud service providers may be fraudulent. The report, based on information gathered between April and June, analyzes 1.1 billion active user accounts, 1.5 million email domains, 231,000 device types and 562 cloud hosting providers and data centers, among other indicators. Read more →

When managers respond to online critics, more negative reviews ensue

A new study in the INFORMS journal Marketing Science found that when managers respond to online reviews it’s possible that those responses could actually stimulate additional reviewing activity and an increased number of negative reviews. The study, “Channels of Impact: User Reviews When Quality is Dynamic and Managers Respond,” is authored by Judith Chevalier of the Yale School of Management and NBER, Yaniv Dover of the Hebrew University of Jerusalem and Dina Mayzlin of the Marshal School of Business at the University of Southern California. Read more →

IE student designs software to optimize snow removal at Penn State

It is well known among the State College and Penn State communities that it takes a lot for university officials to shut the campus down after a major snowfall. In fact, since 2010, the University Park campus has been shut down just three full days due to snowfall. Much to the chagrin of students – and faculty and staff – the snow day at Penn State may just have become even more elusive, thanks to software developed by recent industrial engineering graduate Achal Goel. Read more →



Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden

INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.


Applied AI & Machine Learning | Comprehensive
Dec. 3, 2018 (live online)

Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to