Share with your friends


Analytics Magazine

How prescriptive analytics can reshape fracking in oil and gas fields

March/April 2014

Atanu BasuBy Atanu Basu

The United States is re-emerging as an energy superpower. According to the International Energy Agency, by 2016 the U.S. will surpass Saudi Arabia and become the world’s largest oil producer.

The domestic energy industry’s recent rise is the result of lower demand through energy efficiency and the rise in production of unconventional oil and gas discovered in underground shale formations. Horizontal drilling and hydraulic fracturing have made it possible to economically produce oil and gas from tight rocks. In October 2013, U.S. oil production reached its highest monthly total in the last 25 years. In Texas, with crude oil production of more than 2.7 million barrels per day, two shale oil fields alone – Eagle Ford and Permian Basin – are on target to produce nearly 2 million barrels of oil equivalent a day in 2013.

However, while abundant, shale oil and gas can be difficult to locate and extract. Horizontal drilling and hydraulic fracturing processes are expensive and, some say, potentially harmful to the environment. Another relatively unknown fact – especially to industry outsiders – is that fracking is quite inefficient today: 80 percent of the production comes from 20 percent of the fracking stages. Today, horizontal drilling and hydraulic fracturing recover about 20 percent, probably less, of the oil in the shale rocks. According to PacWest, drillers will spend $31 billion in 2013 on suboptimal frack stages across 26,100 wells in the United States. In response, some of the largest oil and gas companies are using big data analytics technologies to improve their exploration and production.

Big data analytics includes three categories: descriptive analytics, which tells you what happened and why; predictive analytics, which tells you what will happen; and prescriptive analytics, which tells you what will happen, when, why and how to improve this predicted future.

Marketers, operations experts, financial officers and other business leaders have already used prescriptive analytics to improve customer experience, reduce churn, increase up-selling and cross-selling revenue, streamline logistics and enhance other important applications. For the oil and gas industry, prescriptive analytics can help locate fields with the richest concentrations of oil and gas, make pipelines safer, and improve the fracking process for greater output and fewer threats to the environment.

Horizontal drilling and hydraulic fracturing have made it possible to economically produce oil and gas from tight rocks.

About 80 percent of the world’s data today is unstructured – videos, images, sounds and texts. Until recently, most big data analytics technologies looked only at numbers. The oil and gas industry looked at images and numbers, but in separate silos. However, the ability to analyze hybrid data – a combination unstructured and structured data – provides a much clearer and more complete picture of the current and future problems and opportunities, along with the best actions to achieve the desired outcomes. For example, to improve hydraulic fracturing performance, the following datasets must be analyzed together:
• images from well logs, mud logs, seismic reports,
• videos from down-hole cameras of fluid flow,
• sounds from fracking recorded by fiber optic sensors,
• texts from drillers’ and frack pumpers’ notes, and
• numbers from production and artificial lift data.

Taking hybrid data into account is critical because of the multi-billion dollar investment and drilling decisions that are being made by the energy companies regarding where to drill, where to frack and how to frack. It calls for combining disparate computational and scientific disciplines to be able to interpret different types of data together. For example, to algorithmically interpret images (such as well logs), machine learning needs to be combined with pattern recognition, computer vision and image processing. Mixing these different disciplines provides more holistic recommendations regarding where and how to drill and frack, while reducing the chances of problems that could emerge along the way.

For example, by developing detailed analytical signatures – using data from production, subsurface, completion and other sources – one can better predict performing and non-performing wells in a field. This process is supported by the prescriptive analytics technology’s ability to automatically digitize and interpret well logs to create depositional maps of the subsurface. With a better idea of where to drill, companies save invaluable resources by skipping wells that shouldn’t be drilled in the first place. At the same time, they minimize damage to that particular landscape.

Prescriptive analytics can be used in other areas of oil and gas production. In both traditional and unconventional wells, by using data from pumps, production, completion and subsurface characteristics, one can predict failures of electric submersible pumps and prescribe actions to mitigate production loss. Apache Corp., for example, is using analytics to predict failures in pumps that pull oil out from subsurface and preempt the associated production loss from these pump failures.

Another potential application of prescriptive analytics is that it can possibly predict corrosion development or cracks in pipelines and prescribe preventive and preemptive actions by analyzing video data from cameras along with other data from robotic devices called “smart pigs” inside these pipelines.

Smarter decisions equal fewer resources, lower environmental impact and greater yields. Successful companies will be the ones that know how to prioritize resources to extract, produce and transport oil and gas in the most efficient and safest manner. Look for big data and prescriptive analytics to play a much bigger role in this space over the coming years.

Atanu Basu is CEO of AYATA, a software company headquartered in Austin, Texas. AYATA’s prescriptive analytics software focuses on improving oil and gas exploration and production. Basu is a member of INFORMS. A version of this article appeared in DataInformed.

business analytics news and articles

Related Posts

  • 62
    July/August 2013 By Warren Wilson Throughout the century-and-a-half since the dawn of the commercial petroleum industry, oilmen have always hoped for the gusher – the big find that would spew enough oil to make them rich. There have always been far more “dry holes” than gushers, however, and the proportion…
    Tags: oil, data, analytics, gas, production, prescriptive
  • 60
    November/December 2014 The marriage of two “natural resources” – hydrocarbons and data – will transform unconventional oil development. By Atanu Basu (right), Daniel Mohan and Marc Marshall Known-knowns, known-unknowns and unknown-unknowns. Donald Rumsfeld’s notable turn of phrase is an apt characterization of where we are with unconventional oil development today.…
    Tags: data, oil, analytics, prescriptive, energy, gas
  • 53
    November/December 2012 The advent of the “digital oil field” helps produce cost-effective energy while addressing safety and environmental concerns. Finding and producing hydrocarbons is technically challenging and economically risky. By Adam Farris Everyone needs it, few know how we get it, and many feel compelled to slow down efforts to…
    Tags: data, oil, gas, energy, big
  • 42
    As the Big Data Analytics space continues to evolve, one of the breakthrough technologies that businesses will be talking about in the coming years is prescriptive analytics. The promise of prescriptive analytics is certainly alluring: it enables decision-makers to not only look into the future of their mission critical processes…
    Tags: analytics, prescriptive, data
  • 41
    July/August 2014 The story of how IBM not only survived but thrived by realizing business value from big data. By (l-r) Brenda Dietrich, Emily Plachy and Maureen Norton This is the story of how an iconic company founded more than a century ago, and once deemed a “dinosaur” that would…
    Tags: analytics, data, big


Does negative political advertising actually work?

While many potential voters dread campaign season because of pervasive negative political advertising, a new study has found that negative political advertising actually works, but perhaps not in the way that many may assume. The study “A Border Strategy Analysis of Ad Source and Message Tone in Senatorial Campaigns,” which will be published in the June edition of INFORMS’ journal Marketing Science, is co-authored by Yanwen Wang of the University of British Columbia in Vancouver, Michael Lewis of Emory University in Atlanta and David A. Schweidel of Georgetown University in Washington, D.C. Read more →

Meet Summit, world’s most powerful, smartest scientific supercomputer

The U.S. Department of Energy’s Oak Ridge National Laboratory on June 8 unveiled Summit as the world’s most powerful and smartest scientific supercomputer. With a peak performance of 200,000 trillion calculations per second – or 200 petaflops – Summit will be eight times more powerful than ORNL’s previous top-ranked system, Titan. For certain scientific applications, Summit will also be capable of more than three billion billion mixed precision calculations per second, or 3.3 exaops. Read more →

Employee engagement a top concern affecting customer experience

Employee engagement has surfaced as a major concern in delivering improvements in customer experience (CX), with 86 percent of CX executives in a Gartner, Inc. survey ranking it as having an equal or greater impact than other factors such as project management and data skills. “CX is a people issue,” says Olive Huang, research vice president at Gartner. “In some instances, the best technology investments have been derailed by employee factors, such as a lack of training or incentives, low morale or commitment, and poor communication of goals." Read more →



INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix


Advancing the Analytics-Driven Organization
July 16-19, noon-5 p.m.

Making Data Science Pay
July 30-31, 12:30 p.m.-5 p.m.

Predictive Analytics: Failure to Launch Webinar
Aug. 18, 11 a.m.

Applied AI & Machine Learning | Comprehensive
Sept. 10-13, 17-20 and 24-25


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to