Share with your friends


Analytics Magazine

How prescriptive analytics can reshape fracking in oil and gas fields

March/April 2014

Atanu BasuBy Atanu Basu

The United States is re-emerging as an energy superpower. According to the International Energy Agency, by 2016 the U.S. will surpass Saudi Arabia and become the world’s largest oil producer.

The domestic energy industry’s recent rise is the result of lower demand through energy efficiency and the rise in production of unconventional oil and gas discovered in underground shale formations. Horizontal drilling and hydraulic fracturing have made it possible to economically produce oil and gas from tight rocks. In October 2013, U.S. oil production reached its highest monthly total in the last 25 years. In Texas, with crude oil production of more than 2.7 million barrels per day, two shale oil fields alone – Eagle Ford and Permian Basin – are on target to produce nearly 2 million barrels of oil equivalent a day in 2013.

However, while abundant, shale oil and gas can be difficult to locate and extract. Horizontal drilling and hydraulic fracturing processes are expensive and, some say, potentially harmful to the environment. Another relatively unknown fact – especially to industry outsiders – is that fracking is quite inefficient today: 80 percent of the production comes from 20 percent of the fracking stages. Today, horizontal drilling and hydraulic fracturing recover about 20 percent, probably less, of the oil in the shale rocks. According to PacWest, drillers will spend $31 billion in 2013 on suboptimal frack stages across 26,100 wells in the United States. In response, some of the largest oil and gas companies are using big data analytics technologies to improve their exploration and production.

Big data analytics includes three categories: descriptive analytics, which tells you what happened and why; predictive analytics, which tells you what will happen; and prescriptive analytics, which tells you what will happen, when, why and how to improve this predicted future.

Marketers, operations experts, financial officers and other business leaders have already used prescriptive analytics to improve customer experience, reduce churn, increase up-selling and cross-selling revenue, streamline logistics and enhance other important applications. For the oil and gas industry, prescriptive analytics can help locate fields with the richest concentrations of oil and gas, make pipelines safer, and improve the fracking process for greater output and fewer threats to the environment.

Horizontal drilling and hydraulic fracturing have made it possible to economically produce oil and gas from tight rocks.

About 80 percent of the world’s data today is unstructured – videos, images, sounds and texts. Until recently, most big data analytics technologies looked only at numbers. The oil and gas industry looked at images and numbers, but in separate silos. However, the ability to analyze hybrid data – a combination unstructured and structured data – provides a much clearer and more complete picture of the current and future problems and opportunities, along with the best actions to achieve the desired outcomes. For example, to improve hydraulic fracturing performance, the following datasets must be analyzed together:
• images from well logs, mud logs, seismic reports,
• videos from down-hole cameras of fluid flow,
• sounds from fracking recorded by fiber optic sensors,
• texts from drillers’ and frack pumpers’ notes, and
• numbers from production and artificial lift data.

Taking hybrid data into account is critical because of the multi-billion dollar investment and drilling decisions that are being made by the energy companies regarding where to drill, where to frack and how to frack. It calls for combining disparate computational and scientific disciplines to be able to interpret different types of data together. For example, to algorithmically interpret images (such as well logs), machine learning needs to be combined with pattern recognition, computer vision and image processing. Mixing these different disciplines provides more holistic recommendations regarding where and how to drill and frack, while reducing the chances of problems that could emerge along the way.

For example, by developing detailed analytical signatures – using data from production, subsurface, completion and other sources – one can better predict performing and non-performing wells in a field. This process is supported by the prescriptive analytics technology’s ability to automatically digitize and interpret well logs to create depositional maps of the subsurface. With a better idea of where to drill, companies save invaluable resources by skipping wells that shouldn’t be drilled in the first place. At the same time, they minimize damage to that particular landscape.

Prescriptive analytics can be used in other areas of oil and gas production. In both traditional and unconventional wells, by using data from pumps, production, completion and subsurface characteristics, one can predict failures of electric submersible pumps and prescribe actions to mitigate production loss. Apache Corp., for example, is using analytics to predict failures in pumps that pull oil out from subsurface and preempt the associated production loss from these pump failures.

Another potential application of prescriptive analytics is that it can possibly predict corrosion development or cracks in pipelines and prescribe preventive and preemptive actions by analyzing video data from cameras along with other data from robotic devices called “smart pigs” inside these pipelines.

Smarter decisions equal fewer resources, lower environmental impact and greater yields. Successful companies will be the ones that know how to prioritize resources to extract, produce and transport oil and gas in the most efficient and safest manner. Look for big data and prescriptive analytics to play a much bigger role in this space over the coming years.

Atanu Basu is CEO of AYATA, a software company headquartered in Austin, Texas. AYATA’s prescriptive analytics software focuses on improving oil and gas exploration and production. Basu is a member of INFORMS. A version of this article appeared in DataInformed.

business analytics news and articles

Related Posts

  • 62
    July/August 2013 By Warren Wilson Throughout the century-and-a-half since the dawn of the commercial petroleum industry, oilmen have always hoped for the gusher – the big find that would spew enough oil to make them rich. There have always been far more “dry holes” than gushers, however, and the proportion…
    Tags: oil, data, analytics, gas, production, prescriptive
  • 60
    November/December 2014 The marriage of two “natural resources” – hydrocarbons and data – will transform unconventional oil development. By Atanu Basu (right), Daniel Mohan and Marc Marshall Known-knowns, known-unknowns and unknown-unknowns. Donald Rumsfeld’s notable turn of phrase is an apt characterization of where we are with unconventional oil development today.…
    Tags: data, oil, analytics, prescriptive, energy, gas
  • 53
    November/December 2012 The advent of the “digital oil field” helps produce cost-effective energy while addressing safety and environmental concerns. Finding and producing hydrocarbons is technically challenging and economically risky. By Adam Farris Everyone needs it, few know how we get it, and many feel compelled to slow down efforts to…
    Tags: data, oil, gas, energy, big
  • 42
    As the Big Data Analytics space continues to evolve, one of the breakthrough technologies that businesses will be talking about in the coming years is prescriptive analytics. The promise of prescriptive analytics is certainly alluring: it enables decision-makers to not only look into the future of their mission critical processes…
    Tags: analytics, prescriptive, data
  • 41
    July/August 2014 The story of how IBM not only survived but thrived by realizing business value from big data. By (l-r) Brenda Dietrich, Emily Plachy and Maureen Norton This is the story of how an iconic company founded more than a century ago, and once deemed a “dinosaur” that would…
    Tags: analytics, data, big


Using machine learning and optimization to improve refugee integration

Andrew C. Trapp, a professor at the Foisie Business School at Worcester Polytechnic Institute (WPI), received a $320,000 National Science Foundation (NSF) grant to develop a computational tool to help humanitarian aid organizations significantly improve refugees’ chances of successfully resettling and integrating into a new country. Built upon ongoing work with an international team of computer scientists and economists, the tool integrates machine learning and optimization algorithms, along with complex computation of data, to match refugees to communities where they will find appropriate resources, including employment opportunities. Read more →

Gartner releases Healthcare Supply Chain Top 25 rankings

Gartner, Inc. has released its 10th annual Healthcare Supply Chain Top 25 ranking. The rankings recognize organizations across the healthcare value chain that demonstrate leadership in improving human life at sustainable costs. “Healthcare supply chains today face a multitude of challenges: increasing cost pressures and patient expectations, as well as the need to keep up with rapid technology advancement, to name just a few,” says Stephen Meyer, senior director at Gartner. Read more →

Meet CIMON, the first AI-powered astronaut assistant

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, made its debut aboard the International Space Station. The ISS’s newest crew member, developed and built in Germany, was called into action on Nov. 15 with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8. Read more →



INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to