Data Avalanche and the ‘Great Crew Change’ in the Oil and Gas Industry

In an August article for McKinsey Insights the authors – Stefano Martinotti, Jim Nolten, and Jens Arne Steinsbø – write that “the rapid progress of technology such as big data and analytics, sensors, and control systems offers oil and gas companies the chance to automate high-cost, dangerous, or error-prone tasks”, thereby positioning themselves to significantly improve their bottom lines. In their view, the digitization/automation of oil and gas production provides benefits along the upstream value chain of exploration, development, and production. In particular, the authors regard optimizing production efficiency as essential given that average production efficiency dropped in the past decade. This efficiency decline offers several opportunities for improvement: “maximizing asset and well integrity (by which we mean optimizing production without compromising health, safety, and the environment), increasing field recovery,and improving oil throughput.”

Automation Opportunities along the Upstream Value Chain

Source:McKinsey Insights 

Importantly, the authors list the following industry-wide challenges they believe automation can successfully tackle and/or ameliorate:

1. More complex operations due to increasing volume and complexity in hostile, remote
locations require reliable remote and automated or semi-automated operations.

2. Industry Imperative of preventing health, safety, and environmental incidents: Incidents of such nature can be detrimental not only to the bottom line but can put the very existence of any operator in jeopardy. “Automated production control, monitoring the condition of the equipment, and predictive shutdown systems are now basic requirements to prevent or mitigate catastrophic events in geographically dispersed remote operations.”

3. The ‘Great Shift Change’ – resulting in a coming knowledge and experience gap for the industry – spurs “efforts to codify many routine analysis and decision-support processes and, where possible, to automate them” because new recruitment is unlikely to fill the talent gap completely.

What does the above actually mean in practice? The McKinsey authors acknowledge that automation is not without its own challenges and that “converting this complex flood of data into better business and operating decisions requires new, carefully designed capabilities for data manipulation, analysis, and presentation, as well as tools to support decision making.”

Here, a fabulous new book explains how to optimize exploration and production with data-driven models. In “Harness Oil and Gas BIG DATA with Analytics” Keith R. Holdaway advocates for the development of “automated and semi-automated methodologies that [critically] implement soft computing techniques.” ‘Soft computing’ is a branch of computational intelligence used to mimic the human reasoning and decision-making process. Obviously, it is a ‘soft’ technique because any automation based on what humans would regard as intelligent behavior in a particular situation can only be an approximation.

Mr. Holdaway calls it an industry imperative to transform data-driven models because “the aggregation of datasets that are voluminous, complex, disparate, and/or collated at very high frequencies, resulting in substantive analytical difficulties (…) cannot be addressed by traditional data processing applications and tools.” He identifies as most prominent issues in the upstream oil and gas industry “data management, quantifying uncertainty in the subsurface, and risk assessment around field engineering silos” and concludes that “data-driven models offer incredible insight, turning raw Big data into actionable knowledge.” According to Mr. Holdaway, four major trends have an impact on the entire exploration and production value chain: Big data, the cloud, social media, and mobile devices:

“These drivers are steering geoscientists at varying rates toward the implementation of soft computing techniques. (…) Advances in data aggregation, integration, quantification of uncertainties, and soft computing methods are enabling supplementary perspectives on the disparate upstream data to create more accurate reservoir models in a timelier manner.”

In sum, it is not automation alone but the data-driven development of “automated and semi-automated methodologies that implement soft computing techniques” that can help bridge the coming gap as a result of the ‘great shift change’ – see extensive Breaking Energy coverage here – or as Mr. Holdaway puts it:

“Retain the many years of experience by developing a collaborative analytical center of excellence that incorporates soft skills and expertise with the most important asset in any oil and gas operation: data.”

Breaking Energy