Predicting Pipeline Failures Using Data Science

Bottom-side internal corrosion (IC) is a subtle but dangerous threat to pipeline infrastructure. Often progressing silently in regions where water settles due to gravity, this form of corrosion can result in pinhole leaks that go undetected until a failure occurs. At Irth Solutions, we’ve developed a phenomenological model that uses inline inspection (ILI) data to identify potential release sites caused by bottom-side IC, even when pinholes have not been identified by the ILI system.

The Problem: 

Why Bottom-Side Internal Corrosion Matters

According to PHMSA, internal corrosion accounts for nearly 13% of pipeline incidents, with localized pitting responsible for a significant share of these. These failures often occur at low points in the pipeline where corrosive fluids and microbes concentrate.

Yet, pinholes, the smallest but most critical defects, often evade detection or mitigation due to:

• Limitations of older ILI tool technologies
• Conservative thresholds used by some vendors
• Orientation outside critical detection zones (e.g., 4–8 o’clock position)

Theoretical IC Models

Internal corrosion in pipelines is governed by a multitude of factors, such as the chemical composition of fluids, microbiological activity, operational conditions, material factors, and the elevation profile. The intricacy of the internal corrosion process, coupled with the need for detailed knowledge of all variables, makes it impractical to establish a universal theoretical model for accurately predicting corrosion rates at a specific section of a pipeline at a given time. Some theoretical models incorporate over 60 parameters!

Our model is designed to work around these limitations.

Motivation

This model was inspired by a case involving a bottom-side pinhole release reported by an operator, where no prior ILI had detected the pinhole in question. The nearest reported metal loss anomaly was a pinhole with a measured depth of 17% located outside of the 4-to-8 o’clock region. The operator came to Irth Solutions to see if we could somehow have predicted the failure location.

Model Design

We utilized a publicly available dataset of release incidents since 2010 reported by operators of hazardous liquid pipelines. From this dataset, we selected incidents where internal corrosion was identified as the cause and visual inspection confirmed localized pitting. This selection resulted in 438 lines. Of these, ILI data were available for 24 lines from 6 operators. Upon reviewing the ILI data, some lines were excluded due to an insufficient number of anomalies, either because of the tool technology used or other factors. Ultimately, eight lines from 3 operators were selected as the final dataset, comprising 12 release sites. Given the limited data availability, the entire dataset was used for model development without splitting it into training and testing subsets.

How the Model Works

The model calculates a release score using a combination of spatial, statistical, and physical indicators derived from:

• anomaly data from ILI reports
• elevation profiles from ILI reports
• PHMSA release records

Each pipeline is segmented using a rolling window (e.g., 50 ft), and multiple factors are assessed in each segment.

Key Inputs to the Model

Figure 1: The results of the IC susceptibility model, alongside the active corrosion model
outcomes, and the elevation profile for a pipeline. The ILI data plot shows pinholes as red dots
and non-pinholes as blue dots. The release region is highlighted in blue in the ILI data plot.

Model Validation

In the original pipeline, which provided the impetus for creating this model, the model successfuly:

• Detected active corrosion growth in this segment
• Identified depth outliers and elevated pinhole density
• Assigned the highest susceptibility score to the failure zone

The model was subsequently applied to a pipeline with two releases that were not part of the original dataset, successfully identifying one release location. After incorporating the pipeline into the dataset and performing final tuning, the model was able to identify both release locations.

To fine-tune the model, we performed a grid search over parameter ranges (e.g., window sizes, outlier thresholds, valley weights). The best configuration achieved an average performance score of 0.82, with a perfect score of 1.0 for some pipelines. The model was also successful on pipelines outside the original dataset, suggesting strong generalization potential.

Benefits for Pipeline Operators

This model is designed to be practical and accessible. It uses data most operators already have, inline inspection reports, and delivers immediate value in identifying high-risk regions.

With this model, operators can:

• Detect failure-prone regions overlooked by ILI tools
• Supplement ILI analysis with terrain-based context
• Prioritize maintenance and excavation decisions
• Justify integrity spending with quantifiable threat scores
  
  

Want to Learn More?

Do you have ILI data you'd like us to apply the model to? Would you like to see data science models designed for other anomaly types? Contact us. 

This work was presented at the 2025 AMPP Annual Conference & Expo. Fill out the following form to download the paper.