A targeted one dimensional fully convolutional autoencoder network for intelligent compression of magnetic flux leakage data

Abstract

Abstract In response to the issue of massive data volume generated by magnetic flux leakage (MFL) non-destructive testing in oil and gas pipelines, an intelligent data compression method based on a targeted one-dimensional fully convolutional autoencoder network is proposed. Firstly, a data preprocessing module is designed to generate high-quality data required for subsequent processing, taking into account the characteristics of MFL data. Secondly, a data block classification algorithm is developed to calculate peak values for segmented differential data, and based on a predefined targeted threshold, distinguish different types of MFL data. Subsequently, based on the distinct data types, targeted one-dimensional fully convolutional autoencoder models are constructed to effectively achieve dimensionality reduction compression and reconstruction of the MFL data. Through practical experimental analysis, the reconstruction error such as MAE is reduced by about 27.7% and the compression ratio is improved by about 14% compared with traditional methods such as PCA. In addition, compared with ID-AE, the proposed 1D-FCAE reduces 206.8 k, 1.58G, and 80 s in parameters, memory usage, and training time, respectively, and reduces compression and decompression time by 60 ms and 69 ms, respectively, validating that it is easy to be applied in industrial environments with limited resources.