Novel Transformer Based on Gated Convolutional Neural Network for Dynamic Soft Sensor Modeling of Industrial Processes

Industrial process data are usually time-series data collected by sensors, which have the characteristics of high nonlinearity, dynamics, and noises. Many existing soft sensor modeling methods usually focus on dominant variables and auxiliary variables at a single time point while ignoring the timing characteristics of industrial process data. Meanwhile, the soft-sensing methods considering timing characteristics based on the deep learning are usually faced with gradient vanishing and the difficulty in parallel computing. Therefore, a novel Gated Convolutional neural network-based Transformer (GCT) is proposed for dynamic soft sensor modeling of industrial processes. The GCT encodes short-term patterns of the time series data and filters important features adaptively through an improved gated convolutional neural network (CNN). Then, the multihead attention mechanism is applied to modeling the correlation between any two moments. Finally, the prediction results are obtained through a linear neural network layer with the highway connection. In this article, the experiments in the dynamic soft sensor modeling of polypropylene and purified terephthalic acid industrial processes show that the proposed method achieves state-of-the-art comparing with the back propagation neural network, the extreme learning machine, the long short-term memory (LSTM) and the LSTM based on the CNN.