A Global Manufacturing Big Data Ecosystem for Fault Detection in Predictive Maintenance

Artificial intelligence, big data, machine learning, cloud computing, and Internet of Things (IoT) are terms which have driven the fourth industrial revolution. The digital revolution has transformed the manufacturing industry into smart manufacturing through the development of intelligent systems. In this paper, a big data ecosystem is presented for the implementation of fault detection and diagnosis in predictive maintenance with real industrial big data gathered directly from large-scale global manufacturing plants, aiming to provide a complete architecture which could be used in industrial IoT-based smart manufacturing in an industrial 4.0 system. The proposed architecture overcomes multiple challenges including big data ingestion, integration, transformation, storage, analytics, and visualization in a real-time environment using various technologies such as the data lake, NoSQL database, Apache Spark, Apache Drill, Apache Hive, OPC Collector, and other techniques. Transformation protocols, authentication, and data encryption methods are also utilized to address data and network security issues. A MapReduce-based distributed PCA model is designed for fault detection and diagnosis. In a large-scale manufacturing system, not all kinds of failure data are accessible, and the absence of labels precludes all the supervised methods in the predictive phase. Furthermore, the proposed framework takes advantage of some of the characteristics of PCA such as its ease of implementation on Spark, its simple algorithmic structure, and its real-time processing ability. All these elements are essential for smart manufacturing in the evolution to Industry 4.0. The proposed detection system has been implemented into the real-time industrial production system in a cooperated company, running for several years, and the results successfully provide an alarm warning several days before the fault happens. A test case involving several outages in 2014 is reported and analyzed in detail during the experiment section.