The LGP plant maintenance is managed by using:
- reactive maintenance, where the process of repairing assets to standard operating conditions occur after poor performance or breakdown is observed;
- preventive time-based maintenance, where assets are maintained on a calendar-based schedule. This approach leads to challenges with plant efficiency, shorter life expectancies of assets, safety issues, sporadic asset downtime and overspending. Quality control was human based and prone to errors. End customers would return entire product batches in case of defective products.
IoT sensors measuring features such as temperature, humidity, pressure, vibrations, rotational speed, electric current are traditionally used for machine condition monitoring and predictive maintenance. With advances in Computer Audition, acoustic based predictive maintenance is becoming possible.
So an SAP Partner proposed to implement a Smart Factory Application on top of SAP BTP to improve manufacturing operations and raise awareness on sustainability KPIs. Machine condition monitoring based on sound classification was implemented to replace time-based preventive maintenance with proactive (and predictive) maintenance and to prevent downtimes.
- Reduced energy consumption.
- Extended lifetime of machinery and therefore reduced waste.
- Reduced operational and maintenance costs that will lead to reduced price of end product.
- Reduced risk of incidents leading to a safer work place.
Sound is a pressure wave that propagates through air. Sound files record the amplitude of this wave captured by one or more microphones at discrete time intervals. Sounds files come with a few fundamental metadata: sampling rate, bits per sample, number of channels.
Sound can be represented in two domains: time domain or frequency domain. From both representation, it’s possible to extract features that can be the input for analytics model. Log Mel Spectrogram and MFCCs are very commonly used, especially (but not only) for the description of human vocal tract. We use the spectrogram as input for a Deep Learning model and reduce the problem to an image classification task.
A Convolutional Neural Network (CNN) is a Deep Learning algorithm which can take in an input image, assign importance to various aspects/objects in the image and be able to differentiate one from the other. The architecture of a CNN is analogous to that of the connectivity pattern of Neurons in the Human Brain and was inspired by the organization of the Visual Cortex.
Conventionally, the first CNN is responsible for capturing the Low-Level features such as edges, colour, gradient orientation, etc. With added layers, the architecture adapts to the High-Level features as well, giving a network which has the wholesome understanding of images in the dataset.
The Pooling layer is responsible for reducing the spatial size of the Convolved Feature. This is to decrease the computational power required to process the data through dimensionality reduction. Furthermore, it is useful for extracting dominant features which are rotational and positional invariant In the last part of a CNN a regular Neural Network for classification purposes learns non-linear combinations of the high-level features as represented by the output of the convolutional layer.
-
Accuracy is one metric for evaluating classification models. Informally, accuracy is the fraction of predictions our model got right. It is the ratio of number of correct predictions to the total number of input samples.
-
SparseCategoricalCrossentropy is used usually when there are two or more label classes and labels are provided as integers.
We assume that for each machine a dataset of sounds clips is available. Sound clips are annotated in three category: OK, Anomaly Type I, Anomaly Type II . This means we know how the machine sound like in normal vs anomalous conditions. Therefore, we will choose a supervised approach (Acoustic Scene Classification). Disclaimer: The dataset is custom made and the exercise has the only purpuse to demo how we can use deep learning for sound analysis. This is not to be taken blindly as a real-life working model.
In this folder you can find everything used to develop the solution to the business challenge of acoustic based proactive maintenance with Deep Learning techniques in SAP AI Core and SAP AI Launchpad. Please, find below the list and description of each folder and their content:
- code
- train
- It contains the python code to build the sound classification model, to preprocess the data and to train the model itself on SAP AI Core.
- It contains also the requirements.txt and the Docker file to dockerize the train.py python code.
- infer
- It contains the code for the serving web application that will be deployed in SAP AI Core to serve the inference requests to the sound classification model.
- It contains also the requirements.txt and the Docker file to dockerize the infer.py python code.
- train
- LGPsound
- It contains the custom made datasets divided per category: OK, Anomaly Type I, Anomaly Type II.
- This is basically the dataset that is already uploaded in the AWS S3 bucket at the path:
aws s3 ls s3://ai-sustainability-dataset/sound/data/
- exercises
- files
- This folder contains the credential files to be filled in order to go through exercise 2.
- This folder contains also the training and serving template files to be customized in order to go through exercise 2.
- sound-classification-part1.ipynb
- This is the Jupyter notebook with the AI Core one-time configurations that a participant is required to go through during exercise 2.
- It contains detailed instructions for all the steps.
- sound-classification-part2.ipynb
- This is the Jupyter notebook to follow in order to execute the training of the sound classification model and retrieve the metrics in SAP AI Core.
- It contains detailed instructions for all the steps.
- files
- notebooks (ONLY for reference, not needed for exercise 2)
- sound_based_predictive_maintenance.ipynb
- This is the Jupyter notebook where all the development steps are tested: loading the data, preprocess the data, prepare the datasets for training, validation and test, build and initialize the model, test the model).
- sound_based_predictive_maintenance_in_AICore.ipynb
- This is the Jupyter notebook to execute all the one-time configurations in SAP AI Core.
- main.ipynb
- This is the Jupyter notebook to execute the training of the sound classification model and retrieve the metrics in SAP AI Core.
- This is also the Jupyter notebook to execute the deployment of the sound classification model in SAP AI Core and that shows how to use the exposed URL to make an inference.
- NB: These notebooks are intended to be used locally.
- sound_based_predictive_maintenance.ipynb
- workflows
- training_workflow_seg.yaml
- It's the template that specifies the training parameters and the training workflows in terms of Docker containers.
- This file has to be loaded in the GitHub repository that will be connected to SAP AI Core.
- serving_workflow_seg.yaml
- It's the template that specifies all the parameters needed to deploy and serve the web application.
- This file has to be loaded in the GitHub repository that will be connected to SAP AI Core.
- training_workflow_seg.yaml
train.py is a Python class which implements all the required steps developed and tested in the Jupyter development notebook. This piece of code can be run in SAP AI Core and is expected to include the following methods:
- read_dataset: to load the images and apply some pre-processing;
- split_dataset: to split the dataset among train, validation and test samples;
- build_model: to build the chosen mode;
- init_model: to initialize and compile the model:
- train_model: to configure the training and execute it;
- save_model: to save the model;
- get_model: to load the model;
- model_metrics: to define and register the chosen metrics.
Create an issue in this repository if you find a bug or have questions about the content.
For additional support, ask a question in SAP Community.
If you wish to contribute code, offer fixes or improvements, please send a pull request. Due to legal reasons, contributors will be asked to accept a DCO when they create the first pull request to this project. This happens in an automated fashion during the submission process. SAP uses the standard DCO text of the Linux Foundation.
Copyright (c) 2022 SAP SE or an SAP affiliate company. All rights reserved. This project is licensed under the Apache Software License, version 2.0 except as noted otherwise in the LICENSE file.