Integrating AI in Embedded Systems

Since the first open Large Language Models (LLM) were released and anyone could prompt them in their language of choice and receive a human-friendly answer, AI has been a red-hot topic. However, these solutions are not the only use of neural networks and other AI models. When you think about AI you probably think of substantial processing power and big data collections, like a rack of expensive Nvidia hardware, hours of training processes etc. This doesn’t always have to be the case – you can also run a simple AI model on a microcontroller from 15 years ago. Read on to learn how embedded engineers are effectively adding AI solutions to their toolboxes.

NanoEdge AI studio

Thanks to the ST company, which releases a free environment for their products, the process of generating an AI static library is very simple. The typical use case here involves predictive maintenance – you can detect when your million-dollar industrial machine starts to behave in a strange way, so you can schedule maintenance in advance, without having to wait for it to crash and cause big expenses. All you have to do is provide a regular (or abnormal – if you want to use Anomaly Detection mode – see paragraph below) signals (accelerometer, gyroscope, temperature, light, whatever), and the NanoEdge will look for the best model that matches your input data and gives the best predictability. The actual origin of the data doesn’t matter for the software – the data is just numbers, so you can give any measurable input for the NanoEdge and even combine it – e.g. temperature and speed, humidity and raw voltage. Of course, the values should be meaningful in a way that their values correlate to the process they describe – at the end of the day, the algorithms are looking for patterns, so they must exist.

Anomaly detection

The Anomaly Detection mode is the first of the two modes we’re going to look into. It is the only mode that is provided by NanoEdge AI Studio that enables users to do the learning process directly on the edge device – the API allows us to train a model even when a device is already deployed but let’s say, for some reason, we want to align the device operation. This mode is also very convenient to deploy – let’s assume there are 10 industrial machines of the same type that should be monitored for anomalies – each of the machines is slightly different (production date, set of components, etc.). Having software that can be trained on each of the machines independently achieves an easy installation and deploy process. This is the biggest advantage of Anomaly Detection mode.

What is actually needed to use Anomaly Detection mode? In order for NanoEdge AI Studio to choose and compile a library you can later use, you need to provide a set of reference signals collected from the machine you want to monitor (Figure 1) – a ‘good’ signal and an ‘anomaly’ example. In some cases, this may be challenging, but if you’re able to collect abnormal signals, the Anomaly Detection mode is probably the most convenient option to start with. It’s worth noting that the abnormal signals are just needed as a context for further library generation process, they are not used for training a model, so the signal does not have to be an actual anomaly that is expected to happen.

Figure 1: Provide source signal to NanoEdge AI Studio

After you provide a normal and abnormal signal to the NanoEdge, it will do the benchmark (Figure 2) and will try to find a model that best suits the type of signal you provided.

Figure 2: Benchmark process

After you choose the best library for your needs (not only accuracy, but for embedded devices the RAM and FLASH consumption as well), you can compile and download the static library and a header with API description and include it in your application. Of course, the logic of the application should be written according to a particular use case. In the case of Anomaly Detection mode, the function returns the match to the original pattern in a percentage (0-100%). You need to decide yourself at which point the ‘anomaly’ occurs.

1-class classification

A second option worth considering when using predictive maintenance is 1-class classification (binary classification). Here the user doesn’t have to provide the abnormal signals to the NanoEdge AI studio, just the desired ones. As an output, just as in Anomaly Detection mode, the static library is generated, but with a slightly different API, which only allows binary classification, so the result is 0 or 1 (true or false). In this situation the decision is made by the algorithm itself (you cannot decide yourself at which point you should report an issue, like in Anomaly Detection mode, where the output was the percentage of a match). The process of creating a library is the same as for Anomaly Detection mode, the only difference is that you don’t have to provide the abnormal signals.

Issues

Let’s talk a bit about the whole process of creating a solution that uses an AI-generated library. First, you need to obtain data from the object you want to monitor. At this point you need to decide which parameters describe your object in the most reliable way. For our team, dealing with a vibrating object, the acceleration seemed reasonable (gyroscope also wouldn’t hurt), so we decided on a 3-axis accelerometer. Secondly, you need to adjust the sampling parameters like frequency and accuracy, so the measurements will reflect the actual state of the object. For periodic signals it’s crucial to sample at least 2x faster that the oscillation frequency, to avoid aliasing. Once you’ve chosen all the parameters, you need to implement the data acquisition software, ideally as a part of the final solution, so the data will be collected the same way later on in inference mode. This can be a challenge, especially if you sample data very fast, and you want to print it in real time. Other ways, like saving data to a SD card, may be helpful as well. For data collection there is dedicated software provided by ST, but in my case it was unstable and painful to use, so in the end I implemented my own data acquisition software – you have to implement your own application with business logic anyway, and ideally it should collect the data the same way the training data was collected, so this is not much extra work.

If you have already the data obtained, it’s good to confirm there are no glitches in the measurements – that was the case for me. Just after the start, the accelerometer returned some crazy values, so you have to remember, that this part of the job is more analytic and should be regarded as data science. If everything looks fine, then the worst part is done, and if the data collection process is automated you can play with different AI models and see how they behave.

Figure 3: The first line is a glitch – avoid that

Conclusion

Both of the approaches have been tested on a simple vibrating object (a small 3-speed fan). The sensor used was a 3-axis accelerometer with 6kHz sampling rate (which was a bit of overkill for this case actually). I can say that Anomaly Detection mode was far more convenient and easier to use and deploy for me – in this mode you choose the machine learning model only once during initial phase, and then the training, quick and intuitive, is done in real-time and gives satisfying results. For 1-class classification mode, the significant issue is that you need to rebuild the library each time you deploy the device, and sometimes if you unmount and mount the device again it may be in slightly different position and thus the result of operation may be different. In this mode you also loose the 0-100% of certainty, so you have less information on the output.

Companies around the world turn to Software Mind for embedded software expertise. To find out how our specialists can help you, get in touch by filling out the the contact form.