Energy-efficient system for detection of elephants with Machine Learning

Author: Marko Sagadin, Embedded Machine Learning Engineer at IRNAS

Introduction

I recently wrote a Master’s thesis on detecting elephants from thermal cameras, with the help of Machine Learning. Thesis is publicly available on my GitHub repository.

1. Early warning system

But why exactly would we want to classify elephants from thermal cameras?

2. Developing Neural Network

I developed a Neural Network with the help of TensorFlow, Keras and Jupyter Notebooks.
As I wanted to develop a small enough model, which would be able to fit on a microcontroller we decided to only classify four distinct classes: Elephants, humans, cows and nature/random.

model = models.Sequential()model.add(Conv2D(FilterNum1, FilterSize, 
activation='relu',
padding="same",
input_shape=(60,80, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(FilterNum2, FilterSize,
activation='relu',
padding="same")
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(FilterNum3, FilterSize,
activation='relu',
padding="same")
model.add(Flatten())
model.add(Dense(DenseSize, activation='relu'))
model.add(Dropout(DropoutRate))
model.add(Dense(4), activation='softmax')

3. Hardware and firmware

To capture thermal images, we used FLIR Lepton 2.5 thermal camera, which was used by Arribada Initiative to create thermal image dataset. For running inference I chose STM32F767ZI microcontroller with 216 MHz system clock, 2 MB of flash and 512 KB of RAM. Using such powerful microcontroller guaranteed us that we would be able to fit a model in the memory and run it as fast as possible.
However, such high performance comes with a large current consumption, STM32 was reaching over 125 mA during inference. As we did not want both the thermal camera and STM32 microcontroller running all the time we needed another microcontroller which could turn them on and off. For that, we used NRF52840 microcontroller, which has a low current consumption during sleep state and is ideal for monitoring the whole system. The block diagram of the system can be seen below, for wireless communication module we used LoRa transceiver LR1110 chip.

4. Results

We saw that we could train CNN models with an accuracy as high as 98.04 % and we saw that models trained with the Transfer Learning technique reached 98.7 % without much additional work. We ran each trained model on the STM32 and measured its inference time. From the results, we saw that reaching an inference time below 200 ms is not hard if special care is devoted to the low number of parameters and correct microcontroller optimisations.

Conclusion

This thesis shows that the field of Machine Learning on embedded devices has reached a point where it is viable enough to use it in real-life applications. Running inference directly on the embedded device can provide instant feedback and can extend the device’s lifetime, as data do not need to be sent to the server for processing. Many use cases from the Animal Conservation field would benefit from such technology, as there is a need for the embedded devices that require minimal manual care and can provide big value for their cost.

We are applying today’s knowledge to create systems for an open future.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store