Energy-efficient system for detection of elephants with Machine Learning

Author: Marko Sagadin, Embedded Machine Learning Engineer at IRNAS

Introduction

Classifying images has come a long way in the last decade; at this point, it is quite a norm in the field of web, mobile and desktop applications, however, it is not yet widely used in embedded systems.
Recent developments in the field of embedded Machine Learning, also known as TinyML, have pushed boundaries of what is possible. Thanks to TensorFlow Lite for Microcontrollers and Edge Impulse, we can now efficiently run neural networks on small embedded devices.

The main focus of this thesis was the design, implementation and evaluation of Machine Learning models running on a general-purpose microcontroller under low-power conditions. The majority of thermal images was provided by Arribada Initiative, who have already done some grunt work in elephant detection as seen here and here. Thermal images are publicly available on their GitHub repository.

This blog post is a short summary of my work, for an in-depth description of the problem, implementation and results, I suggest looking at my GitHub repository.

1. Early warning system

The reason are Human-Elephant Conflicts (HEC), as they are a major problem in terms of elephant conservation.
According to WILDLABS, an average of 400 people and 100 elephants are killed every year in India alone because of them. One of the proposed solutions to the problem are the early warning systems.
Early warning systems replace the role of human watchers and warn local communities of nearby, potentially life-threatening, elephants, thus minimising Human-Elephant Conflicts.

A system capable of detecting the presence of nearby elephants would warn nearby communities and give them enough time to prepare and respond non-violently. The same kind of system would also provide information about common elephant paths, thus giving farmers knowledge on how to construct and place their fences better to minimise HEC.

The system should consist of several small, deployed devices with mounted cameras that will detect elephants and communicate wirelessly with a server. The server would display messages and alerts, and if needed, directly notify the local community, if elephants were detected in the local area.

Deployed devices will be placed far from the main server, in an area with an unreliable cellular connectivity, which makes sending a large amount of data a problem.
This limits the choice of wireless networks to long-range, low bandwidth technologies.
It is, therefore, preferable that elephant detection is done on deployed devices, and only results (which can be few bytes big) are sent over some radio network to the main server. Deployed devices will be placed in forests, fields, with no access to electricity, therefore, they need to be battery powered.

Low maintenance of deployed devices is a desirable quality, which means that they should be functional for longer periods without any human interactions. To achieve that with a limited power source, they should be energy-efficient, equipped with solar panels, and a low power radio. Devices should spend most of their time in sleep mode, conserving energy, only waking up to take a photo, processing it, and sending results to the main server.

2. Developing Neural Network

These objects were highly likely to be captured by a thermal camera, thus they needed to be reliably classified. I cleaned up the image dataset and extracted images suitable for our classes.

For the model I chose a simple Convolutional Neural Network with a small number of layers, implementation in Keras can be seen below:

model = models.Sequential()model.add(Conv2D(FilterNum1, FilterSize, 
activation='relu',
padding="same",
input_shape=(60,80, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(FilterNum2, FilterSize,
activation='relu',
padding="same")
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(FilterNum3, FilterSize,
activation='relu',
padding="same")
model.add(Flatten())
model.add(Dense(DenseSize, activation='relu'))
model.add(Dropout(DropoutRate))
model.add(Dense(4), activation='softmax')

Notice that I did not hard-code the number of filters, their size or size of fully connected layers.

Because these values are hard to determine heuristically, I decided on a possible hyperparameter range and used Keras Tuner module to perform a random search and train a large number of models. As a comparison we also used Edge Impulse Studio to train Neural Network models, some of them had similar general convolutional structure to ours, others were trained with Transfer Learning technique, which used a pre-trained MobileNetV2 network.

3. Hardware and firmware

General sequence of elephant detection is seen on the figure below.

A large part of this thesis was concerned with porting TensorFlow for Microcontrollers to the libopencm3, our platform of choice. During this process, we developed a small project that we named MicroML. MicroML enables users to develop ML applications on microcontrollers supported by libopencm3.

NRF52840 microcontroller firmware was developed with the help of Zepyhr RTOS.

4. Results

We also evaluated the performance of the embedded system from the battery lifetime perspective and we saw promising results. Assuming that our system would have to process 600 events per day, while connected from 6 battery cells, we estimated the lifetime of the system would be around 10 months.

Conclusion

Machine Learning on the embedded devices is opening doors to various, wonderful applications that were not possible a few years ago. We can see that there is already a demand for intelligence for devices on the “edge” and we expect it to increase as the field matures.

It is quite possible that future embedded engineers would require some amount of Machine Learning knowledge to stay competitive and interesting to the market.

We are applying today’s knowledge to create systems for an open future.