TinyML Benchmark: Fully Connected Neural Networks (now with Raspberry Pi Pico!)

Ever wandered how fast are the major microcontroller boards to run Tensorflow Lite neural networks? In this post we'll find it out for the case of Fully Connected networks.

** 11 April 2021: added Raspberry Pi Pico with Arduino Mbed Core **

In a previous post about TinyML benchmarks for traditional Machine Learning models I benchmarked many different classifiers from the scikit-learn package in terms of resources and execution speed.

In this post I'm going to do something very similar, except that I'll compare different boards on the task of running Tensorflow Lite Neural Networks.

The boards on the list are:

• Arduino Nano 33 BLE Sense (Cortex M4 @ 64 MHz)
• ESP32 (Xtensa dual-core @ 240 MHz)
• Feather M4 Express (Cortex M4F @ 200 MHz)
• STM32 Nucleo H743ZI2 (Cortex M7 @ 480 MHz)
• Arduino Portenta (Cortex M7 @ 480 MHz)
• Teensy 4.0 (Cortex M7 @ 600 MHz)
• Raspberry Pi Pico (Rp2040 / Cortex M0+ @ 125 MHz)

As you can see, they differ in terms of CPU and clock frequency. I discarded less powerful boards for now (Cortex M0 based), but maybe I'll add them in the future.

** update: I added the Raspberry Pi Pico to the benchmark because of the hype it created. **

The benchmarked networks topology are 3 types of full-connected networks:

• 1 layer with 10 neurons
• 2 layers, one with 10 neurons, the other with 50 neurons
• 10 layers, each with 10 neurons

Inference times

The following charts show the inference time (in microseconds) of the different networks for each board, in linear and logarithmic scale.

What's the verdict?

• Teensy 4.0 is the fastest, as you can expect from its faster clock
• Arduino Portenta and Nucleo H743ZI2 are on quite par, since they share two CPUs from the same family, but the Nucleo is faster over all the topologies
• ESP32 still has a greate performance / price ratio, if you consider that I paid mine less than 4 $
• Raspberry Pi Pico is the slowest, despite not having the slowest clock (but the Arduino Nano 33 BLE Sense has a Cortex M4 CPU)

Do these benchmarks match with your conjectures?

Are you surpised from some of these numbers?

Would you like to see other boards benchmarked?

Let me know in the comments.

Raw data

Fully connected benchmarks