![\"Arduino](\"https://eloquentarduino.github.io/wp-content/uploads/2020/02/MNIST.gif\")
In this hands-on guide about on-board SVM training we're going to see a classifier in action, training it on the Iris dataset and evaluating its performance.
\n\n
In this demo project we're going to take a know dataset (iris flowers) and interactively train an SVM classifier on it, adjusting the number of samples to see the effects on both training time, inference time and accuracy.
\n#ifdef ESP32\n#define min(a, b) (a) < (b) ? (a) : (b)\n#define max(a, b) (a) > (b) ? (a) : (b)\n#define abs(x) ((x) > 0 ? (x) : -(x))\n#endif\n\n#include <EloquentSVMSMO.h>\n#include "iris.h"\n\n#define TOTAL_SAMPLES (POSITIVE_SAMPLES + NEGATIVE_SAMPLES)\n\nusing namespace Eloquent::ML;\n\nfloat X_train[TOTAL_SAMPLES][FEATURES_DIM];\nfloat X_test[TOTAL_SAMPLES][FEATURES_DIM];\nint y_train[TOTAL_SAMPLES];\nint y_test[TOTAL_SAMPLES];\nSVMSMO<FEATURES_DIM> classifier(linearKernel);First of all we need to include a couple files, namely EloquentSVMSMO.h for the SVM classifier and iris.h for the dataset.
iris.h defines a couple constants:
FEATURES_DIM: the number of features each sample has (4 in this case)POSITIVE_SAMPLES: the number of samples that belong to the positive class (50)NEGATIVE_SAMPLES: the number of samples that belong to the negative class (50)The we declare the array that hold the data: X_train and y_train for the training process, X_test and y_test for the inference process.
void setup() {\n Serial.begin(115200);\n delay(5000);\n\n // configure classifier\n classifier.setC(5);\n classifier.setTol(1e-5);\n classifier.setMaxIter(10000);\n}Here we just set a few parameters for the classifier. You could actually skip this step in this demo, since the defaults will work well. Those lines are there so you know you can tweak them, if needed.
\nPlease refer to the demo for color classification for an explanation of each parameter.
\nvoid loop() {\n int positiveSamples = readSerialNumber("How many positive samples will you use for training? ", POSITIVE_SAMPLES);\n\n if (positiveSamples > POSITIVE_SAMPLES - 1) {\n Serial.println("Too many positive samples entered. All but one will be used instead");\n positiveSamples = POSITIVE_SAMPLES - 1;\n }\n\n int negativeSamples = readSerialNumber("How many negative samples will you use for training? ", NEGATIVE_SAMPLES);\n\n if (negativeSamples > NEGATIVE_SAMPLES - 1) {\n Serial.println("Too many negative samples entered. All but one will be used instead");\n negativeSamples = NEGATIVE_SAMPLES - 1;\n }\n\n loadDataset(positiveSamples, negativeSamples);\n\n // ...\n}\n\n/**\n * Ask the user to enter a numeric value\n */\nint readSerialNumber(String prompt, int maxAllowed) {\n Serial.print(prompt);\n Serial.print(" (");\n Serial.print(maxAllowed);\n Serial.print(" max) ");\n\n while (!Serial.available()) delay(1);\n\n int n = Serial.readStringUntil('\\n').toInt();\n\n Serial.println(n);\n\n return n;\n}\n\n/**\n * Divide training and test data\n */\nvoid loadDataset(int positiveSamples, int negativeSamples) {\n int positiveTestSamples = POSITIVE_SAMPLES - positiveSamples;\n\n for (int i = 0; i < positiveSamples; i++) {\n memcpy(X_train[i], X_positive[i], FEATURES_DIM);\n y_train[i] = 1;\n }\n\n for (int i = 0; i < negativeSamples; i++) {\n memcpy(X_train[i + positiveSamples], X_negative[i], FEATURES_DIM);\n y_train[i + positiveSamples] = -1;\n }\n\n for (int i = 0; i < positiveTestSamples; i++) {\n memcpy(X_test[i], X_positive[i + positiveSamples], FEATURES_DIM);\n y_test[i] = 1;\n }\n\n for (int i = 0; i < NEGATIVE_SAMPLES - negativeSamples; i++) {\n memcpy(X_test[i + positiveTestSamples], X_negative[i + negativeSamples], FEATURES_DIM);\n y_test[i + positiveTestSamples] = -1;\n }\n}The code above is a preliminary step where you're asked to enter how many samples you will use for training of both positive and negative classes.
\nThis way you can have multiple run of benchmarking without the need to re-compile and re-upload the sketch.
\nIt also shows that the training process can be "dynamic", in the sense that you can tweak it at runtime as per your need.
\ntime_t start = millis();\nclassifier.fit(X_train, y_train, positiveSamples + negativeSamples);\nSerial.print("It took ");\nSerial.print(millis() - start);\nSerial.print("ms to train on ");\nSerial.print(positiveSamples + negativeSamples);\nSerial.println(" samples");Training is actually a one line operation. Here we'll also logging how much time it takes to train.
\nvoid loop() {\n // ...\n\n int tp = 0;\n int tn = 0;\n int fp = 0;\n int fn = 0;\n\n start = millis();\n\n for (int i = 0; i < TOTAL_SAMPLES - positiveSamples - negativeSamples; i++) {\n int y_pred = classifier.predict(X_train, X_test[i]);\n int y_true = y_test[i];\n\n if (y_pred == y_true && y_pred == 1) tp += 1;\n if (y_pred == y_true && y_pred == -1) tn += 1;\n if (y_pred != y_true && y_pred == 1) fp += 1;\n if (y_pred != y_true && y_pred == -1) fn += 1;\n }\n\n Serial.print("It took ");\n Serial.print(millis() - start);\n Serial.print("ms to test on ");\n Serial.print(TOTAL_SAMPLES - positiveSamples - negativeSamples);\n Serial.println(" samples");\n\n printConfusionMatrix(tp, tn, fp, fn);\n}\n\n/**\n * Dump confusion matrix to Serial monitor\n */\nvoid printConfusionMatrix(int tp, int tn, int fp, int fn) {\n Serial.print("Overall accuracy ");\n Serial.print(100.0 * (tp + tn) / (tp + tn + fp + fn));\n Serial.println("%");\n Serial.println("Confusion matrix");\n Serial.print(" | Predicted 1 | Predicted -1 |\\n");\n Serial.print("----------------------------------------\\n");\n Serial.print("Actual 1 | ");\n Serial.print(tp);\n Serial.print(" | ");\n Serial.print(fn);\n Serial.print(" |\\n");\n Serial.print("----------------------------------------\\n");\n Serial.print("Actual -1 | ");\n Serial.print(fp);\n Serial.print(" | ");\n Serial.print(tn);\n Serial.print(" |\\n");\n Serial.print("----------------------------------------\\n\\n\\n");\n}Finally we can run the classification on our test set and get the overall accuracy.
\nWe also print the confusion matrix to double-check each class accuracy.
\n\r\nCheck the full project code on Github where you'll also find another dataset to test, which is characterized by a number of features much higher (30 instead of 4).
\nL'articolo How to train a IRIS classification Machine learning classifier directly on your Arduino board proviene da Eloquent Arduino Blog.
\n", "content_text": "In this hands-on guide about on-board SVM training we're going to see a classifier in action, training it on the Iris dataset and evaluating its performance.\n\nWhat we'll make\nIn this demo project we're going to take a know dataset (iris flowers) and interactively train an SVM classifier on it, adjusting the number of samples to see the effects on both training time, inference time and accuracy.\nDefinitions\n#ifdef ESP32\n#define min(a, b) (a) < (b) ? (a) : (b)\n#define max(a, b) (a) > (b) ? (a) : (b)\n#define abs(x) ((x) > 0 ? (x) : -(x))\n#endif\n\n#include <EloquentSVMSMO.h>\n#include "iris.h"\n\n#define TOTAL_SAMPLES (POSITIVE_SAMPLES + NEGATIVE_SAMPLES)\n\nusing namespace Eloquent::ML;\n\nfloat X_train[TOTAL_SAMPLES][FEATURES_DIM];\nfloat X_test[TOTAL_SAMPLES][FEATURES_DIM];\nint y_train[TOTAL_SAMPLES];\nint y_test[TOTAL_SAMPLES];\nSVMSMO<FEATURES_DIM> classifier(linearKernel);\nFirst of all we need to include a couple files, namely EloquentSVMSMO.h for the SVM classifier and iris.h for the dataset.\niris.h defines a couple constants:\n\nFEATURES_DIM: the number of features each sample has (4 in this case)\nPOSITIVE_SAMPLES: the number of samples that belong to the positive class (50)\nNEGATIVE_SAMPLES: the number of samples that belong to the negative class (50)\n\nThe we declare the array that hold the data: X_train and y_train for the training process, X_test and y_test for the inference process.\nSetup\nvoid setup() {\n Serial.begin(115200);\n delay(5000);\n\n // configure classifier\n classifier.setC(5);\n classifier.setTol(1e-5);\n classifier.setMaxIter(10000);\n}\nHere we just set a few parameters for the classifier. You could actually skip this step in this demo, since the defaults will work well. Those lines are there so you know you can tweak them, if needed.\nPlease refer to the demo for color classification for an explanation of each parameter.\nInteractivity\nvoid loop() {\n int positiveSamples = readSerialNumber("How many positive samples will you use for training? ", POSITIVE_SAMPLES);\n\n if (positiveSamples > POSITIVE_SAMPLES - 1) {\n Serial.println("Too many positive samples entered. All but one will be used instead");\n positiveSamples = POSITIVE_SAMPLES - 1;\n }\n\n int negativeSamples = readSerialNumber("How many negative samples will you use for training? ", NEGATIVE_SAMPLES);\n\n if (negativeSamples > NEGATIVE_SAMPLES - 1) {\n Serial.println("Too many negative samples entered. All but one will be used instead");\n negativeSamples = NEGATIVE_SAMPLES - 1;\n }\n\n loadDataset(positiveSamples, negativeSamples);\n\n // ...\n}\n\n/**\n * Ask the user to enter a numeric value\n */\nint readSerialNumber(String prompt, int maxAllowed) {\n Serial.print(prompt);\n Serial.print(" (");\n Serial.print(maxAllowed);\n Serial.print(" max) ");\n\n while (!Serial.available()) delay(1);\n\n int n = Serial.readStringUntil('\\n').toInt();\n\n Serial.println(n);\n\n return n;\n}\n\n/**\n * Divide training and test data\n */\nvoid loadDataset(int positiveSamples, int negativeSamples) {\n int positiveTestSamples = POSITIVE_SAMPLES - positiveSamples;\n\n for (int i = 0; i < positiveSamples; i++) {\n memcpy(X_train[i], X_positive[i], FEATURES_DIM);\n y_train[i] = 1;\n }\n\n for (int i = 0; i < negativeSamples; i++) {\n memcpy(X_train[i + positiveSamples], X_negative[i], FEATURES_DIM);\n y_train[i + positiveSamples] = -1;\n }\n\n for (int i = 0; i < positiveTestSamples; i++) {\n memcpy(X_test[i], X_positive[i + positiveSamples], FEATURES_DIM);\n y_test[i] = 1;\n }\n\n for (int i = 0; i < NEGATIVE_SAMPLES - negativeSamples; i++) {\n memcpy(X_test[i + positiveTestSamples], X_negative[i + negativeSamples], FEATURES_DIM);\n y_test[i + positiveTestSamples] = -1;\n }\n}\nThe code above is a preliminary step where you're asked to enter how many samples you will use for training of both positive and negative classes.\nThis way you can have multiple run of benchmarking without the need to re-compile and re-upload the sketch.\nIt also shows that the training process can be "dynamic", in the sense that you can tweak it at runtime as per your need.\nTraining\ntime_t start = millis();\nclassifier.fit(X_train, y_train, positiveSamples + negativeSamples);\nSerial.print("It took ");\nSerial.print(millis() - start);\nSerial.print("ms to train on ");\nSerial.print(positiveSamples + negativeSamples);\nSerial.println(" samples");\nTraining is actually a one line operation. Here we'll also logging how much time it takes to train.\nPredicting\nvoid loop() {\n // ...\n\n int tp = 0;\n int tn = 0;\n int fp = 0;\n int fn = 0;\n\n start = millis();\n\n for (int i = 0; i < TOTAL_SAMPLES - positiveSamples - negativeSamples; i++) {\n int y_pred = classifier.predict(X_train, X_test[i]);\n int y_true = y_test[i];\n\n if (y_pred == y_true && y_pred == 1) tp += 1;\n if (y_pred == y_true && y_pred == -1) tn += 1;\n if (y_pred != y_true && y_pred == 1) fp += 1;\n if (y_pred != y_true && y_pred == -1) fn += 1;\n }\n\n Serial.print("It took ");\n Serial.print(millis() - start);\n Serial.print("ms to test on ");\n Serial.print(TOTAL_SAMPLES - positiveSamples - negativeSamples);\n Serial.println(" samples");\n\n printConfusionMatrix(tp, tn, fp, fn);\n}\n\n/**\n * Dump confusion matrix to Serial monitor\n */\nvoid printConfusionMatrix(int tp, int tn, int fp, int fn) {\n Serial.print("Overall accuracy ");\n Serial.print(100.0 * (tp + tn) / (tp + tn + fp + fn));\n Serial.println("%");\n Serial.println("Confusion matrix");\n Serial.print(" | Predicted 1 | Predicted -1 |\\n");\n Serial.print("----------------------------------------\\n");\n Serial.print("Actual 1 | ");\n Serial.print(tp);\n Serial.print(" | ");\n Serial.print(fn);\n Serial.print(" |\\n");\n Serial.print("----------------------------------------\\n");\n Serial.print("Actual -1 | ");\n Serial.print(fp);\n Serial.print(" | ");\n Serial.print(tn);\n Serial.print(" |\\n");\n Serial.print("----------------------------------------\\n\\n\\n");\n}\nFinally we can run the classification on our test set and get the overall accuracy.\nWe also print the confusion matrix to double-check each class accuracy.\n\r\n\r\n\r\n \r\n\tFinding this content useful?\r\n\r\n\t\r\n\r\n\t\r\n\t\t\r\n\t\t\r\n\t \r\n \r\n \r\n \r\n\r\n\r\n\r\n\n\nCheck the full project code on Github where you'll also find another dataset to test, which is characterized by a number of features much higher (30 instead of 4).\nL'articolo How to train a IRIS classification Machine learning classifier directly on your Arduino board proviene da Eloquent Arduino Blog.", "date_published": "2020-03-28T19:02:09+01:00", "date_modified": "2020-04-05T19:02:20+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ] }, { "id": "https://eloquentarduino.github.io/?p=931", "url": "https://eloquentarduino.github.io/2020/02/handwritten-digit-classification-with-arduino-and-microml/", "title": "Handwritten digit classification with Arduino and MicroML", "content_html": "We continue exploring the endless possibilities on the MicroML (Machine Learning for Microcontrollers) framework on Arduino and ESP32 boards: in this post we're back to image classification. In particular, we'll distinguish handwritten digits using an ESP32 camera.
\n
\n
If this is the first time you're reading my blog, you may have missed that I'm on a journey to push the limits of Machine learning on embedded devices like the Arduino boards and ESP32.
\nI started with accelerometer data classification, then did Wifi indoor positioning as a proof of concept.
\nIn the last weeks, though, I undertook a more difficult path that is image classification.
\nImage classification is where Convolutional Neural Networks really shine, but I'm here to question this settlement and demostrate that it is possible to come up with much lighter alternatives.
\nIn this post we continue with the examples, replicating a "benchmark" dataset in Machine learning: the handwritten digits classification.
\nThe objective of this example is to be able to tell what an handwritten digit is, taking as input a photo from the ESP32 camera.
\nIn particular, we have 3 handwritten numbers and the task of our model will be to distinguish which image is what number.
\n![\"Handwritten](\"https://eloquentarduino.github.io/wp-content/uploads/2020/02/mnist-examples.jpg\")
I only have a single image per digit, but you're free to draw as many samples as you like: it should help improve the performance of you're classifier.
\nWhen dealing with images, if you use a CNN this step is often overlooked: CNNs are made on purpose to handle raw pixel values, so you just throw the image in and it is handled properly.
\nWhen using other types of classifiers, it could help add a bit of feature engineering to help the classifier doing its job and achieve high accuracy.
\nBut not this time.
\nI wanted to be as "light" as possible in this demo, so I only took a couple steps during the feature acquisition:
\nI would hardly call this feature engineering.
\nThis is an example of the result of this pipeline.
\n![\"Handwritten](\"https://eloquentarduino.github.io/wp-content/uploads/2020/02/mnist-feature-extraction.jpg\")
The code for this pipeline is really simple and is almost the same from the example on motion detection.
\n#include "esp_camera.h"\n\n#define PWDN_GPIO_NUM -1\n#define RESET_GPIO_NUM 15\n#define XCLK_GPIO_NUM 27\n#define SIOD_GPIO_NUM 22\n#define SIOC_GPIO_NUM 23\n#define Y9_GPIO_NUM 19\n#define Y8_GPIO_NUM 36\n#define Y7_GPIO_NUM 18\n#define Y6_GPIO_NUM 39\n#define Y5_GPIO_NUM 5\n#define Y4_GPIO_NUM 34\n#define Y3_GPIO_NUM 35\n#define Y2_GPIO_NUM 32\n#define VSYNC_GPIO_NUM 25\n#define HREF_GPIO_NUM 26\n#define PCLK_GPIO_NUM 21\n\n#define FRAME_SIZE FRAMESIZE_QQVGA\n#define WIDTH 160\n#define HEIGHT 120\n#define BLOCK_SIZE 5\n#define W (WIDTH / BLOCK_SIZE)\n#define H (HEIGHT / BLOCK_SIZE)\n#define THRESHOLD 127\n\ndouble features[H*W] = { 0 };\n\nvoid setup() {\n Serial.begin(115200);\n Serial.println(setup_camera(FRAME_SIZE) ? "OK" : "ERR INIT");\n delay(3000);\n}\n\nvoid loop() {\n if (!capture_still()) {\n Serial.println("Failed capture");\n delay(2000);\n return;\n }\n\n print_features();\n delay(3000);\n}\n\nbool setup_camera(framesize_t frameSize) {\n camera_config_t config;\n\n config.ledc_channel = LEDC_CHANNEL_0;\n config.ledc_timer = LEDC_TIMER_0;\n config.pin_d0 = Y2_GPIO_NUM;\n config.pin_d1 = Y3_GPIO_NUM;\n config.pin_d2 = Y4_GPIO_NUM;\n config.pin_d3 = Y5_GPIO_NUM;\n config.pin_d4 = Y6_GPIO_NUM;\n config.pin_d5 = Y7_GPIO_NUM;\n config.pin_d6 = Y8_GPIO_NUM;\n config.pin_d7 = Y9_GPIO_NUM;\n config.pin_xclk = XCLK_GPIO_NUM;\n config.pin_pclk = PCLK_GPIO_NUM;\n config.pin_vsync = VSYNC_GPIO_NUM;\n config.pin_href = HREF_GPIO_NUM;\n config.pin_sscb_sda = SIOD_GPIO_NUM;\n config.pin_sscb_scl = SIOC_GPIO_NUM;\n config.pin_pwdn = PWDN_GPIO_NUM;\n config.pin_reset = RESET_GPIO_NUM;\n config.xclk_freq_hz = 20000000;\n config.pixel_format = PIXFORMAT_GRAYSCALE;\n config.frame_size = frameSize;\n config.jpeg_quality = 12;\n config.fb_count = 1;\n\n bool ok = esp_camera_init(&config) == ESP_OK;\n\n sensor_t *sensor = esp_camera_sensor_get();\n sensor->set_framesize(sensor, frameSize);\n\n return ok;\n}\n\nbool capture_still() {\n camera_fb_t *frame = esp_camera_fb_get();\n\n if (!frame)\n return false;\n\n // reset all the features\n for (size_t i = 0; i < H * W; i++)\n features[i] = 0;\n\n // for each pixel, compute the position in the downsampled image\n for (size_t i = 0; i < frame->len; i++) {\n const uint16_t x = i % WIDTH;\n const uint16_t y = floor(i / WIDTH);\n const uint8_t block_x = floor(x / BLOCK_SIZE);\n const uint8_t block_y = floor(y / BLOCK_SIZE);\n const uint16_t j = block_y * W + block_x;\n\n features[j] += frame->buf[i];\n }\n\n // apply threshold\n for (size_t i = 0; i < H * W; i++) {\n features[i] = (features[i] / (BLOCK_SIZE * BLOCK_SIZE) > THRESHOLD) ? 1 : 0;\n }\n\n return true;\n}\n\nvoid print_features() {\n for (size_t i = 0; i < H * W; i++) {\n Serial.print(features[i]);\n\n if (i != H * W - 1)\n Serial.print(',');\n }\n\n Serial.println();\n}To create your own dataset, you need a collection of handwritten digits.
\nYou can do this part as you like, by using pieces of paper or a monitor. I used a tablet because it was well illuminated and I could open a bunch of tabs to keep a record of my samples.
\nAs in the apple vs orange, keep in mind that you should be consistent during both the training phase and the inference phase.
\nThis is why I used tape to fix my ESP32 camera to the desk and kept the tablet in the exact same position.
\nIf you desire, you could experiment varying slightly the capturing setup during the training and see if your classifier still achieves good accuracy: this is a test I didn't make.
\nFor a detailed guide refer to the tutorial
\r\n\r\n\r\n
from sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)At this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.
Okay, at this point you should have all the working pieces to do handwritten digit image classification on your ESP32 camera. Include your model in the sketch and run the classification.
\n#include "model.h"\n\nvoid loop() {\n if (!capture_still()) {\n Serial.println("Failed capture");\n delay(2000);\n\n return;\n }\n\n Serial.print("Number: ");\n Serial.println(classIdxToName(predict(features)));\n delay(3000);\n}Done.
\nYou can see a demo of my results in the video below.
\nMy dataset is composed of 25 training samples in total and the SVM with linear kernel produced 17 support vectors.
\nOn my M5Stick camera board, the overhead for the model is 6.8 Kb of flash and the inference takes 7ms: not that bad!
\nCheck the full project code on Github
\nL'articolo Handwritten digit classification with Arduino and MicroML proviene da Eloquent Arduino Blog.
\n", "content_text": "We continue exploring the endless possibilities on the MicroML (Machine Learning for Microcontrollers) framework on Arduino and ESP32 boards: in this post we're back to image classification. In particular, we'll distinguish handwritten digits using an ESP32 camera.\n\n\nIf this is the first time you're reading my blog, you may have missed that I'm on a journey to push the limits of Machine learning on embedded devices like the Arduino boards and ESP32.\nI started with accelerometer data classification, then did Wifi indoor positioning as a proof of concept.\nIn the last weeks, though, I undertook a more difficult path that is image classification.\nImage classification is where Convolutional Neural Networks really shine, but I'm here to question this settlement and demostrate that it is possible to come up with much lighter alternatives.\nIn this post we continue with the examples, replicating a "benchmark" dataset in Machine learning: the handwritten digits classification.\n\nIf you are curious about a specific image classification task you would like to see implemented, let me know in the comments: I'm always open to new ideas\n\nThe task\nThe objective of this example is to be able to tell what an handwritten digit is, taking as input a photo from the ESP32 camera.\nIn particular, we have 3 handwritten numbers and the task of our model will be to distinguish which image is what number.\n\nI only have a single image per digit, but you're free to draw as many samples as you like: it should help improve the performance of you're classifier.\n1. Feature extraction\nWhen dealing with images, if you use a CNN this step is often overlooked: CNNs are made on purpose to handle raw pixel values, so you just throw the image in and it is handled properly.\nWhen using other types of classifiers, it could help add a bit of feature engineering to help the classifier doing its job and achieve high accuracy.\nBut not this time.\nI wanted to be as "light" as possible in this demo, so I only took a couple steps during the feature acquisition:\n\nuse a grayscale image\ndownsample to a manageable size\nconvert it to black/white with a threshold\n\nI would hardly call this feature engineering.\nThis is an example of the result of this pipeline.\n\nThe code for this pipeline is really simple and is almost the same from the example on motion detection.\n#include "esp_camera.h"\n\n#define PWDN_GPIO_NUM -1\n#define RESET_GPIO_NUM 15\n#define XCLK_GPIO_NUM 27\n#define SIOD_GPIO_NUM 22\n#define SIOC_GPIO_NUM 23\n#define Y9_GPIO_NUM 19\n#define Y8_GPIO_NUM 36\n#define Y7_GPIO_NUM 18\n#define Y6_GPIO_NUM 39\n#define Y5_GPIO_NUM 5\n#define Y4_GPIO_NUM 34\n#define Y3_GPIO_NUM 35\n#define Y2_GPIO_NUM 32\n#define VSYNC_GPIO_NUM 25\n#define HREF_GPIO_NUM 26\n#define PCLK_GPIO_NUM 21\n\n#define FRAME_SIZE FRAMESIZE_QQVGA\n#define WIDTH 160\n#define HEIGHT 120\n#define BLOCK_SIZE 5\n#define W (WIDTH / BLOCK_SIZE)\n#define H (HEIGHT / BLOCK_SIZE)\n#define THRESHOLD 127\n\ndouble features[H*W] = { 0 };\n\nvoid setup() {\n Serial.begin(115200);\n Serial.println(setup_camera(FRAME_SIZE) ? "OK" : "ERR INIT");\n delay(3000);\n}\n\nvoid loop() {\n if (!capture_still()) {\n Serial.println("Failed capture");\n delay(2000);\n return;\n }\n\n print_features();\n delay(3000);\n}\n\nbool setup_camera(framesize_t frameSize) {\n camera_config_t config;\n\n config.ledc_channel = LEDC_CHANNEL_0;\n config.ledc_timer = LEDC_TIMER_0;\n config.pin_d0 = Y2_GPIO_NUM;\n config.pin_d1 = Y3_GPIO_NUM;\n config.pin_d2 = Y4_GPIO_NUM;\n config.pin_d3 = Y5_GPIO_NUM;\n config.pin_d4 = Y6_GPIO_NUM;\n config.pin_d5 = Y7_GPIO_NUM;\n config.pin_d6 = Y8_GPIO_NUM;\n config.pin_d7 = Y9_GPIO_NUM;\n config.pin_xclk = XCLK_GPIO_NUM;\n config.pin_pclk = PCLK_GPIO_NUM;\n config.pin_vsync = VSYNC_GPIO_NUM;\n config.pin_href = HREF_GPIO_NUM;\n config.pin_sscb_sda = SIOD_GPIO_NUM;\n config.pin_sscb_scl = SIOC_GPIO_NUM;\n config.pin_pwdn = PWDN_GPIO_NUM;\n config.pin_reset = RESET_GPIO_NUM;\n config.xclk_freq_hz = 20000000;\n config.pixel_format = PIXFORMAT_GRAYSCALE;\n config.frame_size = frameSize;\n config.jpeg_quality = 12;\n config.fb_count = 1;\n\n bool ok = esp_camera_init(&config) == ESP_OK;\n\n sensor_t *sensor = esp_camera_sensor_get();\n sensor->set_framesize(sensor, frameSize);\n\n return ok;\n}\n\nbool capture_still() {\n camera_fb_t *frame = esp_camera_fb_get();\n\n if (!frame)\n return false;\n\n // reset all the features\n for (size_t i = 0; i < H * W; i++)\n features[i] = 0;\n\n // for each pixel, compute the position in the downsampled image\n for (size_t i = 0; i < frame->len; i++) {\n const uint16_t x = i % WIDTH;\n const uint16_t y = floor(i / WIDTH);\n const uint8_t block_x = floor(x / BLOCK_SIZE);\n const uint8_t block_y = floor(y / BLOCK_SIZE);\n const uint16_t j = block_y * W + block_x;\n\n features[j] += frame->buf[i];\n }\n\n // apply threshold\n for (size_t i = 0; i < H * W; i++) {\n features[i] = (features[i] / (BLOCK_SIZE * BLOCK_SIZE) > THRESHOLD) ? 1 : 0;\n }\n\n return true;\n}\n\nvoid print_features() {\n for (size_t i = 0; i < H * W; i++) {\n Serial.print(features[i]);\n\n if (i != H * W - 1)\n Serial.print(',');\n }\n\n Serial.println();\n}\n2. Samples recording\nTo create your own dataset, you need a collection of handwritten digits.\nYou can do this part as you like, by using pieces of paper or a monitor. I used a tablet because it was well illuminated and I could open a bunch of tabs to keep a record of my samples.\nAs in the apple vs orange, keep in mind that you should be consistent during both the training phase and the inference phase.\nThis is why I used tape to fix my ESP32 camera to the desk and kept the tablet in the exact same position.\nIf you desire, you could experiment varying slightly the capturing setup during the training and see if your classifier still achieves good accuracy: this is a test I didn't make.\n3. Train and export the classifier\r\n\r\nFor a detailed guide refer to the tutorial\r\n\r\n\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)\r\n\r\nAt this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.\n4. The result\nOkay, at this point you should have all the working pieces to do handwritten digit image classification on your ESP32 camera. Include your model in the sketch and run the classification.\n#include "model.h"\n\nvoid loop() {\n if (!capture_still()) {\n Serial.println("Failed capture");\n delay(2000);\n\n return;\n }\n\n Serial.print("Number: ");\n Serial.println(classIdxToName(predict(features)));\n delay(3000);\n}\nDone.\nYou can see a demo of my results in the video below.\n\nhttps://eloquentarduino.github.io/wp-content/uploads/2020/02/MNIST-mute.mp4\nProject figures\nMy dataset is composed of 25 training samples in total and the SVM with linear kernel produced 17 support vectors.\nOn my M5Stick camera board, the overhead for the model is 6.8 Kb of flash and the inference takes 7ms: not that bad!\n\r\nCheck the full project code on Github\nL'articolo Handwritten digit classification with Arduino and MicroML proviene da Eloquent Arduino Blog.", "date_published": "2020-02-23T11:53:03+01:00", "date_modified": "2020-05-31T18:50:44+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "camera", "esp32", "microml", "svm", "Arduino Machine learning", "Computer vision" ], "attachments": [ { "url": "https://eloquentarduino.github.io/wp-content/uploads/2020/02/MNIST-mute.mp4", "mime_type": "video/mp4", "size_in_bytes": 6424809 } ] }, { "id": "https://eloquentarduino.github.io/?p=893", "url": "https://eloquentarduino.github.io/2020/02/even-smaller-machine-learning-models-for-your-mcu/", "title": "Even smaller Machine learning models for your MCU: up to -82% code size", "content_html": "So far we've used SVM (Support Vector Machine) as our main classifier to port a Machine learning model to a microcontroller: but recently I found an interesting alternative which could be waaaay smaller, mantaining a similar accuracy.
\n![\"RVM](\"https://eloquentarduino.github.io/wp-content/uploads/2020/02/rvm.jpg\")
\n
I chose SVM as my main focus of intereset for the MicroML framework because I knew the support vector encoding could be very memory efficient once ported to plain C. And it really is.
\nI was able to port many real-world models (gesture identification, wake word detection) to tiny microcontrollers like the old Arduino Nano (32 kb flash, 2 kb RAM).
\nThe tradeoff of my implementation was to sacrifice the flash space (which is usually quite big) to save as much RAM as possible, which is usually the most limiting factor.
\nDue to this implementation, if your model grows in size (highly dimensional data or not well separable data), the generated code will still fit in the RAM, but "overflow" the available flash.
\nIn a couple of my previous post I warned that model selection might be a required step before being able to deploy a model to a MCU, since you should first check if it fits. If not, you must train another model hoping to get fewer support vectors, since each of them contributes to the code size increase.
\nIt was by chance that I came across a new algorithm that I never heard of, called Relevance Vector Machine. It was patented by Microsoft until last year (so maybe this is the reason you don't see it in the wild), but now it is free of use as far as I can tell.
\nHere is the link to the paper if you want to read it, it gives some insights into the development process.
\nI'm not a mathematician, so I can't describe it accurately, but in a few words it uses the same formulation of SVM (a weightened sum of kernels), applying a Bayesan model.
\nThis serves in the first place to be able to get the probabilities of the classification results, which is something totally missing in SVM.
\nIn the second place, the algorithm tries to learn a much more sparse representation of the support vectors, as you can see in the following picture.
\n
When I first read the paper my first tought was just "wow"! This is exactly what I need for my MicroML framework: a ultra-lightweight model which can still achieve high accuracy.
\nNow that I knew this algorithm, I searched for it in the sklearn documentation: it was not there.
It seems that, since it was patented, they didn't have an implementation.
\nFortunately, there is an implementation which follows the sklearn paradigm. You have to install it:
\npip install Cython\npip install https://github.com/AmazaspShumik/sklearn_bayes/archive/master.zipSince the interface is the usual fit predict, it is super easy to train a classifier.
from sklearn.datasets import load_iris\nfrom skbayes.rvm_ard_models import RVC\nimport warnings\n\n# I get tons of boring warnings during training, so turn it off\nwarnings.filterwarnings("ignore")\n\niris = load_iris()\nX = iris.data\ny = iris.target\nclf = RVC(kernel='rbf', gamma=0.001)\nclf.fit(X, y)\ny_predict = clf.predict(X)The parameters for the constructor are similar to those of the SVC classifier from sklearn:
kernel: one of linear, poly, rbfdegree: if kernel=polygamma: if kernel=poly or kernel=rbfYou can read the docs from sklearn to learn more.
\nNow that we have a trained classifier, we have to port it to plain C that compiles on our microcontroller of choice.
\nI patched my package micromlgen to do the job for you, so you should install the latest version to get it working.
pip install --upgrade micromlgenNow the export part is almost the same as with an SVM classifier.
\n from micromlgen import port_rvm\n\n clf = get_rvm_classifier()\n c_code = port_rvm(clf)\n print(c_code)And you're done: you have plain C code you can embed in any microcontroller.
\nTo test the effectiveness of this new algorithm, I applied it to the datasets I built in my previous posts, comparing side by side the size and accuracy of both SVM and RVM.
\nThe results are summarized in the next table.
\n\n| Dataset | \nSVM | \nRVM | \nDelta | \n|||
|---|---|---|---|---|---|---|
| \n | Flash(byte) | \nAcc. (%) | \nFlash(byte) | \nAcc. (%) | \nFlash | \nAcc. | \n
| RGB colors | \n4584 | \n100 | \n3580 | \n100 | \n-22% | \n-0% | \n
| Accelerometer gestures(linear kernel) | \n36888 | \n92 | \n7056 | \n85 | \n-80% | \n-7% | \n
| Accelerometer gestures(gaussian kernel) | \n45348 | \n95 | \n7766 | \n95 | \n-82% | \n-0% | \n
| Wifi positioning | \n4641 | \n100 | \n3534 | \n100 | \n-24% | \n-0% | \n
| Wake word(linear kernel) | \n18098 | \n86 | \n3602 | \n53 | \n-80% | \n-33% | \n
| Wake word(gaussian kernel) | \n21788 | \n90 | \n4826 | \n62 | \n-78% | \n-28% | \n
** the accuracy reported are with default parameters, without any tuning, averaged in 30 runs
\nAs you may see, the results are quite surpising:
\nAs in any situation, you should test which one of the two algorithms works best for your use case, but there a couple of guidelines you may follow:
\nAs a reference, here is the codes generated for an SVM classifier and an RVM one to classify the IRIS dataset.
\nuint8_t predict_rvm(double *x) {\n double decision[3] = { 0 };\n decision[0] = -0.6190847299428206;\n decision[1] = (compute_kernel(x, 6.3, 3.3, 6.0, 2.5) - 72.33233 ) * 0.228214 + -2.3609625;\n decision[2] = (compute_kernel(x, 7.7, 2.8, 6.7, 2.0) - 81.0089166 ) * -0.29006 + -3.360963;\n uint8_t idx = 0;\n double val = decision[0];\n for (uint8_t i = 1; i < 3; i++) {\n if (decision[i] > val) {\n idx = i;\n val = decision[i];\n }\n }\n return idx;\n}\n\nint predict_svm(double *x) {\n double kernels[10] = { 0 };\n double decisions[3] = { 0 };\n int votes[3] = { 0 };\n kernels[0] = compute_kernel(x, 6.7 , 3.0 , 5.0 , 1.7 );\n kernels[1] = compute_kernel(x, 6.0 , 2.7 , 5.1 , 1.6 );\n kernels[2] = compute_kernel(x, 5.1 , 2.5 , 3.0 , 1.1 );\n kernels[3] = compute_kernel(x, 6.0 , 3.0 , 4.8 , 1.8 );\n kernels[4] = compute_kernel(x, 7.2 , 3.0 , 5.8 , 1.6 );\n kernels[5] = compute_kernel(x, 4.9 , 2.5 , 4.5 , 1.7 );\n kernels[6] = compute_kernel(x, 6.2 , 2.8 , 4.8 , 1.8 );\n kernels[7] = compute_kernel(x, 6.0 , 2.2 , 5.0 , 1.5 );\n kernels[8] = compute_kernel(x, 4.8 , 3.4 , 1.9 , 0.2 );\n kernels[9] = compute_kernel(x, 5.1 , 3.3 , 1.7 , 0.5 );\n decisions[0] = 20.276395502\n + kernels[0] * 100.0\n + kernels[1] * 100.0\n + kernels[3] * -79.351629954\n + kernels[4] * -49.298850195\n + kernels[6] * -40.585178082\n + kernels[7] * -30.764341769\n ;\n decisions[1] = -0.903345464\n + kernels[2] * 0.743494115\n + kernels[9] * -0.743494115\n ;\n decisions[2] = -1.507856504\n + kernels[5] * 0.203695177\n + kernels[8] * -0.160020702\n + kernels[9] * -0.043674475\n ;\n votes[decisions[0] > 0 ? 0 : 1] += 1;\n votes[decisions[1] > 0 ? 0 : 2] += 1;\n votes[decisions[2] > 0 ? 1 : 2] += 1;\n int classVal = -1;\n int classIdx = -1;\n for (int i = 0; i < 3; i++) {\n if (votes[i] > classVal) {\n classVal = votes[i];\n classIdx = i;\n }\n }\n return classIdx;\n}As you can see, RVM actually only computes 2 kernels and does 2 multiplications. SVM, on the other hand, computes 10 kernels and does 13 multiplications.
\nThis is a recurring pattern, so RVM is much much faster in the inference process.
\nmicromlgen and in particular port_rvm are work in progress: you may experience some glitches or it may not work in your specific case. Please report any issue on the Github repo.
L'articolo Even smaller Machine learning models for your MCU: up to -82% code size proviene da Eloquent Arduino Blog.
\n", "content_text": "So far we've used SVM (Support Vector Machine) as our main classifier to port a Machine learning model to a microcontroller: but recently I found an interesting alternative which could be waaaay smaller, mantaining a similar accuracy.\n\n\nTable of contentsThe current stateA new algorithm: Relevance Vector MachinesTraining a classifierPorting to CPerformace comparisonSize comparisonDisclaimer\nThe current state\nI chose SVM as my main focus of intereset for the MicroML framework because I knew the support vector encoding could be very memory efficient once ported to plain C. And it really is.\nI was able to port many real-world models (gesture identification, wake word detection) to tiny microcontrollers like the old Arduino Nano (32 kb flash, 2 kb RAM).\nThe tradeoff of my implementation was to sacrifice the flash space (which is usually quite big) to save as much RAM as possible, which is usually the most limiting factor.\nDue to this implementation, if your model grows in size (highly dimensional data or not well separable data), the generated code will still fit in the RAM, but "overflow" the available flash.\nIn a couple of my previous post I warned that model selection might be a required step before being able to deploy a model to a MCU, since you should first check if it fits. If not, you must train another model hoping to get fewer support vectors, since each of them contributes to the code size increase.\nA new algorithm: Relevance Vector Machines\nIt was by chance that I came across a new algorithm that I never heard of, called Relevance Vector Machine. It was patented by Microsoft until last year (so maybe this is the reason you don't see it in the wild), but now it is free of use as far as I can tell.\nHere is the link to the paper if you want to read it, it gives some insights into the development process.\nI'm not a mathematician, so I can't describe it accurately, but in a few words it uses the same formulation of SVM (a weightened sum of kernels), applying a Bayesan model.\nThis serves in the first place to be able to get the probabilities of the classification results, which is something totally missing in SVM.\nIn the second place, the algorithm tries to learn a much more sparse representation of the support vectors, as you can see in the following picture.\n\nWhen I first read the paper my first tought was just "wow"! This is exactly what I need for my MicroML framework: a ultra-lightweight model which can still achieve high accuracy.\nTraining a classifier\nNow that I knew this algorithm, I searched for it in the sklearn documentation: it was not there.\nIt seems that, since it was patented, they didn't have an implementation.\nFortunately, there is an implementation which follows the sklearn paradigm. You have to install it:\npip install Cython\npip install https://github.com/AmazaspShumik/sklearn_bayes/archive/master.zip\nSince the interface is the usual fit predict, it is super easy to train a classifier.\nfrom sklearn.datasets import load_iris\nfrom skbayes.rvm_ard_models import RVC\nimport warnings\n\n# I get tons of boring warnings during training, so turn it off\nwarnings.filterwarnings("ignore")\n\niris = load_iris()\nX = iris.data\ny = iris.target\nclf = RVC(kernel='rbf', gamma=0.001)\nclf.fit(X, y)\ny_predict = clf.predict(X)\nThe parameters for the constructor are similar to those of the SVC classifier from sklearn:\n\nkernel: one of linear, poly, rbf\ndegree: if kernel=poly\ngamma: if kernel=poly or kernel=rbf\n\nYou can read the docs from sklearn to learn more.\nPorting to C\nNow that we have a trained classifier, we have to port it to plain C that compiles on our microcontroller of choice.\nI patched my package micromlgen to do the job for you, so you should install the latest version to get it working.\n pip install --upgrade micromlgen\nNow the export part is almost the same as with an SVM classifier.\n from micromlgen import port_rvm\n\n clf = get_rvm_classifier()\n c_code = port_rvm(clf)\n print(c_code)\nAnd you're done: you have plain C code you can embed in any microcontroller.\nPerformace comparison\nTo test the effectiveness of this new algorithm, I applied it to the datasets I built in my previous posts, comparing side by side the size and accuracy of both SVM and RVM.\nThe results are summarized in the next table.\n\n\n\n\nDataset\nSVM\nRVM\nDelta\n\n\n\nFlash(byte)\nAcc. (%)\nFlash(byte)\nAcc. (%)\nFlash\nAcc.\n\n\n\n\nRGB colors\n4584\n100\n3580\n100\n-22%\n-0%\n\n\nAccelerometer gestures(linear kernel)\n36888\n92\n7056\n85\n-80%\n-7%\n\n\nAccelerometer gestures(gaussian kernel)\n45348\n95\n7766\n95\n-82%\n-0%\n\n\nWifi positioning\n4641\n100\n3534\n100\n-24%\n-0%\n\n\nWake word(linear kernel)\n18098\n86\n3602\n53\n-80%\n-33%\n\n\nWake word(gaussian kernel)\n21788\n90\n4826\n62\n-78%\n-28%\n\n\n\n** the accuracy reported are with default parameters, without any tuning, averaged in 30 runs\nAs you may see, the results are quite surpising:\n\nyou can achieve up to 82% space reduction on highly dimensional dataset without any loss in accuracy (accelerometer gestures with gaussian kernel)\nsometimes you may not be able to achieve a decent accuracy (62% at most on the wake word dataset)\n\nAs in any situation, you should test which one of the two algorithms works best for your use case, but there a couple of guidelines you may follow:\n\nif you need top accuracy, probably SVM can achieve slighter better performance if you have enough space\nif you need tiny space or top speed, test if RVM achieves a satisfiable accuracy\nif both SVM and RVM achieve comparable performace, go with RVM: it's much lighter than SVM in most cases and will run faster\n\nSize comparison\nAs a reference, here is the codes generated for an SVM classifier and an RVM one to classify the IRIS dataset.\nuint8_t predict_rvm(double *x) {\n double decision[3] = { 0 };\n decision[0] = -0.6190847299428206;\n decision[1] = (compute_kernel(x, 6.3, 3.3, 6.0, 2.5) - 72.33233 ) * 0.228214 + -2.3609625;\n decision[2] = (compute_kernel(x, 7.7, 2.8, 6.7, 2.0) - 81.0089166 ) * -0.29006 + -3.360963;\n uint8_t idx = 0;\n double val = decision[0];\n for (uint8_t i = 1; i < 3; i++) {\n if (decision[i] > val) {\n idx = i;\n val = decision[i];\n }\n }\n return idx;\n}\n\nint predict_svm(double *x) {\n double kernels[10] = { 0 };\n double decisions[3] = { 0 };\n int votes[3] = { 0 };\n kernels[0] = compute_kernel(x, 6.7 , 3.0 , 5.0 , 1.7 );\n kernels[1] = compute_kernel(x, 6.0 , 2.7 , 5.1 , 1.6 );\n kernels[2] = compute_kernel(x, 5.1 , 2.5 , 3.0 , 1.1 );\n kernels[3] = compute_kernel(x, 6.0 , 3.0 , 4.8 , 1.8 );\n kernels[4] = compute_kernel(x, 7.2 , 3.0 , 5.8 , 1.6 );\n kernels[5] = compute_kernel(x, 4.9 , 2.5 , 4.5 , 1.7 );\n kernels[6] = compute_kernel(x, 6.2 , 2.8 , 4.8 , 1.8 );\n kernels[7] = compute_kernel(x, 6.0 , 2.2 , 5.0 , 1.5 );\n kernels[8] = compute_kernel(x, 4.8 , 3.4 , 1.9 , 0.2 );\n kernels[9] = compute_kernel(x, 5.1 , 3.3 , 1.7 , 0.5 );\n decisions[0] = 20.276395502\n + kernels[0] * 100.0\n + kernels[1] * 100.0\n + kernels[3] * -79.351629954\n + kernels[4] * -49.298850195\n + kernels[6] * -40.585178082\n + kernels[7] * -30.764341769\n ;\n decisions[1] = -0.903345464\n + kernels[2] * 0.743494115\n + kernels[9] * -0.743494115\n ;\n decisions[2] = -1.507856504\n + kernels[5] * 0.203695177\n + kernels[8] * -0.160020702\n + kernels[9] * -0.043674475\n ;\n votes[decisions[0] > 0 ? 0 : 1] += 1;\n votes[decisions[1] > 0 ? 0 : 2] += 1;\n votes[decisions[2] > 0 ? 1 : 2] += 1;\n int classVal = -1;\n int classIdx = -1;\n for (int i = 0; i < 3; i++) {\n if (votes[i] > classVal) {\n classVal = votes[i];\n classIdx = i;\n }\n }\n return classIdx;\n}\nAs you can see, RVM actually only computes 2 kernels and does 2 multiplications. SVM, on the other hand, computes 10 kernels and does 13 multiplications.\nThis is a recurring pattern, so RVM is much much faster in the inference process.\nDisclaimer\nmicromlgen and in particular port_rvm are work in progress: you may experience some glitches or it may not work in your specific case. Please report any issue on the Github repo.\nL'articolo Even smaller Machine learning models for your MCU: up to -82% code size proviene da Eloquent Arduino Blog.", "date_published": "2020-02-15T17:37:32+01:00", "date_modified": "2020-05-31T18:50:57+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "rvm", "svm", "Arduino Machine learning" ] }, { "id": "https://eloquentarduino.github.io/?p=820", "url": "https://eloquentarduino.github.io/2020/01/image-recognition-with-esp32-and-arduino/", "title": "Apple or Orange? Image recognition with ESP32 and Arduino", "content_html": "Do you have an ESP32 camera?
\nWant to do image recognition directly on your ESP32, without a PC?
\nIn this post we'll look into a very basic image recognition task: distinguish apples from oranges with machine learning.
\n![\"Apple](\"/wp-content/uploads/2020/01/Apple-vs-Orange.gif\")
\n
Image recognition is a very hot topic these days in the AI/ML landscape. Convolutional Neural Networks really shines in this task and can achieve almost perfect accuracy on many scenarios.
\nSadly, you can't run CNN on your ESP32, they're just too large for a microcontroller.
\nSince in this series about Machine Learning on Microcontrollers we're exploring the potential of Support Vector Machines (SVMs) at solving different classification tasks, we'll take a look into image classification too.
\nIn a previous post about color identification with Machine learning, we used an Arduino to detect the object we were pointing at with a color sensor (TCS3200) by its color: if we detected yellow, for example, we knew we had a banana in front of us.
\nOf course such a process is not object recognition at all: yellow may be a banane, or a lemon, or an apple.
\nObject inference, in that case, works only if you have exactly one object for a given color.
\nThe objective of this post, instead, is to investigate if we can use the MicroML framework to do simple image recognition on the images from an ESP32 camera.
\nThis is much more similar to the tasks you do on your PC with CNN or any other form of NN you are comfortable with. Sure, we will still apply some restrictions to fit the problem on a microcontroller, but this is a huge step forward compared to the simple color identification.
\nAs any beginning machine learning project about image classification worth of respect, our task will be to distinguish an orange from an apple.
\nI have to admit that I rarely use NN, so I may be wrong here, but from the examples I read online it looks to me that features engineering is not a fundamental task with NN.
\nThose few times I used CNN, I always used the whole image as input, as-is. I didn't extracted any feature from them (e.g. color histogram): the CNN worked perfectly fine with raw images.
\nI don't think this will work best with SVM, but in this first post we're starting as simple as possible, so we'll be using the RGB components of the image as our features. In a future post, we'll introduce additional features to try to improve our results.
\nI said we're using the RGB components of the image. But not all of them.
\nEven at the lowest resolution of 160x120 pixels, a raw RGB image from the camera would generate 160x120x3 = 57600 features: way too much.
\nWe need to reduce this number to the bare minimum.
\nHow much pixels do you think are necessary to get reasonable results in this task of classifying apples from oranges?
\nYou would be surprised to know that I got 90% accuracy with an RGB image of 8x6!
\n![\"You](\"https://eloquentarduino.github.io/wp-content/uploads/2020/01/Orange-and-Apple-pixelated.jpg\"){kind=tracking}
Yes, that's all we really need to do a good enough classification.
\nYou can distinguish apples from oranges on ESP32 with 8x6 pixels only!
Click To Tweet
Of course this is a tradeoff: you can't expect to achieve 99% accuracy while mantaining the model size small enough to fit on a microcontroller. 90% is an acceptable accuracy for me in this context.
\nYou have to keep in mind, moreover, that the features vector size grows quadratically with the image size (if you keep the aspect ratio). A raw RGB image of 8x6 generates 144 features: an image of 16x12 generates 576 features. This was already causing random crashes on my ESP32.
\nSo we'll stick to 8x6 images.
\nNow, how do you compact a 160x120 image to 8x6? With downsampling.
\nThis is the same tecnique we've used in the post about motion detection on ESP32: we define a block size and average all the pixels inside the block to get a single value (you can refer to that post for more details).
\n![\"Image](\"/wp-content/uploads/2020/01/Image-downsampling-example.jpg\")
This time, though, we're working with RGB images instead of grayscale, so we'll repeat the exact same process 3 times, one for each channel.
\nThis is the code excerpt that does the downsampling.
\nuint16_t rgb_frame[HEIGHT / BLOCK_SIZE][WIDTH / BLOCK_SIZE][3] = { 0 };\n\nvoid grab_image() {\n for (size_t i = 0; i < len; i += 2) {\n // get r, g, b from the buffer\n // see later\n\n const size_t j = i / 2;\n // transform x, y in the original image to x, y in the downsampled image\n // by dividing by BLOCK_SIZE\n const uint16_t x = j % WIDTH;\n const uint16_t y = floor(j / WIDTH);\n const uint8_t block_x = floor(x / BLOCK_SIZE);\n const uint8_t block_y = floor(y / BLOCK_SIZE);\n\n // average pixels in block (accumulate)\n rgb_frame[block_y][block_x][0] += r;\n rgb_frame[block_y][block_x][1] += g;\n rgb_frame[block_y][block_x][2] += b;\n }\n}The ESP32 camera can store the image in different formats (of our interest \u2014 there are a couple more available):
\nFor our purpose, we'll use the RGB565 format and extract the 3 components from the 2 bytes with the following code.
\n![\"taken](\"https://s2.www.theimagingsource.com/application-1.1.29/documentation/ic_imaging_control_class/en_US/images/rgb565.gif\")
config.pixel_format = PIXFORMAT_RGB565;\n\nfor (size_t i = 0; i < len; i += 2) {\n const uint8_t high = buf[i];\n const uint8_t low = buf[i+1];\n const uint16_t pixel = (high << 8) | low;\n\n const uint8_t r = (pixel & 0b1111100000000000) >> 11;\n const uint8_t g = (pixel & 0b0000011111100000) >> 6;\n const uint8_t b = (pixel & 0b0000000000011111);\n}Now that we can grab the images from the camera, we'll need to take a few samples of each object we want to racognize.
\nBefore doing so, we'll linearize the image matrix to a 1-dimensional vector, because that's what our prediction function expects.
\n#define H (HEIGHT / BLOCK_SIZE)\n#define W (WIDTH / BLOCK_SIZE)\n\nvoid linearize_features() {\n size_t i = 0;\n double features[H*W*3] = {0};\n\n for (int y = 0; y < H; y++) {\n for (int x = 0; x < W; x++) {\n features[i++] = rgb_frame[y][x][0];\n features[i++] = rgb_frame[y][x][1];\n features[i++] = rgb_frame[y][x][2];\n }\n }\n\n // print to serial\n for (size_t i = 0; i < H*W*3; i++) {\n Serial.print(features[i]);\n Serial.print('\\t');\n }\n\n Serial.println();\n}Now you can setup your acquisition environment and take the samples: 15-20 of each object will do the job.
\nTo train the classifier, save the features for each object in a file, one features vector per line. Then follow the steps on how to train a ML classifier for Arduino to get the exported model.
\nYou can experiment with different classifier configurations.
\nMy features were well distinguishable, so I had great results (100% accuracy) with any kernel (even linear).
\nOne odd thing happened with the RBF kernel: I had to use an extremely low gamma value (0.0000001). Does anyone can explain me why? I usually go with a default value of 0.001.
\nThe model produced 13 support vectors.
\nI did no features scaling: you could try it if classifying more than 2 classes and having poor results.
\n![\"Apple](\"https://eloquentarduino.github.io/wp-content/uploads/2020/01/Apple-vs-Orange-decision-boundaries.png\")
If you followed all the steps above, you should now have a model capable of detecting if your camera is shotting an apple or an orange, as you can see in the following video.
\n\n\nThe little white object you see at the bottom of the image is the camera, taped to the desk.
\nDid you think it was possible to do simple image classification on your ESP32?
\nThis is not full-fledged object recognition: it can't label objects while you walk as Tensorflow can do, for example.
\nYou have to carefully craft your setup and be as consistent as possible between training and inferencing.
\nStill, I think this is a fun proof-of-concept that can have useful applications in simple scenarios where you can live with a fixed camera and don't want to use a full Raspberry Pi.
\nIn the next weeks I settled to finally try TensorFlow Lite for Microcontrollers on my ESP32, so I'll try to do a comparison between them and this example and report my results.
\nNow that you can do image classification on your ESP32, can you think of a use case you will be able to apply this code to?
\nLet me know in the comments, we could even try realize it together if you need some help.
\nCheck the full project code on Github
\nL'articolo Apple or Orange? Image recognition with ESP32 and Arduino proviene da Eloquent Arduino Blog.
\n", "content_text": "Do you have an ESP32 camera? \nWant to do image recognition directly on your ESP32, without a PC?\nIn this post we'll look into a very basic image recognition task: distinguish apples from oranges with machine learning.\n\n\nImage recognition is a very hot topic these days in the AI/ML landscape. Convolutional Neural Networks really shines in this task and can achieve almost perfect accuracy on many scenarios.\nSadly, you can't run CNN on your ESP32, they're just too large for a microcontroller.\nSince in this series about Machine Learning on Microcontrollers we're exploring the potential of Support Vector Machines (SVMs) at solving different classification tasks, we'll take a look into image classification too.\nTable of contentsWhat we're going to doFeatures definitionExtracting RGB componentsRecord samples imageTraining the classifierReal world exampleDisclaimer\nWhat we're going to do\nIn a previous post about color identification with Machine learning, we used an Arduino to detect the object we were pointing at with a color sensor (TCS3200) by its color: if we detected yellow, for example, we knew we had a banana in front of us.\nOf course such a process is not object recognition at all: yellow may be a banane, or a lemon, or an apple.\nObject inference, in that case, works only if you have exactly one object for a given color.\nThe objective of this post, instead, is to investigate if we can use the MicroML framework to do simple image recognition on the images from an ESP32 camera.\nThis is much more similar to the tasks you do on your PC with CNN or any other form of NN you are comfortable with. Sure, we will still apply some restrictions to fit the problem on a microcontroller, but this is a huge step forward compared to the simple color identification.\n\nIn this context, image recognition means deciding which class (from the trained ones) the current image belongs to. This algorithm can't locate interesting objects in the image, neither detect if an object is present in the frame. It will classify the current image based on the samples recorded during training.\n\nAs any beginning machine learning project about image classification worth of respect, our task will be to distinguish an orange from an apple.\nFeatures definition\nI have to admit that I rarely use NN, so I may be wrong here, but from the examples I read online it looks to me that features engineering is not a fundamental task with NN.\nThose few times I used CNN, I always used the whole image as input, as-is. I didn't extracted any feature from them (e.g. color histogram): the CNN worked perfectly fine with raw images.\nI don't think this will work best with SVM, but in this first post we're starting as simple as possible, so we'll be using the RGB components of the image as our features. In a future post, we'll introduce additional features to try to improve our results.\nI said we're using the RGB components of the image. But not all of them.\nEven at the lowest resolution of 160x120 pixels, a raw RGB image from the camera would generate 160x120x3 = 57600 features: way too much.\nWe need to reduce this number to the bare minimum.\nHow much pixels do you think are necessary to get reasonable results in this task of classifying apples from oranges?\nYou would be surprised to know that I got 90% accuracy with an RGB image of 8x6!\n\nYes, that's all we really need to do a good enough classification.\nYou can distinguish apples from oranges on ESP32 with 8x6 pixels only!Click To Tweet\nOf course this is a tradeoff: you can't expect to achieve 99% accuracy while mantaining the model size small enough to fit on a microcontroller. 90% is an acceptable accuracy for me in this context.\nYou have to keep in mind, moreover, that the features vector size grows quadratically with the image size (if you keep the aspect ratio). A raw RGB image of 8x6 generates 144 features: an image of 16x12 generates 576 features. This was already causing random crashes on my ESP32.\nSo we'll stick to 8x6 images.\nNow, how do you compact a 160x120 image to 8x6? With downsampling.\nThis is the same tecnique we've used in the post about motion detection on ESP32: we define a block size and average all the pixels inside the block to get a single value (you can refer to that post for more details).\n\nThis time, though, we're working with RGB images instead of grayscale, so we'll repeat the exact same process 3 times, one for each channel.\nThis is the code excerpt that does the downsampling.\nuint16_t rgb_frame[HEIGHT / BLOCK_SIZE][WIDTH / BLOCK_SIZE][3] = { 0 };\n\nvoid grab_image() {\n for (size_t i = 0; i < len; i += 2) {\n // get r, g, b from the buffer\n // see later\n\n const size_t j = i / 2;\n // transform x, y in the original image to x, y in the downsampled image\n // by dividing by BLOCK_SIZE\n const uint16_t x = j % WIDTH;\n const uint16_t y = floor(j / WIDTH);\n const uint8_t block_x = floor(x / BLOCK_SIZE);\n const uint8_t block_y = floor(y / BLOCK_SIZE);\n\n // average pixels in block (accumulate)\n rgb_frame[block_y][block_x][0] += r;\n rgb_frame[block_y][block_x][1] += g;\n rgb_frame[block_y][block_x][2] += b;\n }\n}\n\r\n\r\n\r\n \r\n\tFinding this content useful?\r\n\r\n\t\r\n\r\n\t\r\n\t\t\r\n\t\t\r\n\t \r\n \r\n \r\n \r\n\r\n\r\n\r\n\nExtracting RGB components\nThe ESP32 camera can store the image in different formats (of our interest \u2014 there are a couple more available):\n\ngrayscale: no color information, just the intensity is stored. The buffer has size HEIGHT*WIDTH\nRGB565: stores each RGB pixel in two bytes, with 5 bit for red, 6 for green and 5 for blue. The buffer has size HEIGHT * WIDTH * 2\nJPEG: encodes (in hardware?) the image to jpeg. The buffer has a variable length, based on the encoding results\n\nFor our purpose, we'll use the RGB565 format and extract the 3 components from the 2 bytes with the following code.\n\nconfig.pixel_format = PIXFORMAT_RGB565;\n\nfor (size_t i = 0; i < len; i += 2) {\n const uint8_t high = buf[i];\n const uint8_t low = buf[i+1];\n const uint16_t pixel = (high << 8) | low;\n\n const uint8_t r = (pixel & 0b1111100000000000) >> 11;\n const uint8_t g = (pixel & 0b0000011111100000) >> 6;\n const uint8_t b = (pixel & 0b0000000000011111);\n}\nRecord samples image\nNow that we can grab the images from the camera, we'll need to take a few samples of each object we want to racognize.\nBefore doing so, we'll linearize the image matrix to a 1-dimensional vector, because that's what our prediction function expects.\n#define H (HEIGHT / BLOCK_SIZE)\n#define W (WIDTH / BLOCK_SIZE)\n\nvoid linearize_features() {\n size_t i = 0;\n double features[H*W*3] = {0};\n\n for (int y = 0; y < H; y++) {\n for (int x = 0; x < W; x++) {\n features[i++] = rgb_frame[y][x][0];\n features[i++] = rgb_frame[y][x][1];\n features[i++] = rgb_frame[y][x][2];\n }\n }\n\n // print to serial\n for (size_t i = 0; i < H*W*3; i++) {\n Serial.print(features[i]);\n Serial.print('\\t');\n }\n\n Serial.println();\n}\nNow you can setup your acquisition environment and take the samples: 15-20 of each object will do the job.\n\nImage acquisition is a very noisy process: even keeping the camera still, you will get fluctuating values. You need to be very accurate during this phase if you want to achieve good results. I suggest you immobilize your camera with tape to a flat surface or use some kind of photographic easel.\n\nTraining the classifier\nTo train the classifier, save the features for each object in a file, one features vector per line. Then follow the steps on how to train a ML classifier for Arduino to get the exported model.\nYou can experiment with different classifier configurations. \nMy features were well distinguishable, so I had great results (100% accuracy) with any kernel (even linear).\nOne odd thing happened with the RBF kernel: I had to use an extremely low gamma value (0.0000001). Does anyone can explain me why? I usually go with a default value of 0.001.\nThe model produced 13 support vectors.\nI did no features scaling: you could try it if classifying more than 2 classes and having poor results.\n\nReal world example\nIf you followed all the steps above, you should now have a model capable of detecting if your camera is shotting an apple or an orange, as you can see in the following video.\nhttps://eloquentarduino.github.io/wp-content/uploads/2020/01/Apple-vs-Orange.mp4\n\nThe little white object you see at the bottom of the image is the camera, taped to the desk.\nDid you think it was possible to do simple image classification on your ESP32?\nDisclaimer\nThis is not full-fledged object recognition: it can't label objects while you walk as Tensorflow can do, for example.\nYou have to carefully craft your setup and be as consistent as possible between training and inferencing.\nStill, I think this is a fun proof-of-concept that can have useful applications in simple scenarios where you can live with a fixed camera and don't want to use a full Raspberry Pi.\nIn the next weeks I settled to finally try TensorFlow Lite for Microcontrollers on my ESP32, so I'll try to do a comparison between them and this example and report my results.\nNow that you can do image classification on your ESP32, can you think of a use case you will be able to apply this code to? \nLet me know in the comments, we could even try realize it together if you need some help.\n\r\nCheck the full project code on Github\nL'articolo Apple or Orange? Image recognition with ESP32 and Arduino proviene da Eloquent Arduino Blog.", "date_published": "2020-01-12T11:32:08+01:00", "date_modified": "2020-05-31T18:51:27+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "camera", "esp32", "microml", "svm", "Arduino Machine learning", "Computer vision" ], "attachments": [ { "url": "https://eloquentarduino.github.io/wp-content/uploads/2020/01/Apple-vs-Orange.mp4", "mime_type": "video/mp4", "size_in_bytes": 1642079 } ] }, { "id": "https://eloquentarduino.github.io/?p=688", "url": "https://eloquentarduino.github.io/2019/12/machine-learning-on-attiny85/", "title": "Embedded Machine learning on Attiny85", "content_html": "You won't believe it, but you can run Machine learning on embedded systems like an Attiny85 (and many others Attiny)!
\n![\"ri-elaborated](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/attiny85_ml.png\")
\n
When I first run a Machine learning project on my Arduino Nano (old generation), it already felt a big achievement. I mean, that board has only 32 Kb of program space and 2 Kb of RAM and you can buy a chinese clone for around 2.50 $.
\nIt already opened the path to a embedded machine learning at a new scale, given the huge amount of microcontrollers ready to become "intelligent".
\nBut it was not enough for me: after all, the MicroML generator exports plain C that should run on any embedded system, not only on Arduino boards.
\nSo I setup to test if I could go even smaller and run it on the #1 of tiny chips: the Attiny85.
\nMicroML exports plain C that could run anywhere, not only on Arduino boards.
Click To Tweet
No, I couldn't.
\nThe generated code makes use of a variadic function, which seems to not be supported by the Attiny compiler in the Arduino IDE.
\nSo I had to come up with an alternative implementation to make it work.
\nFortunately I already experimented with a non-variadic version when first writing the porter, so it was a matter of refreshing that algorithm and try it out.
\nGuess what? It compiled!
\nSo I tried porting one my earliear tutorial (the color identification one) to the Attiny and...
\nBoom! Machine learning on an Attiny85!
Click To Tweet
Here's a step-by-step tutorial on how you can do it too.
\nWe're going to use the RGB components of a color sensor (TCS3200 in my case) to infer which object we're pointing it at. This means our features are going to be 3-dimensional, which leads to a really simple model with very high accuracy.
\nYou must do this step on a board with a Serial interface, like an Arduino Uno / Nano / Pro Mini. See the original tutorial for the code of this step.
\nThis part is exactly the same as the original, except for a single parameter: you will pass platform=attiny to the port function.
from sklearn.svm import SVC\nfrom micromlgen import port\n\n# put your samples in the dataset folder\n# one class per file\n# one feature vector per line, in CSV format\nfeatures, classmap = load_features('dataset/')\nX, y = features[:, :-1], features[:, -1]\nclassifier = SVC(kernel='linear').fit(X, y)\nc_code = port(classifier, classmap=classmap, platform='attiny')\nprint(c_code)At this point you have to copy the printed code and import it in your project, in a file called model.h.
Since we don't have a Serial, we will blink a LED a number of times dependant on the prediction result.
\n#include "model.h"\n\n#define LED 0\n\nvoid loop() {\n readRGB();\n classify();\n delay(1000);\n}\n\nvoid classify() {\n for (uint8_t times = predict(features) + 1; times > 0; times--) {\n digitalWrite(LED, HIGH);\n delay(10);\n digitalWrite(LED, LOW);\n delay(10);\n }\n}Here we are: put some colored object in front of the sensor and see the LED blink.
\nOn my machine, the sketch requires 3434 bytes (41%) of program space and 21 bytes (4%) of RAM. This means you could actually run machine learning in even less space than what the Attiny85 provides.
\nThis model in particular it's so tiny you can run in even on an Attiny45, which has only 4 Kb of flash and 256 bytes of RAM.
\nI'd like you to look at the RAM figure for a moment: 21 bytes. 21 bytes is all the memory you need to run a Machine learning algorithm on a microcontroller. This is the result of the implementation I chose: the least RAM overhead possible. I challenge you to go any lower than this.
\n21 bytes is all the memory you need to run a Machine learning algorithm on a microcontroller
Click To Tweet
Did you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.
Check the full project code on Github
\nL'articolo Embedded Machine learning on Attiny85 proviene da Eloquent Arduino Blog.
\n", "content_text": "You won't believe it, but you can run Machine learning on embedded systems like an Attiny85 (and many others Attiny)!\n\n\nWhen I first run a Machine learning project on my Arduino Nano (old generation), it already felt a big achievement. I mean, that board has only 32 Kb of program space and 2 Kb of RAM and you can buy a chinese clone for around 2.50 $.\nIt already opened the path to a embedded machine learning at a new scale, given the huge amount of microcontrollers ready to become "intelligent". \nBut it was not enough for me: after all, the MicroML generator exports plain C that should run on any embedded system, not only on Arduino boards.\nSo I setup to test if I could go even smaller and run it on the #1 of tiny chips: the Attiny85.\nMicroML exports plain C that could run anywhere, not only on Arduino boards.Click To Tweet\nNo, I couldn't.\nThe generated code makes use of a variadic function, which seems to not be supported by the Attiny compiler in the Arduino IDE.\nSo I had to come up with an alternative implementation to make it work.\nFortunately I already experimented with a non-variadic version when first writing the porter, so it was a matter of refreshing that algorithm and try it out.\nGuess what? It compiled!\nSo I tried porting one my earliear tutorial (the color identification one) to the Attiny and...\nBoom! Machine learning on an Attiny85!Click To Tweet\nHere's a step-by-step tutorial on how you can do it too.\n\nI strongly suggest you read the original tutorial before following this one, because I won't go into too much details on the common steps here.\n\nTable of contentsFeatures definitionRecord sample dataTrain and export the SVM classifierRun the inferenceProject figures\n1. Features definition\nWe're going to use the RGB components of a color sensor (TCS3200 in my case) to infer which object we're pointing it at. This means our features are going to be 3-dimensional, which leads to a really simple model with very high accuracy.\n\nThe Attiny85 has 8 Kb of flash and 512 bytes of RAM, so you won't be able to load any model that uses more than a few features (probably less than 10).\n\n2. Record sample data\nYou must do this step on a board with a Serial interface, like an Arduino Uno / Nano / Pro Mini. See the original tutorial for the code of this step.\n3. Train and export the SVM classifier\nThis part is exactly the same as the original, except for a single parameter: you will pass platform=attiny to the port function.\nfrom sklearn.svm import SVC\nfrom micromlgen import port\n\n# put your samples in the dataset folder\n# one class per file\n# one feature vector per line, in CSV format\nfeatures, classmap = load_features('dataset/')\nX, y = features[:, :-1], features[:, -1]\nclassifier = SVC(kernel='linear').fit(X, y)\nc_code = port(classifier, classmap=classmap, platform='attiny')\nprint(c_code)\n\nThe Attiny mode has been implemented in version 0.8 of micromlgen: if you installed an earlier version, first update\n\nAt this point you have to copy the printed code and import it in your project, in a file called model.h.\n\r\n\r\n\r\n \r\n\tFinding this content useful?\r\n\r\n\t\r\n\r\n\t\r\n\t\t\r\n\t\t\r\n\t \r\n \r\n \r\n \r\n\r\n\r\n\r\n\n4. Run the inference\nSince we don't have a Serial, we will blink a LED a number of times dependant on the prediction result.\n#include "model.h"\n\n#define LED 0\n\nvoid loop() {\n readRGB();\n classify();\n delay(1000);\n}\n\nvoid classify() {\n for (uint8_t times = predict(features) + 1; times > 0; times--) {\n digitalWrite(LED, HIGH);\n delay(10);\n digitalWrite(LED, LOW);\n delay(10);\n }\n}\nHere we are: put some colored object in front of the sensor and see the LED blink.\nProject figures\nOn my machine, the sketch requires 3434 bytes (41%) of program space and 21 bytes (4%) of RAM. This means you could actually run machine learning in even less space than what the Attiny85 provides. \nThis model in particular it's so tiny you can run in even on an Attiny45, which has only 4 Kb of flash and 256 bytes of RAM.\nI'd like you to look at the RAM figure for a moment: 21 bytes. 21 bytes is all the memory you need to run a Machine learning algorithm on a microcontroller. This is the result of the implementation I chose: the least RAM overhead possible. I challenge you to go any lower than this.\n21 bytes is all the memory you need to run a Machine learning algorithm on a microcontrollerClick To Tweet\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.\n\r\nCheck the full project code on Github\nL'articolo Embedded Machine learning on Attiny85 proviene da Eloquent Arduino Blog.", "date_published": "2019-12-23T17:57:15+01:00", "date_modified": "2020-05-31T18:51:39+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ] }, { "id": "https://eloquentarduino.github.io/?p=180", "url": "https://eloquentarduino.github.io/2019/12/word-classification-using-arduino/", "title": "Word classification using Arduino and MicroML", "content_html": "In this Arduno Machine learning tutorial we're going to use a microphone to identify the word you speak.
\nThis is going to run on an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM.
![\"from](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/WakeWord.jpg\")
\n
In this project the features are going to be the Fast Fourier Transform of 50 analog readings from a microphone, taken starting from when a loud sound is detected, sampled at intervals of 5 millis.
\nThe microphone we're going to use is a super simple device: it produces an analog signal (0-1024) based on the sound it detects.
\n![\"from](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/WakeWord-Sensor.jpg\")
When working with audio you almost always don't want to use raw readings, since they're hardly useful. Instead you often go with Fourier Transform, which extracts the frequency information from a time signal. That's going to become our features vector: let's see how in the next step.
\nFirst of all, we start with raw audio data. The following plot is me saying random words.
\n![\"Raw](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Raw-audio.gif\")
#define MIC A0\n#define INTERVAL 5\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(MIC, INPUT);\n}\n\nvoid loop() {\n Serial.println(analogRead(MIC));\n delay(INTERVAL);\n}For the Fourier Transform to work, we need to provide as input an array of values both positive and negative. analogRead() is returning only positive values, tough, so we need to translate them.
int16_t readMic() {\n // this translated the analog value to a proper interval\n return (analogRead(MIC) - 512) >> 2;\n}As in the tutorial about gesture classification, we'll start recording the features when a word is beginning to be pronounced. Also in this project we'll use a threshold to detect the start of a word.
\nTo do this, we first record a "background" sound level, that is the value produced by the sensor when we're not talking at all.
\nfloat backgroundSound = 0;\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(MIC, INPUT);\n calibrate();\n}\n\nvoid calibrate() {\n for (int i = 0; i < 200; i++)\n backgroundSound += readMic();\n\n backgroundSound /= 200;\n\n Serial.print("Background sound level is ");\n Serial.println(backgroundSound);\n}At this point we can check for the starting of a word when the detected sound level exceeds tha background one by a given threshold.
\n// adjust as per your need\n// it will depend on the sensitivity of you microphone\n#define SOUND_THRESHOLD 3\n\nvoid loop() {\n if (!soundDetected()) {\n delay(10);\n return;\n }\n}\n\nbool soundDetected() {\n return abs(read() - backgroundSound) >= SOUND_THRESHOLD;\n}As for the gestures, we'll record a fixed number of readings at a fixed interval.
\nHere a tradeoff arises: you want to have a decent number of readings to be able to accurately describe the words you want to classify, but not too much otherwise your model is going to be too large to fit in your board.
I made some experiments, and I got good results with 32 samples at 5 millis interval, which covers ~150 ms of speech.
\n#define NUM_SAMPLES 32\n#define INTERVAL 5\n\nfloat features[NUM_SAMPLES];\ndouble featuresForFFT[NUM_SAMPLES];\n\nvoid loop() {\n if (!soundDetected()) {\n delay(10);\n return;\n }\n\n captureWord();\n printFeatures();\n delay(1000);\n}\n\nvoid captureWord() {\n for (uint16_t i = 0; i < NUM_SAMPLES; i++) {\n features[i] = readMic();\n delay(INTERVAL);\n }\n}\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\nHere we are with the Fourier Transform. When implemented in software, the most widely implementation of the FT is actually called Fast Fourier Transform (FFT), which is - as you may guess - a fast implementation of the FT.
\nLuckily for us, there exists a library for Arduino that does FFT.
\nAnd is so easy to use that we only need a line to get usable results!
\n#include <arduinoFFT.h>\n\narduinoFFT fft;\n\nvoid captureWord() {\n for (uint16_t i = 0; i < NUM_SAMPLES; i++) {\n featuresForFFT[i] = readMic();\n delay(INTERVAL);\n }\n\n fft.Windowing(featuresForFFT, NUM_SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);\n\n for (int i = 0; i < NUM_SAMPLES; i++)\n features[i] = featuresForFFT[i];\n}You don't need to know what the Windowing function actually does (I don't either): what matters is that it extracts meaningful informations from our signal. Since it overwrites the features array, after calling that line we have what we need to input to our classifier.
At this point, record 10-15 samples for each word and save them to a file, one for each word.
\nFor a detailed guide refer to the tutorial
\r\n\r\n\r\n
from sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)At this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.
In this project on Machine learning we're not achieving 100% accuracy easily.
\nAudio is quite noise, so you should experiment with a few params for the classifier and choose the ones that perform best. I'll showcase a few examples:
![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Word-classification-linear-kernel-87-accuracy.png\")
![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Word-classification-poly-3-kernel-91-accuracy.png\")
![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Word-classification-rbf-kernel-94-accuracy.png\")
Here's an overview table of the 3 tests I did.
\n| Kernel | \nNo. support vectors | \nAvg. accuracy | \n
|---|---|---|
| Linear | \n22 | \n87% | \n
| Poly 3 | \n29 | \n91% | \n
| RBF | \n36 | \n94% | \n
Of course the one with the RBF kernel would be the most desiderable since it has a very high accuracy: 36 support vectors, tough, will produce a model too large to fit on an Arduino Nano.
\nSo you're forced to pick the one with the highest accuracy that fit on your board: in my case it was the Linear kernel one.
\n#include "model.h"\n\nvoid loop() {\n if (!soundDetected()) {\n delay(10);\n return;\n }\n\n captureWord();\n Serial.print("You said ");\n Serial.println(classIdxToName(predict(features)));\n\n delay(1000);\n}And that's it: word classification through machine learning on your Arduino board! Say some word and see the classification result on the Serial monitor.
\nHere's me testing the system (English is not my language, so forgive my bad pronounce). The video quality is very low, I know, but you get the point.
\n\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.
Check the full project code on Github
\nL'articolo Word classification using Arduino and MicroML proviene da Eloquent Arduino Blog.
\n", "content_text": "In this Arduno Machine learning tutorial we're going to use a microphone to identify the word you speak.\nThis is going to run on an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM.\n\n\nIn this project the features are going to be the Fast Fourier Transform of 50 analog readings from a microphone, taken starting from when a loud sound is detected, sampled at intervals of 5 millis.\n\nThis tutorial is not about \"Wake word\" detection: it can't distinguish a known word from any other word. It can classify the word you speak among the ones you trained it to recognize!!!\n\nTable of contentsFeatures definitionRecord sample dataTranslate the raw valuesDetect soundRecord the wordsFast Fourier TransformTrain and export the classifierSelect a suitable modelRun the inference\n1. Features definition\nThe microphone we're going to use is a super simple device: it produces an analog signal (0-1024) based on the sound it detects. \n\nWhen working with audio you almost always don't want to use raw readings, since they're hardly useful. Instead you often go with Fourier Transform, which extracts the frequency information from a time signal. That's going to become our features vector: let's see how in the next step.\n2. Record sample data\nFirst of all, we start with raw audio data. The following plot is me saying random words.\n\n#define MIC A0\n#define INTERVAL 5\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(MIC, INPUT);\n}\n\nvoid loop() {\n Serial.println(analogRead(MIC));\n delay(INTERVAL);\n}\n2.1 Translate the raw values\nFor the Fourier Transform to work, we need to provide as input an array of values both positive and negative. analogRead() is returning only positive values, tough, so we need to translate them.\nint16_t readMic() {\n // this translated the analog value to a proper interval\n return (analogRead(MIC) - 512) >> 2;\n}\n2.2 Detect sound\nAs in the tutorial about gesture classification, we'll start recording the features when a word is beginning to be pronounced. Also in this project we'll use a threshold to detect the start of a word.\nTo do this, we first record a "background" sound level, that is the value produced by the sensor when we're not talking at all.\nfloat backgroundSound = 0;\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(MIC, INPUT);\n calibrate();\n}\n\nvoid calibrate() {\n for (int i = 0; i < 200; i++)\n backgroundSound += readMic();\n\n backgroundSound /= 200;\n\n Serial.print("Background sound level is ");\n Serial.println(backgroundSound);\n}\nAt this point we can check for the starting of a word when the detected sound level exceeds tha background one by a given threshold.\n// adjust as per your need\n// it will depend on the sensitivity of you microphone\n#define SOUND_THRESHOLD 3\n\nvoid loop() {\n if (!soundDetected()) {\n delay(10);\n return;\n }\n}\n\nbool soundDetected() {\n return abs(read() - backgroundSound) >= SOUND_THRESHOLD;\n}\n2.3 Record the words\nAs for the gestures, we'll record a fixed number of readings at a fixed interval.\nHere a tradeoff arises: you want to have a decent number of readings to be able to accurately describe the words you want to classify, but not too much otherwise your model is going to be too large to fit in your board.\nI made some experiments, and I got good results with 32 samples at 5 millis interval, which covers ~150 ms of speech.\n\nThe dilemma here is that the Fourier Transform to work needs a number of samples that is a power of 2. So, if you think 32 features are not enough for you, you're forced to go with at least 64: this has a REALLY bad impact on the model size.\n\n#define NUM_SAMPLES 32\n#define INTERVAL 5\n\nfloat features[NUM_SAMPLES];\ndouble featuresForFFT[NUM_SAMPLES];\n\nvoid loop() {\n if (!soundDetected()) {\n delay(10);\n return;\n }\n\n captureWord();\n printFeatures();\n delay(1000);\n}\n\nvoid captureWord() {\n for (uint16_t i = 0; i < NUM_SAMPLES; i++) {\n features[i] = readMic();\n delay(INTERVAL);\n }\n}\n\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\n\n2.4 Fast Fourier Transform\nHere we are with the Fourier Transform. When implemented in software, the most widely implementation of the FT is actually called Fast Fourier Transform (FFT), which is - as you may guess - a fast implementation of the FT. \nLuckily for us, there exists a library for Arduino that does FFT.\nAnd is so easy to use that we only need a line to get usable results!\n#include <arduinoFFT.h>\n\narduinoFFT fft;\n\nvoid captureWord() {\n for (uint16_t i = 0; i < NUM_SAMPLES; i++) {\n featuresForFFT[i] = readMic();\n delay(INTERVAL);\n }\n\n fft.Windowing(featuresForFFT, NUM_SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);\n\n for (int i = 0; i < NUM_SAMPLES; i++)\n features[i] = featuresForFFT[i];\n}\nYou don't need to know what the Windowing function actually does (I don't either): what matters is that it extracts meaningful informations from our signal. Since it overwrites the features array, after calling that line we have what we need to input to our classifier.\nAt this point, record 10-15 samples for each word and save them to a file, one for each word.\n\nAfter you have recorded the samples for a word, I suggest you to manually check them. It is sufficient to look at the first 3 values: if one of them seems to be clearly out of range, I suggest you to delete it. You will lose some accuracy, but your model will be smaller.\n\n3. Train and export the classifier\r\n\r\nFor a detailed guide refer to the tutorial\r\n\r\n\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)\r\n\r\nAt this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.\nIn this project on Machine learning we're not achieving 100% accuracy easily.\nAudio is quite noise, so you should experiment with a few params for the classifier and choose the ones that perform best. I'll showcase a few examples:\n\n\n\n2.5 Select a suitable model\nHere's an overview table of the 3 tests I did.\n\n\n\nKernel\nNo. support vectors\nAvg. accuracy\n\n\n\n\nLinear\n22\n87%\n\n\nPoly 3\n29\n91%\n\n\nRBF\n36\n94%\n\n\n\nOf course the one with the RBF kernel would be the most desiderable since it has a very high accuracy: 36 support vectors, tough, will produce a model too large to fit on an Arduino Nano.\nSo you're forced to pick the one with the highest accuracy that fit on your board: in my case it was the Linear kernel one.\n4. Run the inference\n#include "model.h"\n\nvoid loop() {\n if (!soundDetected()) {\n delay(10);\n return;\n }\n\n captureWord();\n Serial.print("You said ");\n Serial.println(classIdxToName(predict(features)));\n\n delay(1000);\n}\nAnd that's it: word classification through machine learning on your Arduino board! Say some word and see the classification result on the Serial monitor. \nHere's me testing the system (English is not my language, so forgive my bad pronounce). The video quality is very low, I know, but you get the point.\nhttps://eloquentarduino.github.io/wp-content/uploads/2019/12/Word-classification.mp4\n\nAs you can hear from the video, you should be quite accurate when pronouncing the words. I have to admit there are cases where the system totally fails to classify correctly the words. Restarting helps most of the time, so I'm suspecting there could be some kind of leak that \"corrupts\" the inference procedure.\n\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.\n\r\nCheck the full project code on Github\nL'articolo Word classification using Arduino and MicroML proviene da Eloquent Arduino Blog.", "date_published": "2019-12-22T19:12:59+01:00", "date_modified": "2020-12-24T08:16:24+01:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ], "attachments": [ { "url": "https://eloquentarduino.github.io/wp-content/uploads/2019/12/Word-classification.mp4", "mime_type": "video/mp4", "size_in_bytes": 2055653 } ] }, { "id": "https://eloquentarduino.github.io/?p=224", "url": "https://eloquentarduino.com/projects/arduino-indoor-positioning", "title": "Wifi indoor positioning using Arduino and Machine Learning in 4 steps", "content_html": "In this Arduno Machine learning project we're going to use the nearby WiFi access points to locate where we are. For this project to work you will need a Wifi equipped board, such as ESP8266 or ESP32.
\n![\"Wifi](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/illustrations_ambient-wifi-site-survey2.jpg\")
\n
We all are used to GPS positioning: our device will use satellites to track our position on Earth. GPS works very well and with a very high accuracy (you can expect only a few meters of error).
\nBut it suffers a problem: it needs Line of Sight (a clear path from your device to the satellites). If you're not in an open place, like inside a building, you're out of luck.
\nThe task of detecting where you are when GPS localization is not an option is called indoor positioning: it could be in a building, an airport, a parking garage.
\nThere are lots of different approaches to this task (Wikipedia lists more than 10 of them), each with a varying level of commitment, difficulty, cost and accuracy.
\nFor this tutorial I opted for one that is both cost-efficient, easy to implement and readily available in most of the locations: WiFi indoor positioning.
\nIn this tutorial about Machine learning on Arduino we're going to use Wifi indoor positioning to detect in which room of our house / office we are. This is the most basic task we can accomplish and will get us a feeling of level of accuracy we can achieve with such a simple setup.
\nOn this basis, we'll construct more sophisticated projects in future posts.
\nTo accomplish this tutorial, you really need 2 things:
\nIf you're doing this at home or in your office, there's a good change your neighbours have WiFi networks in their apartments you can leverage. If you live in an isolated contryside, sorry, this will not work for you.
\nSo, how exactly does Wifi indoor positioning works in conjuction with Machine learning?
\nLet's pretend there are 5 different WiFi networks around you, like in the picture below.
\n![\"Wifi](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Wifi1.png\")
As you can see there are two markers on the map: each of these markers will "see" different networks, with different signal strengths (a.k.a RSSI).
\nAs you move around, those numbers will change: each room will be identified by the unique combination of the RSSIs.
\nThe features for this project are going to be the RSSIs (Received signal strength indication) of the known WiFi networks. If a network is out of range, it will have an RSSI equal to 0.
\nBefore actually recording the sample data to train our classifier, we need to do some preliminary work. This is because not all networks will be visible all the time: we have to work, however, with a fixed number of features.
\nFirst of all we need to enumerate all the networks we will encounter during the inference process.
\nTo begin, we take a "reconnaissance tour" of the locations we want to predict and log all the networks we detect. Load the following sketch and take note of all the networks that appear on the Serial monitor.
\n#include <WiFi.h>\n\nvoid setup() {\n Serial.begin(115200);\n WiFi.mode(WIFI_STA);\n WiFi.disconnect();\n}\n\nvoid loop() {\n int numNetworks = WiFi.scanNetworks();\n\n for (int i = 0; i < numNetworks; i++) {\n Serial.println(WiFi.SSID(i));\n\n delay(3000);\n}Now that we have a bunch of SSIDs, we need to assign each SSID to a fixed index, from 0 to MAX_NETWORKS.
You can implement this part as you like, but in this demo I'll make use of a class I wrote called Array (you can see the source code and example on Github), which implements 2 useful functions:
push() to add an element to the arrayindexOf() to get the index of an element.See how to install the Eloquent library if you don't have it already installed.
\nAt this point we populate the array with all the networks we saved from the reconnaissance tour.
#include <eDataStructures.h>\n\n#define MAX_NETWORKS 10\n\nusing namespace Eloquent::DataStructures;\n\ndouble features[MAX_NETWORKS];\nArray<String, MAX_NETWORKS> knownNetworks("");\n\nvoid setup() {\n Serial.begin(115200);\n WiFi.mode(WIFI_STA);\n WiFi.disconnect();\n\n knownNetworks.push("SSID #0");\n knownNetworks.push("SSID #1");\n knownNetworks.push("SSID #2");\n knownNetworks.push("SSID #3");\n // and so on\n}The second step is to convert the scan results into a features vector. Each feature will be the RSSI of the given SSID, in the exact order we populated the knownNetworks array.
In practice:
\nfeatures[0] == RSSI of SSID #0;\nfeatures[1] == RSSI of SSID #1;\nfeatures[2] == RSSI of SSID #2;\nfeatures[3] == RSSI of SSID #3;\n// and so onThe code below will do the job.
\nvoid loop() {\n scan();\n printFeatures();\n delay(3000);\n}\n\nvoid scan() {\n int numNetworks = WiFi.scanNetworks();\n\n resetFeatures();\n\n // assign RSSIs to feature vector\n for (int i = 0; i < numNetworks; i++) {\n String ssid = WiFi.SSID(i);\n uint16_t networkIndex = knownNetworks.indexOf(ssid);\n\n // only create feature if the current SSID is a known one\n if (!isnan(networkIndex))\n features[networkIndex] = WiFi.RSSI(i);\n }\n}\n\n// reset all features to 0\nvoid resetFeatures() {\n const uint16_t numFeatures = sizeof(features) / sizeof(double);\n\n for (int i = 0; i < numFeatues; i++)\n features[i] = 0;\n}\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\nGrab some recordings just staying in a location for a few seconds and save the serial output to a file; then move to the next location and repeat: 10-15 samples for each location will suffice.
\nIf you do a good job, you should end with distinguible features, as show in the plot below.
\n![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-from-Wifi-indoor-positioning-features.png\")
\n// replace\nfeatures[networkIndex] = WiFi.RSSI(i);\n\n// with\n#define MIN_RSSI -90 // adjust to your needs\n\nfeatures[networkIndex] = WiFi.RSSI(i) > MIN_RSSI ? WiFi.RSSI(i) : 0;\nFor a detailed guide refer to the tutorial
\r\n\r\n\r\n
from sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)At this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.
#include "model.h"\n\nvoid loop() {\n scan();\n classify();\n delay(3000);\n}\n\nvoid classify() {\n Serial.print("You are in ");\n Serial.println(classIdxToName(predict(features)));\n}Move around your house/office/whatever and see your location printed on the serial monitor!
\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.
Check the full project code on Github
\nL'articolo Wifi indoor positioning using Arduino and Machine Learning in 4 steps proviene da Eloquent Arduino Blog.
\n", "content_text": "In this Arduno Machine learning project we're going to use the nearby WiFi access points to locate where we are. For this project to work you will need a Wifi equipped board, such as ESP8266 or ESP32.\n\n\n\nI published an upgraded version of this project, with all the code and tools you need: check it out\nTable of contentsWhat is Wifi indoor positioning?What will we use it for?What you needHow it worksFeatures definitionRecord sample dataEnumerate the access pointsCreate an access point arrayConvert to features vectorTrain and export the classifierRun the inference\nWhat is Wifi indoor positioning?\nWe all are used to GPS positioning: our device will use satellites to track our position on Earth. GPS works very well and with a very high accuracy (you can expect only a few meters of error).\nBut it suffers a problem: it needs Line of Sight (a clear path from your device to the satellites). If you're not in an open place, like inside a building, you're out of luck.\nThe task of detecting where you are when GPS localization is not an option is called indoor positioning: it could be in a building, an airport, a parking garage. \nThere are lots of different approaches to this task (Wikipedia lists more than 10 of them), each with a varying level of commitment, difficulty, cost and accuracy.\nFor this tutorial I opted for one that is both cost-efficient, easy to implement and readily available in most of the locations: WiFi indoor positioning.\nWhat will we use it for?\nIn this tutorial about Machine learning on Arduino we're going to use Wifi indoor positioning to detect in which room of our house / office we are. This is the most basic task we can accomplish and will get us a feeling of level of accuracy we can achieve with such a simple setup.\nOn this basis, we'll construct more sophisticated projects in future posts.\nWhat you need\nTo accomplish this tutorial, you really need 2 things:\n\na WiFi equipped board (ESP8266, ESP32, Arduino MKR WiFi 1010...)\nbe in a place with a few WiFi networks around\n\nIf you're doing this at home or in your office, there's a good change your neighbours have WiFi networks in their apartments you can leverage. If you live in an isolated contryside, sorry, this will not work for you.\nHow it works\nSo, how exactly does Wifi indoor positioning works in conjuction with Machine learning? \nLet's pretend there are 5 different WiFi networks around you, like in the picture below.\n\nAs you can see there are two markers on the map: each of these markers will "see" different networks, with different signal strengths (a.k.a RSSI).\nAs you move around, those numbers will change: each room will be identified by the unique combination of the RSSIs.\n1. Features definition\nThe features for this project are going to be the RSSIs (Received signal strength indication) of the known WiFi networks. If a network is out of range, it will have an RSSI equal to 0.\n2. Record sample data\nBefore actually recording the sample data to train our classifier, we need to do some preliminary work. This is because not all networks will be visible all the time: we have to work, however, with a fixed number of features.\n2.1 Enumerate the access points\nFirst of all we need to enumerate all the networks we will encounter during the inference process. \nTo begin, we take a "reconnaissance tour" of the locations we want to predict and log all the networks we detect. Load the following sketch and take note of all the networks that appear on the Serial monitor.\n#include <WiFi.h>\n\nvoid setup() {\n Serial.begin(115200);\n WiFi.mode(WIFI_STA);\n WiFi.disconnect();\n}\n\nvoid loop() {\n int numNetworks = WiFi.scanNetworks();\n\n for (int i = 0; i < numNetworks; i++) {\n Serial.println(WiFi.SSID(i));\n\n delay(3000);\n}\n2.2 Create an access point array\nNow that we have a bunch of SSIDs, we need to assign each SSID to a fixed index, from 0 to MAX_NETWORKS.\nYou can implement this part as you like, but in this demo I'll make use of a class I wrote called Array (you can see the source code and example on Github), which implements 2 useful functions: \n\npush() to add an element to the array\nindexOf() to get the index of an element.\n\nSee how to install the Eloquent library if you don't have it already installed.\nAt this point we populate the array with all the networks we saved from the reconnaissance tour.\n#include <eDataStructures.h>\n\n#define MAX_NETWORKS 10\n\nusing namespace Eloquent::DataStructures;\n\ndouble features[MAX_NETWORKS];\nArray<String, MAX_NETWORKS> knownNetworks("");\n\nvoid setup() {\n Serial.begin(115200);\n WiFi.mode(WIFI_STA);\n WiFi.disconnect();\n\n knownNetworks.push("SSID #0");\n knownNetworks.push("SSID #1");\n knownNetworks.push("SSID #2");\n knownNetworks.push("SSID #3");\n // and so on\n}\n2.3 Convert to features vector\nThe second step is to convert the scan results into a features vector. Each feature will be the RSSI of the given SSID, in the exact order we populated the knownNetworks array.\nIn practice:\nfeatures[0] == RSSI of SSID #0;\nfeatures[1] == RSSI of SSID #1;\nfeatures[2] == RSSI of SSID #2;\nfeatures[3] == RSSI of SSID #3;\n// and so on\nThe code below will do the job.\nvoid loop() {\n scan();\n printFeatures();\n delay(3000);\n}\n\nvoid scan() {\n int numNetworks = WiFi.scanNetworks();\n\n resetFeatures();\n\n // assign RSSIs to feature vector\n for (int i = 0; i < numNetworks; i++) {\n String ssid = WiFi.SSID(i);\n uint16_t networkIndex = knownNetworks.indexOf(ssid);\n\n // only create feature if the current SSID is a known one\n if (!isnan(networkIndex))\n features[networkIndex] = WiFi.RSSI(i);\n }\n}\n\n// reset all features to 0\nvoid resetFeatures() {\n const uint16_t numFeatures = sizeof(features) / sizeof(double);\n\n for (int i = 0; i < numFeatues; i++)\n features[i] = 0;\n}\n\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\n\nGrab some recordings just staying in a location for a few seconds and save the serial output to a file; then move to the next location and repeat: 10-15 samples for each location will suffice.\nIf you do a good job, you should end with distinguible features, as show in the plot below.\n\n\nRSSIs may be a little noisy, mostly on the boundaries where weak networks may appear and disappear with a very low RSSI: this was not a problem for me, but if you're getting bad results you may filter out those low values.\n\n// replace\nfeatures[networkIndex] = WiFi.RSSI(i);\n\n// with\n#define MIN_RSSI -90 // adjust to your needs\n\nfeatures[networkIndex] = WiFi.RSSI(i) > MIN_RSSI ? WiFi.RSSI(i) : 0;\n\n\n3. Train and export the classifier\r\n\r\nFor a detailed guide refer to the tutorial\r\n\r\n\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)\r\n\r\nAt this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.\n4. Run the inference\n#include "model.h"\n\nvoid loop() {\n scan();\n classify();\n delay(3000);\n}\n\nvoid classify() {\n Serial.print("You are in ");\n Serial.println(classIdxToName(predict(features)));\n}\nMove around your house/office/whatever and see your location printed on the serial monitor!\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.\n\r\nCheck the full project code on Github\nL'articolo Wifi indoor positioning using Arduino and Machine Learning in 4 steps proviene da Eloquent Arduino Blog.", "date_published": "2019-12-20T18:31:16+01:00", "date_modified": "2020-08-09T16:16:51+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ] }, { "id": "https://eloquentarduino.github.io/?p=35", "url": "https://eloquentarduino.github.io/2019/12/how-to-do-gesture-identification-on-arduino/", "title": "How to do Gesture identification through machine learning on Arduino", "content_html": "In this Arduno Machine learning project we're going to use an accelerometer sensor to identify the gestures you play.
\nThis is a remake of the project found on the Tensorflow blog. We're going to use a lot less powerful chip in this tutorial, tough: an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM.
![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Gestures-features-RBF-kernel-0.001-gamma.png\")
\n
We're going to use the accelerations along the 3 axis (X, Y, Z) coming from an IMU to infer which gesture we're playing. We'll use a fixed number of recordings (NUM_SAMPLES) starting from the first detection of movement.
This means our feature vectors are going to be of dimension 3 * NUM_SAMPLES, which can become too large to fit in the memory of the Arduino Nano. We'll start with a low value for NUM_SAMPLES to keep it as leaner as possible: if your classifications suffer from poor accuracy, you can increase this number.
First of all, we need to read the raw data from the IMU. This piece of code will be different based on the specific chip you use. To keep things consistent, we'll wrap the IMU logic in 2 functions: imu_setup and imu_read.
I'll report a couple of example implementations for the MPU6050 and the MPU9250 (these are the chip I have at hand). You should save whichever code you use in a file called imu.h.
#include "Wire.h"\n// library from https://github.com/jrowberg/i2cdevlib/tree/master/Arduino/MPU6050\n#include "MPU6050.h"\n#define OUTPUT_READABLE_ACCELGYRO\n\nMPU6050 imu;\n\nvoid imu_setup() {\n Wire.begin();\n imu.initialize();\n}\n\nvoid imu_read(float *ax, float *ay, float *az) {\n int16_t _ax, _ay, _az, _gx, _gy, _gz;\n\n imu.getMotion6(&_ax, &_ay, &_az, &_gx, &_gy, &_gz);\n\n *ax = _ax;\n *ay = _ay;\n *az = _az;\n}#include "Wire.h"\n// library from https://github.com/bolderflight/MPU9250\n#include "MPU9250.h"\n\nMPU9250 imu(Wire, 0x68);\n\nvoid imu_setup() {\n Wire.begin();\n imu.begin();\n}\n\nvoid imu_read(float *ax, float *ay, float *az) {\n imu.readSensor();\n\n *ax = imu.getAccelX_mss();\n *ay = imu.getAccelY_mss();\n *az = imu.getAccelZ_mss();\n}In the main .ino file, we dump the values to Serial monitor / plotter.
\n#include "imu.h"\n\n#define NUM_SAMPLES 30\n#define NUM_AXES 3\n// sometimes you may get "spikes" in the readings\n// set a sensible value to truncate too large values\n#define TRUNCATE_AT 20\n\ndouble features[NUM_SAMPLES * NUM_AXES];\n\nvoid setup() {\n Serial.begin(115200);\n imu_setup();\n}\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax, -TRUNCATE_AT, TRUNCATE_AT);\n ay = constrain(ay, -TRUNCATE_AT, TRUNCATE_AT);\n az = constrain(az, -TRUNCATE_AT, TRUNCATE_AT);\n\n Serial.print(ax);\n Serial.print('\\t');\n Serial.print(ay);\n Serial.print('\\t');\n Serial.println(az);\n}Open the Serial plotter and make some movement to have an idea of the range of your readings.
\n![\"Raw](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Raw-gestures.gif"\")
Due to gravity, we get a stable value of -9.8 on the Z axis at rest (you can see this in the previous image). Since I'd like to have almost 0 at rest, I created a super simple calibration procedure to remove this fixed offset from the readings.
\ndouble baseline[NUM_AXES];\ndouble features[NUM_SAMPLES * NUM_AXES];\n\nvoid setup() {\n Serial.begin(115200);\n imu_setup();\n calibrate();\n}\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n}\n\nvoid calibrate() {\n float ax, ay, az;\n\n for (int i = 0; i < 10; i++) {\n imu_read(&ax, &ay, &az);\n delay(100);\n }\n\n baseline[0] = ax;\n baseline[1] = ay;\n baseline[2] = az;\n}![\"Calibrated](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Calibrated-gestures.gif\")
Much better.
\nNow we need to check if motion is happening. To keep it simple, we'll use a naive approach that will look for an high value in the acceleration: if a threshold is exceeded, a gesture is starting.
\nIf you did the calibration step, a threshold of 5 should work well. If you didn't calibrate, you have to come up with a value that suits your needs.
\n#include imu.h\n\n#define ACCEL_THRESHOLD 5\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n if (!motionDetected(ax, ay, az)) {\n delay(10);\n return;\n }\n}\n\nbool motionDetected(float ax, float ay, float az) {\n return (abs(ax) + abs(ay) + abs(az)) > ACCEL_THRESHOLD;\n}If no motion is happening, we don't take any action and keep watching. If motion is happening, we print the next NUM_SAMPLES readings to Serial.
void loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n if (!motionDetected(ax, ay, az)) {\n delay(10);\n return;\n }\n\n recordIMU();\n printFeatures();\n delay(2000);\n}\n\nvoid recordIMU() {\n float ax, ay, az;\n\n for (int i = 0; i < NUM_SAMPLES; i++) {\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n features[i * NUM_AXES + 0] = ax;\n features[i * NUM_AXES + 1] = ay;\n features[i * NUM_AXES + 2] = az;\n\n delay(INTERVAL);\n }\n}\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\nRecord 15-20 samples for each geasture and save them to a file, one for each gesture. Since we're dealing with highly dimensional data, you should collect as much samples as possible, to average out the noise.
\nFor a detailed guide refer to the tutorial
\r\n\r\n\r\n
from sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)At this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.
In this project on Machine learning, differently from the previous and simpler ones, we're not achieving 100% accuracy easily. Motion is quite noise, so you should experiment with a few params for the classifier and choose the ones that perform best. I'll showcase a few examples:
\n![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Gestures-features-Linear-kernel.png\")
![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Gestures-features-Polynomial-kernel.png\")
![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-of-Gestures-features-RBF-kernel-0.01-gamma.png\")
Now that we selected the best model, we have to export it to C code. Here comes the culprit: not all models will fit on your board.
\nThe core of SVM (Support Vector Machines) are support vectors: each trained classifier will be characterized by a certain number of them. The problem is: if there're too much, the generated code will be too large to fit in your flash.
\nFor this reason, instead of selecting the best model on accuracy, you should make a ranking, from the best performing to the worst. For each model, starting from the top, you should import it in your Arduino project and try to compile: if it fits, fine, you're done. Otherwise you should pick the next and try again.
\nIt may seem a tedious process, but keep in mind that we're trying to infer a class from 90 features in 2 Kb of RAM and 32 Kb of flash: I think this is an acceptable tradeoff.
\nWe're fitting a model to infer a class from 90 features in 2 Kb of RAM and 32 Kb of flash!
Click To Tweet
I'll report a few figures for different combinations I tested.
\n| Kernel | \nC | \nGamma | \nDegree | \nVectors | \nFlash size | \nRAM (b) | \nAvg accuracy | \n
|---|---|---|---|---|---|---|---|
| RBF | \n10 | \n0.001 | \n- | \n37 | \n53 Kb | \n1228 | \n99% | \n
| Poly | \n100 | \n0.001 | \n2 | \n12 | \n25 Kb | \n1228 | \n99% | \n
| Poly | \n100 | \n0.001 | \n3 | \n25 | \n40 Kb | \n1228 | \n97% | \n
| Linear | \n50 | \n- | \n1 | \n40 | \n55 Kb | \n1228 | \n95% | \n
| RBF | \n100 | \n0.01 | \n- | \n61 | \n80 Kb | \n1228 | \n95% | \n
As you can see, we achieved a very high accuracy on the test set for all the classifiers: only one, though, fitted on the Arduino Nano. Of course, if you use a larger board, you can deploy the others too.
\nRAM column: all the values are equal: this is because in the implementation is independant from the number of support vectors and only depends on the number of features.#include "model.h"\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n if (!motionDetected(ax, ay, az)) {\n delay(10);\n return;\n }\n\n recordIMU();\n classify();\n delay(2000);\n}\n\nvoid classify() {\n Serial.print("Detected gesture: ");\n Serial.println(classIdxToName(predict(features)));\n}Here we are: it has been a long post, but now you can classify gestures with an Arduino Nano and 2 Kb of RAM.
\nNo fancy Neural Networks, no Tensorflow, no 32-bit ARM processors: plain old SVM on plain old 8 bits with 97% accuracy.
Click To Tweet
Here's a short demo of me playing 3 gestures and getting the results on the serial monitor.
\n\nOn my machine, the sketch targeted at the Arduino Nano (old generation) requires 25310 bytes (82%) of program space and 1228 bytes (59%) of RAM. This means you could actually run machine learning in even less space than what the Arduino Nano provides. So, the answer to the question Can I run machine learning on Arduino? is definetly YES.
\n
Did you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.
Check the full project code on Github
\nL'articolo How to do Gesture identification through machine learning on Arduino proviene da Eloquent Arduino Blog.
\n", "content_text": "In this Arduno Machine learning project we're going to use an accelerometer sensor to identify the gestures you play.\nThis is a remake of the project found on the Tensorflow blog. We're going to use a lot less powerful chip in this tutorial, tough: an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM.\n\n\nTable of contentsFeatures definitionRecord sample dataRead the IMU sensorCalibrationDetect first motionRecord featuresTrain and export the classifierSelect a suitable modelRun the inference\n1. Features definition\nWe're going to use the accelerations along the 3 axis (X, Y, Z) coming from an IMU to infer which gesture we're playing. We'll use a fixed number of recordings (NUM_SAMPLES) starting from the first detection of movement. \nThis means our feature vectors are going to be of dimension 3 * NUM_SAMPLES, which can become too large to fit in the memory of the Arduino Nano. We'll start with a low value for NUM_SAMPLES to keep it as leaner as possible: if your classifications suffer from poor accuracy, you can increase this number.\n2. Record sample data\n2.1 Read the IMU sensor\nFirst of all, we need to read the raw data from the IMU. This piece of code will be different based on the specific chip you use. To keep things consistent, we'll wrap the IMU logic in 2 functions: imu_setup and imu_read. \nI'll report a couple of example implementations for the MPU6050 and the MPU9250 (these are the chip I have at hand). You should save whichever code you use in a file called imu.h. \n#include "Wire.h"\n// library from https://github.com/jrowberg/i2cdevlib/tree/master/Arduino/MPU6050\n#include "MPU6050.h"\n#define OUTPUT_READABLE_ACCELGYRO\n\nMPU6050 imu;\n\nvoid imu_setup() {\n Wire.begin();\n imu.initialize();\n}\n\nvoid imu_read(float *ax, float *ay, float *az) {\n int16_t _ax, _ay, _az, _gx, _gy, _gz;\n\n imu.getMotion6(&_ax, &_ay, &_az, &_gx, &_gy, &_gz);\n\n *ax = _ax;\n *ay = _ay;\n *az = _az;\n}\n#include "Wire.h"\n// library from https://github.com/bolderflight/MPU9250\n#include "MPU9250.h"\n\nMPU9250 imu(Wire, 0x68);\n\nvoid imu_setup() {\n Wire.begin();\n imu.begin();\n}\n\nvoid imu_read(float *ax, float *ay, float *az) {\n imu.readSensor();\n\n *ax = imu.getAccelX_mss();\n *ay = imu.getAccelY_mss();\n *az = imu.getAccelZ_mss();\n}\nIn the main .ino file, we dump the values to Serial monitor / plotter.\n#include "imu.h"\n\n#define NUM_SAMPLES 30\n#define NUM_AXES 3\n// sometimes you may get "spikes" in the readings\n// set a sensible value to truncate too large values\n#define TRUNCATE_AT 20\n\ndouble features[NUM_SAMPLES * NUM_AXES];\n\nvoid setup() {\n Serial.begin(115200);\n imu_setup();\n}\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax, -TRUNCATE_AT, TRUNCATE_AT);\n ay = constrain(ay, -TRUNCATE_AT, TRUNCATE_AT);\n az = constrain(az, -TRUNCATE_AT, TRUNCATE_AT);\n\n Serial.print(ax);\n Serial.print('\\t');\n Serial.print(ay);\n Serial.print('\\t');\n Serial.println(az);\n}\nOpen the Serial plotter and make some movement to have an idea of the range of your readings.\n\n2.2 Calibration\nDue to gravity, we get a stable value of -9.8 on the Z axis at rest (you can see this in the previous image). Since I'd like to have almost 0 at rest, I created a super simple calibration procedure to remove this fixed offset from the readings.\ndouble baseline[NUM_AXES];\ndouble features[NUM_SAMPLES * NUM_AXES];\n\nvoid setup() {\n Serial.begin(115200);\n imu_setup();\n calibrate();\n}\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n}\n\nvoid calibrate() {\n float ax, ay, az;\n\n for (int i = 0; i < 10; i++) {\n imu_read(&ax, &ay, &az);\n delay(100);\n }\n\n baseline[0] = ax;\n baseline[1] = ay;\n baseline[2] = az;\n}\n\nMuch better.\n2.3 Detect first motion\nNow we need to check if motion is happening. To keep it simple, we'll use a naive approach that will look for an high value in the acceleration: if a threshold is exceeded, a gesture is starting. \nIf you did the calibration step, a threshold of 5 should work well. If you didn't calibrate, you have to come up with a value that suits your needs.\n#include imu.h\n\n#define ACCEL_THRESHOLD 5\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n if (!motionDetected(ax, ay, az)) {\n delay(10);\n return;\n }\n}\n\nbool motionDetected(float ax, float ay, float az) {\n return (abs(ax) + abs(ay) + abs(az)) > ACCEL_THRESHOLD;\n}\n2.4 Record features\nIf no motion is happening, we don't take any action and keep watching. If motion is happening, we print the next NUM_SAMPLES readings to Serial. \nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n if (!motionDetected(ax, ay, az)) {\n delay(10);\n return;\n }\n\n recordIMU();\n printFeatures();\n delay(2000);\n}\n\nvoid recordIMU() {\n float ax, ay, az;\n\n for (int i = 0; i < NUM_SAMPLES; i++) {\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n features[i * NUM_AXES + 0] = ax;\n features[i * NUM_AXES + 1] = ay;\n features[i * NUM_AXES + 2] = az;\n\n delay(INTERVAL);\n }\n}\n\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\n\nRecord 15-20 samples for each geasture and save them to a file, one for each gesture. Since we're dealing with highly dimensional data, you should collect as much samples as possible, to average out the noise.\n3. Train and export the classifier\r\n\r\nFor a detailed guide refer to the tutorial\r\n\r\n\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)\r\n\r\nAt this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.\nIn this project on Machine learning, differently from the previous and simpler ones, we're not achieving 100% accuracy easily. Motion is quite noise, so you should experiment with a few params for the classifier and choose the ones that perform best. I'll showcase a few examples:\n\n\n\n\n3.1 Select a suitable model\nNow that we selected the best model, we have to export it to C code. Here comes the culprit: not all models will fit on your board.\nThe core of SVM (Support Vector Machines) are support vectors: each trained classifier will be characterized by a certain number of them. The problem is: if there're too much, the generated code will be too large to fit in your flash.\nFor this reason, instead of selecting the best model on accuracy, you should make a ranking, from the best performing to the worst. For each model, starting from the top, you should import it in your Arduino project and try to compile: if it fits, fine, you're done. Otherwise you should pick the next and try again.\nIt may seem a tedious process, but keep in mind that we're trying to infer a class from 90 features in 2 Kb of RAM and 32 Kb of flash: I think this is an acceptable tradeoff.\nWe're fitting a model to infer a class from 90 features in 2 Kb of RAM and 32 Kb of flash!Click To Tweet\nI'll report a few figures for different combinations I tested.\n\n\n\nKernel\nC\nGamma\nDegree\nVectors\nFlash size\nRAM (b)\nAvg accuracy\n\n\n\n\nRBF\n10\n0.001\n-\n37\n53 Kb\n1228\n99%\n\n\nPoly\n100\n0.001\n2\n12\n25 Kb\n1228\n99%\n\n\nPoly\n100\n0.001\n3\n25\n40 Kb\n1228\n97%\n\n\nLinear\n50\n-\n1\n40\n55 Kb\n1228\n95%\n\n\nRBF\n100\n0.01\n-\n61\n80 Kb\n1228\n95%\n\n\n\nAs you can see, we achieved a very high accuracy on the test set for all the classifiers: only one, though, fitted on the Arduino Nano. Of course, if you use a larger board, you can deploy the others too.\nAs a side note, take a look at the RAM column: all the values are equal: this is because in the implementation is independant from the number of support vectors and only depends on the number of features.\n4. Run the inference\n#include "model.h"\n\nvoid loop() {\n float ax, ay, az;\n\n imu_read(&ax, &ay, &az);\n\n ax = constrain(ax - baseline[0], -TRUNCATE, TRUNCATE);\n ay = constrain(ay - baseline[1], -TRUNCATE, TRUNCATE);\n az = constrain(az - baseline[2], -TRUNCATE, TRUNCATE);\n\n if (!motionDetected(ax, ay, az)) {\n delay(10);\n return;\n }\n\n recordIMU();\n classify();\n delay(2000);\n}\n\nvoid classify() {\n Serial.print("Detected gesture: ");\n Serial.println(classIdxToName(predict(features)));\n}\nHere we are: it has been a long post, but now you can classify gestures with an Arduino Nano and 2 Kb of RAM. \nNo fancy Neural Networks, no Tensorflow, no 32-bit ARM processors: plain old SVM on plain old 8 bits with 97% accuracy.Click To Tweet\nHere's a short demo of me playing 3 gestures and getting the results on the serial monitor.\nhttps://eloquentarduino.github.io/wp-content/uploads/2019/12/Gesture-identification-in-action.mp4\nProject figures\r\nOn my machine, the sketch targeted at the Arduino Nano (old generation) requires 25310 bytes (82%) of program space and 1228 bytes (59%) of RAM. This means you could actually run machine learning in even less space than what the Arduino Nano provides. So, the answer to the question Can I run machine learning on Arduino? is definetly YES.\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.\n\r\nCheck the full project code on Github\nL'articolo How to do Gesture identification through machine learning on Arduino proviene da Eloquent Arduino Blog.", "date_published": "2019-12-19T14:25:46+01:00", "date_modified": "2020-06-11T13:31:17+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ], "attachments": [ { "url": "https://eloquentarduino.github.io/wp-content/uploads/2019/12/Gesture-identification-in-action.mp4", "mime_type": "video/mp4", "size_in_bytes": 1035484 } ] }, { "id": "https://eloquentarduino.github.io/?p=194", "url": "https://eloquentarduino.github.io/2019/12/morse-identification-on-arduino/", "title": "Morse alphabet identification on Arduino with Machine learning", "content_html": "In this Arduno Machine learning project we're going to identify the letters from the Morse alphabet.
\nIn practice, we'll translate dots (\u2022) and dashes (\u2012) "typed" with a push button into meaningful characters.
\nIn this tutorial we're going to target an Arduino Nano board (old generation), equipped with 32 kb of flash and only 2 kb of RAM.
\n
![\"credits](\"https://i.ytimg.com/vi/L6gxfX4GrbI/maxresdefault.jpg\")
For our task we'll use a simple push button as input and a fixed number of samples taken at a fixed interval (100 ms), starting from the first detection of the button press. I chose to record 30 samples for each letter, but you can easily customize the value as per your needs.
\nWith 30 samples at 100 ms frequency, we'll have 3 seconds to "type" the letter and on the Serial monitor will appear a sequence of 0s and 1s, representing if the button was pressed or not; the inference procedure will translate this sequence into a letter.
\nAs a reference, here are a couple example of what we'll be working with.
// A (\u2022\u2012)\n0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1\n\n// D (\u2012\u2022\u2022)\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1\n\n// E (\u2022)\n0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1To the bare minimum, we'll need a push button and two wires: one to ground and the other to a digital pin. Since in the example we'll make the button an INPUT_PULLUP, we'll read 0 when the button is pressed and 1 when not.
![\"credits](\"https://www.arduino.cc/en/uploads/Tutorial/PullUp_bbd.png\")
All we need to do is detect a press and record the following 30 samples of the digital pin:
\n#define IN 4\n#define NUM_SAMPLES 30\n#define INTERVAL 100\n\ndouble features[NUM_SAMPLES];\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(IN, INPUT_PULLUP);\n}\n\nvoid loop() {\n if (digitalRead(IN) == 0) {\n recordButtonStatus();\n printFeatures();\n delay(1000);\n }\n\n delay(10);\n}\n\nvoid recordButtonStatus() {\n for (int i = 0; i < NUM_SAMPLES; i++) {\n features[i] = digitalRead(IN);\n delay(INTERVAL);\n } \n}\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\nOpen the Serial monitor and type a few times each letter: try to introduce some variations each time, for example waiting some more milliseconds before releasing the dash.
\nSave the recordings for each letter in a file named after the letter, so you will get meaningful results later on.
\nYou may end with duplicate recordings: don't worry, that's not a problem. I'll paste my recordings for a few letters, as a reference.
\n// A (\u2022\u2012)\n0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1\n\n// D (\u2012\u2022\u2022)\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,0,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,1,1,1,1,1,0,0,1,1,1,1,1,1,1\n\n// E (\u2022)\n0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n\n// S (\u2022\u2022\u2022)\n0,0,0,1,1,1,0,0,0,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,0,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n\n// T (\u2012)\n0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1If you do a good job, you should end with quite distinguible features, as show in the plot below.
\n![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-from-Morse-alphabet-identification-features.svg\")
For a detailed guide refer to the tutorial
\r\n\r\n\r\n
from sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)At this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.
#include "model.h"\n\nvoid loop() {\n if (digitalRead(IN) == 0) {\n recordButtonStatus();\n Serial.print("Detected letter: ");\n Serial.println(classIdxToName(predict(features)));\n delay(1000);\n }\n\n delay(10);\n}Type some letter using the push button and see the identified value printed on the serial monitor.
\nThat\u2019s it: you deployed machine learning in 2 Kb!
\nOn my machine, the sketch targeted at the Arduino Nano (old generation) requires 12546 bytes (40%) of program space and 366 bytes (17%) of RAM. This means you could actually run machine learning in even less space than what the Arduino Nano provides. So, the answer to the question Can I run machine learning on Arduino? is definetly YES.
\n
Did you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.
Check the full project code on Github
\nL'articolo Morse alphabet identification on Arduino with Machine learning proviene da Eloquent Arduino Blog.
\n", "content_text": "In this Arduno Machine learning project we're going to identify the letters from the Morse alphabet.\nIn practice, we'll translate dots (\u2022) and dashes (\u2012) "typed" with a push button into meaningful characters.\nIn this tutorial we're going to target an Arduino Nano board (old generation), equipped with 32 kb of flash and only 2 kb of RAM.\n\n\nTable of contentsFeatures definitionRecord sample dataTrain and export the classifierRun the inference\n1. Features definition\nFor our task we'll use a simple push button as input and a fixed number of samples taken at a fixed interval (100 ms), starting from the first detection of the button press. I chose to record 30 samples for each letter, but you can easily customize the value as per your needs. \nWith 30 samples at 100 ms frequency, we'll have 3 seconds to "type" the letter and on the Serial monitor will appear a sequence of 0s and 1s, representing if the button was pressed or not; the inference procedure will translate this sequence into a letter.\nAs a reference, here are a couple example of what we'll be working with.\n// A (\u2022\u2012)\n0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1\n\n// D (\u2012\u2022\u2022)\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1\n\n// E (\u2022)\n0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n2. Record sample data\nTo the bare minimum, we'll need a push button and two wires: one to ground and the other to a digital pin. Since in the example we'll make the button an INPUT_PULLUP, we'll read 0 when the button is pressed and 1 when not. \n\nAll we need to do is detect a press and record the following 30 samples of the digital pin:\n#define IN 4\n#define NUM_SAMPLES 30\n#define INTERVAL 100\n\ndouble features[NUM_SAMPLES];\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(IN, INPUT_PULLUP);\n}\n\nvoid loop() {\n if (digitalRead(IN) == 0) {\n recordButtonStatus();\n printFeatures();\n delay(1000);\n }\n\n delay(10);\n}\n\nvoid recordButtonStatus() {\n for (int i = 0; i < NUM_SAMPLES; i++) {\n features[i] = digitalRead(IN);\n delay(INTERVAL);\n } \n}\n\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\n\nOpen the Serial monitor and type a few times each letter: try to introduce some variations each time, for example waiting some more milliseconds before releasing the dash.\n If you've never typed morse code before (as me), choose letters with few keystrokes and quite differentiable, otherwise you will need to be very good with the timing.\nSave the recordings for each letter in a file named after the letter, so you will get meaningful results later on.\nYou may end with duplicate recordings: don't worry, that's not a problem. I'll paste my recordings for a few letters, as a reference.\n// A (\u2022\u2012)\n0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1\n\n// D (\u2012\u2022\u2022)\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,0,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,1,1,1,1,1,0,0,1,1,1,1,1,1,1\n\n// E (\u2022)\n0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n\n// S (\u2022\u2022\u2022)\n0,0,0,1,1,1,0,0,0,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,0,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,1,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,1,1,1,1,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,1,1,1,1,0,0,1,1,1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n\n// T (\u2012)\n0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\n0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\nIf you do a good job, you should end with quite distinguible features, as show in the plot below.\n\n3. Train and export the classifier\r\n\r\nFor a detailed guide refer to the tutorial\r\n\r\n\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)\r\n\r\nAt this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.\n4. Run the inference\n#include "model.h"\n\nvoid loop() {\n if (digitalRead(IN) == 0) {\n recordButtonStatus();\n Serial.print("Detected letter: ");\n Serial.println(classIdxToName(predict(features)));\n delay(1000);\n }\n\n delay(10);\n}\nType some letter using the push button and see the identified value printed on the serial monitor.\nThat\u2019s it: you deployed machine learning in 2 Kb! \nProject figures\r\nOn my machine, the sketch targeted at the Arduino Nano (old generation) requires 12546 bytes (40%) of program space and 366 bytes (17%) of RAM. This means you could actually run machine learning in even less space than what the Arduino Nano provides. So, the answer to the question Can I run machine learning on Arduino? is definetly YES.\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.\n\r\nCheck the full project code on Github\nL'articolo Morse alphabet identification on Arduino with Machine learning proviene da Eloquent Arduino Blog.", "date_published": "2019-12-06T13:07:38+01:00", "date_modified": "2020-05-31T18:52:15+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ] }, { "id": "https://eloquentarduino.github.io/?p=6", "url": "https://eloquentarduino.github.io/2019/12/color-identification-on-arduino/", "title": "How to do color identification through machine learning on Arduino", "content_html": "In this Arduno Machine learning project we're going to use an RGB sensor to identify objects based on their color.
\nThis is a remake of the project found on the Tensorflow blog. We're going to use a lot less powerful chip in this tutorial, tough: an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM.
\n
We're going to use the RGB components of a color sensor (TCS3200 in my case) to infer which object we're pointing it at. This means our features are going to be of 3-dimensional, which leads to a really simple model with very high accuracy.
\nYou can do color identification on Arduino using Machine learning without Neural Networks #Arduino #microml #ml #tinyml #MachineLearning #ai #svm
Click To Tweet
We don't need any processing to get from the sensor readings to the feature vector, so the code will be straight-forward: read each component from the sensor and assign it to the features array. This part will vary based on the specific chip you have: I'll report the code for a TCS 230/3200.
\n#define S2 2\n#define S3 3\n#define sensorOut 4\n\ndouble features[3];\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(S2, OUTPUT);\n pinMode(S3, OUTPUT);\n pinMode(sensorOut, INPUT);\n}\n\nvoid loop() {\n readRGB();\n printFeatures();\n delay(100);\n}\n\nint readComponent(bool s2, bool s3) {\n delay(10);\n digitalWrite(S2, s2);\n digitalWrite(S3, s3);\n\n return pulseIn(sensorOut, LOW);\n}\n\nvoid readRGB() {\n features[0] = readComponent(LOW, LOW);\n features[1] = readComponent(HIGH, HIGH);\n features[2] = readComponent(LOW, HIGH);\n}\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\nOpen the Serial monitor and put some colored objects in front of the sensor: move the object a bit and rotate it, so the samples will include different shades of the color.
\nSave the recordings for each color in a file named after the color, so you will get meaningful results later on.
\nIf you do a good job, you should end with distinguible features, as show in the contour plot below.
\n![\"Decision](\"https://eloquentarduino.github.io/wp-content/uploads/2019/12/Decision-boundaries-of-2-PCA-components-from-the-colors-features.png\")
For a detailed guide refer to the tutorial
\r\n\r\n\r\n
from sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)At this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.
#include model.h\n\nvoid loop() {\n readRGB();\n Serial.println(classIdxToName(predict(features)));\n delay(1000);\n}Put some colored object in front of the sensor and see the identified object name printed on the serial monitor.
\nGiven the simplicity of the task, you should easily achieve near 100% accuracy for different colors (I had some troubles distinguishing orange from yellow because of the bad illumination). Just be sure to replicate the exact same setup both during training and classification.
\nThat\u2019s it: you deployed machine learning in 2 Kb!
\nOn my machine, the sketch targeted at the Arduino Nano (old generation) requires 5570 bytes (18%) of program space and 266 bytes (12%) of RAM. This means you could actually run machine learning in even less space than what the Arduino Nano provides. So, the answer to the question Can I run machine learning on Arduino? is definetly YES.
\n
Did you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.
Check the full project code on Github
\nL'articolo How to do color identification through machine learning on Arduino proviene da Eloquent Arduino Blog.
\n", "content_text": "In this Arduno Machine learning project we're going to use an RGB sensor to identify objects based on their color.\nThis is a remake of the project found on the Tensorflow blog. We're going to use a lot less powerful chip in this tutorial, tough: an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM.\n\nTable of contentsFeatures definitionRecord sample dataTrain and export the classifierRun the inference\n1. Features definition\nWe're going to use the RGB components of a color sensor (TCS3200 in my case) to infer which object we're pointing it at. This means our features are going to be of 3-dimensional, which leads to a really simple model with very high accuracy.\nYou can do color identification on Arduino using Machine learning without Neural Networks #Arduino #microml #ml #tinyml #MachineLearning #ai #svmClick To Tweet\n2. Record sample data\nWe don't need any processing to get from the sensor readings to the feature vector, so the code will be straight-forward: read each component from the sensor and assign it to the features array. This part will vary based on the specific chip you have: I'll report the code for a TCS 230/3200. \n#define S2 2\n#define S3 3\n#define sensorOut 4\n\ndouble features[3];\n\nvoid setup() {\n Serial.begin(115200);\n pinMode(S2, OUTPUT);\n pinMode(S3, OUTPUT);\n pinMode(sensorOut, INPUT);\n}\n\nvoid loop() {\n readRGB();\n printFeatures();\n delay(100);\n}\n\nint readComponent(bool s2, bool s3) {\n delay(10);\n digitalWrite(S2, s2);\n digitalWrite(S3, s3);\n\n return pulseIn(sensorOut, LOW);\n}\n\nvoid readRGB() {\n features[0] = readComponent(LOW, LOW);\n features[1] = readComponent(HIGH, HIGH);\n features[2] = readComponent(LOW, HIGH);\n}\n\r\nvoid printFeatures() {\r\n const uint16_t numFeatures = sizeof(features) / sizeof(float);\r\n \r\n for (int i = 0; i < numFeatures; i++) {\r\n Serial.print(features[i]);\r\n Serial.print(i == numFeatures - 1 ? 'n' : ',');\r\n }\r\n}\r\n\nOpen the Serial monitor and put some colored objects in front of the sensor: move the object a bit and rotate it, so the samples will include different shades of the color.\nSave the recordings for each color in a file named after the color, so you will get meaningful results later on.\n\nDon\u2019t forget to sample the \u201cempty color\u201d too: don\u2019t put anything in front of the sensor and let it record for a while.\n\nIf you do a good job, you should end with distinguible features, as show in the contour plot below.\n\n3. Train and export the classifier\r\n\r\nFor a detailed guide refer to the tutorial\r\n\r\n\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom micromlgen import port\r\n\r\n# put your samples in the dataset folder\r\n# one class per file\r\n# one feature vector per line, in CSV format\r\nfeatures, classmap = load_features('dataset/')\r\nX, y = features[:, :-1], features[:, -1]\r\nclassifier = RandomForestClassifier(n_estimators=30, max_depth=10).fit(X, y)\r\nc_code = port(classifier, classmap=classmap)\r\nprint(c_code)\r\n\r\nAt this point you have to copy the printed code and import it in your Arduino project, in a file called model.h.\n4. Run the inference\n#include model.h\n\nvoid loop() {\n readRGB();\n Serial.println(classIdxToName(predict(features)));\n delay(1000);\n}\nPut some colored object in front of the sensor and see the identified object name printed on the serial monitor.\nDo you remember the \"empty color\"? It needs to be recorded so you will get \"empty\" when no object is present, otherwise you'll get unexpected predictions\nGiven the simplicity of the task, you should easily achieve near 100% accuracy for different colors (I had some troubles distinguishing orange from yellow because of the bad illumination). Just be sure to replicate the exact same setup both during training and classification.\nThat\u2019s it: you deployed machine learning in 2 Kb! \nProject figures\r\nOn my machine, the sketch targeted at the Arduino Nano (old generation) requires 5570 bytes (18%) of program space and 266 bytes (12%) of RAM. This means you could actually run machine learning in even less space than what the Arduino Nano provides. So, the answer to the question Can I run machine learning on Arduino? is definetly YES.\nDid you find this tutorial useful? Was is it easy to follow or did I miss something? Let me know in the comments so I can keep improving the blog.\n\r\nCheck the full project code on Github\nL'articolo How to do color identification through machine learning on Arduino proviene da Eloquent Arduino Blog.", "date_published": "2019-12-01T11:35:29+01:00", "date_modified": "2020-05-31T18:52:23+02:00", "authors": [ { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" } ], "author": { "name": "simone", "url": "https://eloquentarduino.github.io/author/simone/", "avatar": "http://1.gravatar.com/avatar/d670eb91ca3b1135f213ffad83cb8de4?s=512&d=mm&r=g" }, "tags": [ "microml", "svm", "Arduino Machine learning" ] } ] }