TinyML is the practice of running a trained machine learning model directly on a microcontroller — a chip costing a few dollars, drawing milliwatts, with kilobytes rather than gigabytes of memory. It is not a research curiosity: keyword spotting, gesture recognition, vibration-based fault detection, and simple visual classification all run today on hardware you can put in a battery-powered consumer product. The model is trained in the cloud on a real machine; only inference runs on the chip, and inference is far cheaper than training.

Why Run the Model on the Device Instead of the Cloud

Four reasons, and usually more than one applies:

  • Latency. A local inference completes in milliseconds with no network round trip. Anything reacting to a physical event — a gesture, an impact, a spoken wake word — needs that.
  • Power. Transmitting raw sensor data is expensive. A radio is typically the largest consumer in a battery-powered sensor. Processing locally and sending only a result, or nothing at all until something happens, can extend battery life from weeks to years.
  • Privacy. Audio and images that never leave the device are audio and images you never have to secure, store, or explain to a customer. For consumer and healthcare products this is increasingly a selling point rather than a technicality.
  • Availability and cost. No connectivity requirement means the product works in a basement, a factory, or a field, and you are not paying cloud inference and bandwidth costs on every unit forever — a recurring per-unit expense that quietly grows with your install base.

Applications That Are Genuinely Mature

Be realistic about what fits in a few hundred kilobytes:

  • Audio keyword spotting. Detecting a small vocabulary of wake words or commands. Well understood, widely deployed, modest resource needs.
  • Accelerometer and IMU classification. Gesture recognition, activity detection, fall detection, counting repetitions, distinguishing normal from abnormal motion. Among the best value-for-effort applications in the whole field.
  • Vibration and acoustic anomaly detection. Predictive maintenance on motors, pumps, and bearings — often the highest commercial value, since it turns a cheap sensor into a service. See predictive maintenance from product data.
  • Low-resolution vision. Person present or absent, object counting, orientation checks, simple defect detection on a constrained scene. Note the emphasis on constrained — general-purpose object recognition is not a microcontroller task.
  • Sensor fusion and denoising. Small models that clean up or interpret readings from cheap sensors, effectively buying accuracy with compute instead of hardware.

What does not fit: large language models, high-resolution video analysis, and anything needing a broad general-purpose understanding of the world. If your feature description reads like a cloud AI product, plan a cloud AI product — the broader tradeoff is covered in AI in hardware products.

How the Workflow Actually Goes

  1. Collect data from the real sensor. This is the step teams underestimate and the one that decides success. A model trained on data from a different sensor, mounting position, or enclosure will not work in your product. Collect from your hardware, in your mechanical configuration, in the environments you will ship into.
  2. Label it. Tedious, unavoidable, and usually the largest line item of effort.
  3. Train off-device. Standard tooling on a normal machine or in the cloud.
  4. Compress the model. Quantization from floating point to 8-bit integers is the standard move — it typically shrinks the model roughly fourfold and speeds it up substantially with modest accuracy loss. Pruning and architecture reduction come next.
  5. Convert and embed. The model becomes a byte array or a flash region that a lightweight inference runtime executes on the target.
  6. Validate on the hardware. Measure real accuracy, real inference time, and real energy per inference on the actual chip. Desktop accuracy figures do not transfer.
  7. Plan for updates. Models drift as products meet the real world. Shipping a new model without shipping a new device means you need OTA firmware updates designed in from the start.

What It Demands From the Hardware

TinyML is a memory problem before it is a compute problem. Practical guidance:

  • RAM is the binding constraint. The runtime needs space for the model's intermediate tensors, and that arena often dominates your memory budget. Hundreds of kilobytes of RAM opens up far more than tens.
  • Flash holds the weights. A quantized model of a few hundred kilobytes is common; make sure it coexists with your application, radio stack, and OTA slots.
  • A DSP or vector extension helps a lot. Arm Cortex-M4 and M7 class parts with SIMD extensions run these workloads meaningfully faster than a basic M0. Some newer parts include dedicated neural accelerators.
  • Clock speed trades against power. A faster inference at a higher clock can use less total energy than a slow one, because the chip returns to sleep sooner. Measure energy per inference, not current draw.
  • Sensor quality and sampling matter enormously. A better sensor or a cleaner mechanical mount often beats a better model.

Choosing the part is the pivotal decision, and it should be made with the model's memory footprint in hand rather than guessed — our guide to choosing a microcontroller applies directly, and if the workload is larger than expected the honest answer may be the step up described in microcontroller vs embedded Linux. Popular starting points for experimentation include widely available development boards on both mainstream families; see ESP32 vs STM32 for the practical differences.

Pitfalls Worth Knowing in Advance

  • The dataset is the project. Expect data collection and labeling to consume more of the schedule than model development. Budget accordingly.
  • Lab accuracy is not field accuracy. Real environments have noise, temperature drift, mounting variation, and users who behave unexpectedly. Validate in situ.
  • False positives destroy trust faster than misses. A device that alerts wrongly gets unplugged. Tune the operating threshold for the cost of each error type, not for headline accuracy.
  • Quantization can hurt unevenly. Overall accuracy may barely move while one important class degrades badly. Check per-class results, not the aggregate.
  • Memory fragmentation and stack overflow. Inference arenas interact badly with careless dynamic allocation. Static allocation and a measured worst case are the norm, and the choice of RTOS versus bare metal affects how you manage that.
  • Duty cycling is where the battery savings live. The model should only run when a cheap trigger says something interesting happened. Wiring that into your sleep-mode strategy is what turns a demo into a shippable product.

Where to Start

Start with the sensor and the data, not the model. Instrument a rough prototype, collect a few hours of labeled data from the real thing, and train a small baseline. If a simple model on honest data gets you most of the way, you have a viable product and a clear path. If it does not, the problem is almost always the data or the sensor placement — and finding that out in week two is enormously cheaper than finding it out in month six.

Projects House builds embedded systems with on-device inference — sensor selection, data collection rigs, model training and quantization, firmware integration, and the power budgeting that makes it run on a battery. If you have a product idea that depends on recognizing something in sensor data, describe it through our contact form and we will tell you whether it fits on a microcontroller.