Small CNNs and edge AI

What "small" actually means

The model behind DrawLa has 2,004,345 parameters and occupies exactly 8.0 megabytes in fp32. Quantised to 8-bit integers, roughly two megabytes would remain — less than a single photo from the same phone's camera.

That order of magnitude changes the discussion entirely. With a language model the question is whether it fits on a device at all. At two megabytes that question disappears; all that remains is whether you want it there.

Why CNNs are predictable on mobile hardware

The decisive advantage is the fixed input size. A tensor of shape 1 × 96 × 96 means every prediction occupies the same memory, takes the same time and produces the same intermediates. A mobile compiler can fix the entire memory plan ahead of time instead of allocating at runtime.

With variable-length sequences things are fundamentally different: demand grows with the input, and the worst case must always be budgeted for. For a drawing game that would mean slowing down precisely on elaborate drawings.

On top of that, convolutions, batch normalisation and simple activations are exactly the operations mobile accelerators were built for. At a batch size of one — the normal case in an interactive app — short latency and low heat matter more than throughput.

Quantisation

A model becomes small not only through few parameters but through how precisely the numbers are stored. Quantisation reduces weights and activations from 32-bit floating point to 8-bit integers: a quarter of the memory, a quarter of the memory bandwidth and considerably less energy per prediction.

Sketch recognition is a grateful case for this. The input is a greyscale image with values between 0 and 1, the structure is regular and the number of classes fixed. The accuracy loss for such models typically stays within a few tenths of a percentage point — negligible for a task whose top-1 accuracy is bounded by the ambiguity of its source material anyway.

What a mobile pipeline would look like

collect touch strokes
  → centre, scale, rasterise
  → tensor 1 × 96 × 96
  → quantised CNN on NPU/GPU/CPU
  → top-5 probabilities

Note what stands out: rasterising is the only part that would have to be rewritten — today it runs in Python on the server. And it must do pixel-identical work, otherwise the model sees inputs it never encountered during training. That port is the real work, not the model itself.

The hardware side is by now well developed. Qualcomm, for instance, describes GenieX as an on-device runtime that distributes models across NPU, GPU or CPU as appropriate. That targets large language and vision models, but it shows a trend small networks benefit from automatically.

Why DrawLa does not do it anyway

Everything so far argues for moving onto the device. Three reasons argue against it in a competitive game — and they weigh more.

Fairness. Recognition awards points. Run it on the device and the speed of that device, and the quantisation used there, become part of the outcome. Two players with identical drawings would get different results — unacceptable once a leaderboard is involved.

Tamperability. Whatever runs on the device can be modified. A local classifier that awards points is an invitation: you would not even have to touch the model, only the reported answer. Server-side scoring makes that entire class of attack moot.

Updatability. A better model is rolled out on the server in minutes and takes effect for everyone immediately. On the device it would depend on an app update, on store review and on whether players install it at all. For weeks several model versions would then run side by side — in the same match.

The rule of thumb: if the model computes only for the user, it belongs on the device. If it judges them against others, it belongs on the server.

Where on-device clearly wins

For other kinds of application the trade-off reverses. A pure practice mode without competition has no fairness question — there, local recognition would simply be better: instant, offline, and without any drawing leaving the device.

The same goes for anything using sketches as an input method: note-taking apps, form recognition, drawing tools. There is no opponent, no score and therefore no reason for a network round trip.

A hybrid setup is conceivable too: local for solo play and for the live preview while drawing, server-side for everything that is scored. The price is two recognition paths that must agree permanently — effort that only pays if offline operation is a genuine goal.

The export routes available for iOS and Android are described in iPhone & Android. How the model came about is covered in QuickDraw with PyTorch, and the model families are compared in RNN versus CNN.