Models in the browser.
WHAT IT PROVIDES
Running models in a browser, using the device's processing.
WHAT THAT SUITS
Interactive features responding immediately Processing that should not leave the device Demonstrations and prototypes
WHAT IT COSTS
Model download, on every visit unless cached Processing on the visitor's device Battery consumption
WHY THAT MATTERS
A large model downloaded by every visitor is expensive for them and slow.
WHAT TO DO
Keep models small Cache aggressively Load only when the feature is used
WHAT TO NEVER DO
Load a model on page load for a feature most visitors never use.
WHAT ACCELERATION IS AVAILABLE
Graphics processing, where the browser and device support it.
WHAT TO HANDLE
Acceleration unavailable, falling back to slower processing Devices too slow to be usable
WHAT TO PROVIDE
An indication that processing is happening, and a way to cancel.
WHAT TO CONSIDER INSTEAD
Server-side inference, where the model is large or the device is modest.
WHAT TO MEASURE
Download size, and inference time on a mid-range phone.
