Whitepaper
Edge AI: when to run AI on the device
What edge AI is
Edge AI runs a model on a device near the data, not on a distant cloud server. The device can be a sensor, a camera, a vehicle, or an industrial controller. Edge AI is not a replacement for the cloud. Many systems use both.
When to use it
Use edge AI if one or more of these conditions apply:
- Response time: The system must act before a cloud server can answer.
- Connection: The site has a slow connection, or the connection can stop.
- Data volume: Devices make more data than you can send at an acceptable cost.
- Data location: Sensitive data must stay on the site.
Use the cloud if the model is large, if you must change it frequently, or if it needs data from many sites. A plant can detect a machine fault on the device and send a summary to the cloud for analysis of the full fleet.
What makes it possible
Devices now have special chips for AI, such as the Apple Neural Engine, Google Edge TPU, and Qualcomm Hexagon. These chips run models with less power than general processors.
Teams can also make models smaller with quantisation, pruning, and knowledge distillation. These methods can decrease accuracy. Test the smaller model on real data from the site before you release it.
Privacy and security
Local processing can decrease the personal data that you send to other systems. It does not remove your legal obligations. If a camera processes images of people on the device, data protection law such as the GDPR can still apply.
Devices also have a different risk. People can get physical access to them. Protect the model, the data, and the update procedure on each device.
How to operate it
One model on one device is a test. A fleet of devices is an operation. Plan these items before you start:
- Safe updates for each device, with a procedure to go back to the previous version
- Regular checks of model quality when the site conditions change
- Clear responsibility for faults at each site
- Support with an agreed service level
These items are the reason why edge AI needs consulting, engineering, and operations together.
Get the full whitepaper
This page is a summary. Contact us to ask for the full whitepaper.