Edge or cloud inference? A decision checklist
Engineering guides Updated
If any of bandwidth, latency or privacy constrains you, run inference at the edge: millisecond response, video stays on-site, works offline. Cloud suits fast-iterating models and bursty compute; most factory solutions combine both.
Ask three questions: can you afford the bandwidth to stream video to the cloud? Does the control loop need millisecond response? Must data stay on-site? One yes puts inference on an edge device - which is exactly why Jetson- and RK3588-class hardware exists.
Conversely, weekly-retrained models, bursty compute or centrally managed fleets favour cloud or an on-prem cluster with L4/L40S nodes. Most lines combine both: real-time verdicts at the edge, iteration and data return in the cloud.