AI: Cloud, Edge or Hybrid?

One of the biggest trends in technology (and the world!) right now is Artificial Intelligence, especially Deep Learning.  Availability of huge quantities of data, data storage and processing power at low cost, algorithm innovation and the availability of AI toolkits like Caffe, Torch and TensorFlow have all contributed to the explosive adoption of AI in the past few years.


Technology companies are racing to add AI to their existing applications and platforms, and to create platforms to support AI.  Google, Microsoft, Amazon and IBM all have AI cloud offerings.  Facebook and Salesforce have embedded AI into their applications.  Samsung, Apple and Google are implementing AI in mobile devices.  This raises the question of where AI is best applied: the cloud, the edge or a combination of both.


The Case for AI in the Cloud

It takes loads of data and loads of computational power to successfully train accurate AI models.  For example, to achieve highly accurate word vectors for natural language processing, a team of researchers from Stanford University trained their model on a data set with 42 billion words1.  Similarly, AlphaGo used 1202 CPUs and 176 GPUs to ‘learn’ the game of Go well enough to beat the former world champion, Lee Sedol, in 20162.


Besides the need for large amounts of data storage and computing resources, training AI models is usually an activity that is done in batches, rather than continuously.  Using cloud resources to train neural networks and other AI models allows companies to only pay for the resources they need, rather than investing in infrastructure that may be underutilized most of the time.


The Case for AI on the Edge

There are many uses of AI that wouldn’t work efficiently if all data needed to be processed in the cloud, at least with today’s network speeds.  An autonomous vehicle, for example, must gather and process 0.75 GB of data per second from cameras, LiDAR and GPS to function3.  Besides the fact that today’s wide-area networks can’t handle that much bandwidth (for a single device, not to mention for the cars millions of cars on the road), cars depending on data processing in the cloud would also introduce latency and additional failure points that could be catastrophic.


I Want My Cake… and to Eat It Too

Why not use both models together?  Security cameras that are always running capture huge amounts of video data.  Usually, that data is only stored for a few days or weeks.  If a crime was captured on video, but nobody knew about it until after the data had expired, the footage would be useless.  A smart camera, like Nest Cam Outdoor4 and Nest Cam IQ5 can intelligently identify events that are worthy of an alert (e.g. a person has entered the field of vision).  Using the camera’s edge AI coupled with more sophisticated AI in the cloud to identify who the person is and what they are doing, could be used to trigger higher-level alerts (e.g. call the police) or to automatically make decisions to save a video segment until it has been reviewed.


Another use case for a hybrid model would be for autonomous vehicles to identify scenarios that could be used to train the fleet’s AI models to better handle certain conditions.  When the vehicle encountered such a situation, it could choose to store additional data for upload to the cloud, where it could be leveraged for model updates that would get pushed out to the vendor’s fleet.



It is exciting to see the incredible advances in AI that are happening.  Individuals and companies will need to make decisions about which platforms and approaches to buy.  As noted above, there are situations where cloud-based approaches, edge approaches and hybrid approaches have advantages.  This will be an important criterion for investment and purchasing decisions in the future.



  1. Pennington, Socher and Manning, 2014, GloVe: Global Vectors for Word Representation, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing
  2. Silver et all, 2016, Mastering the game of Go with deep neural networks and tree search, published in Nature (vol 529)




6 comments on “AI: Cloud, Edge or Hybrid?”

  1. An interesting comparison Eldon! You raised a crucial point about the current infrastructure in the world not being able to process data at the level required for AI to reach its peak. As Mr Donovan from AT&T highlighted, they are trying to future-proof their network, to allow the processing of information in a more interconnected world.

    1. Thanks, Jaykishen. It was really interesting to hear Mr Donovan’s comments about how they are overcoming obstacles to handle ever-higher bandwidth and provide ever-lower latency. He promoted the hybrid approach as well, with local GPUs being distributed broadly. I think this will be necessary until quantum networking is available. When it is, I’m not sure that it will be available for the last mile for a very long time – given last week’s comments by Dr Wesler about the difficulty in keeping strings of qbits stable. It seems like it would be extremely difficult to do that at scale. It makes me wonder if/when we’ll hit a wall like we did with Moore’s law.

  2. It is definitely exciting to think about combining Cloud technology with Artificial Intelligence! When it comes to AI the most important thing is how to enable it to self-learn and that can be achieved or we can say that it has already been achieved since Google developed the AlphaGo.
    The hybrid model does sound promising, however as in this week’s lecture, the most important thing in this model would still be the internet connection and the responsibility thus goes to companies like AT&T and Verizon. These companies would have to make sure that the internet covers almost everywhere in the country so that when a car is using cloud it does bump into a situation where there’s no internet or the internet is very slow so that the upload and download of data could not be completed in time and thus causing a problem.

    Jiading Zhu

    1. Hi Jiading – Good point on the reliability of the network. I would say that systems like autonomous cars need to be designed in a way that they don’t rely on the network 100% of the time – the driving decisions need to be handled locally so the car doesn’t crash if the network is unreachable. However, the network can be used to share information for overall system learning – the learning rate will improve if all of the cars on the network share their data. That could be done centrally, with periodic updates to the model parameters on each car when they are on the network.

  3. Well written post Eldon! Just a thought that arose in my mind was that AI will not be the same in a couple years from today. I feel it will be more smarter and capable of doing a lot more things such as IoT, cloud and other forms of integration to create one big chain of “smart solutions”. In regards to the other comment by Jaykishen, as network capabilities so will the possibility of having these linkages in technologies.

    1. Hi Saran – Yes, it will be exciting to see the advances in AI, IoT and cloud and the interplay between them. Your comment underlines the fact that company leaders in these spaces need to plan for ever-improving network reliability and speeds, as that shape decisions of where to place the computing capacity. To the degree that network bandwidth improves and latency goes down, it is more economical to compute in the cloud. On the other hand, as the price for computing goes down, more can be done on the edge, by IoT devices, for example. It seems that vendors in this space need to be aware of both curves so they can plot an optimal path for their offerings.


Comments are closed.