Jeff Dean on Large-Scale Deep Learning at Google
Wednesday, March 16, 2016 at 8:56AM
Todd Hoff in AI, Deep Learning, Example, google

If you can’t understand what’s in information then it’s going to be very difficult to organize it.


This quote is from Jeff Dean, currently a Wizard, er, Fellow in Google’s Systems Infrastructure Group. It’s taken from his recent talk: Large-Scale Deep Learning for Intelligent Computer Systems.

Since AlphaGo vs Lee Se-dol, the modern version of John Henry’s fatal race against a steam hammer, has captivated the world, as has the generalized fear of an AI apocalypse, it seems like an excellent time to gloss Jeff’s talk. And if you think AlphaGo is good now, just wait until it reaches beta.

Jeff is referring, of course, to Google’s infamous motto: organize the world’s information and make it universally accessible and useful.

Historically we might associate ‘organizing’ with gathering, cleaning, storing, indexing, reporting, and searching data. All the stuff early Google mastered. With that mission accomplished Google has moved on to the next challenge.

Now organizing means understanding.

Some highlights from the talk for me:

Jeff always gives great talks and this one is no exception. It’s straightforward, interesting, in-depth, and relatively easy to understand. If you are trying to get a handle on Deep Learning or just want to see what Google is up to, then it's a must see.

There’s not a lot of fluff in the talk. It’s packed. So I’m not sure how much value add this article will give you. So if you want to just watch the video I’ll understand.

As often happens with Google talks there’s this feeling you get that we’ve only been invited into the lobby of Willy Wonka’s Chocolate factory. In front of us is a locked door and we're not invited in. What’s beyond that door must be full of wonders. But even Willy Wonka’s lobby is interesting.

So let’s learn what Jeff has to say about the future…it’s fascinating...


What is Meant by Understanding?


A Little History of Deep Neural Nets at Google


What is a Deep Neural Net?


What is Deep Learning?


Some Nice Properties of Neural Nets


Where does Deep Learning Have a Significant Impact at Google?

Speech Recognition

ImageNet Challenge

What are neural net models good at?

Google Photos Search

Street View Imagery

RankBrain in Google Search Ranking

Sequence to Sequence Model

Smart Reply

Image Captioning

Combined Vision + Translation


Turnaround Time and Effect on Research


How Do You Train Large Models Quickly

Model Parallelism


Data Parallelism


The next part of the talk is on TensorFlow, which I won’t cover here. This post is already too long.




Related Articles

Article originally appeared on (
See website for complete article licensing information.