Deep Learning Will Accelerate Self-Driving Cars

By Danny Shapiro, NVIDIA

Punch Buggy. Slug Bug. The names differ, but we’ve all played this game. See a Volkswagen Beetle, punch your sibling. Deep learning works the same way. Just with more math — and fewer bruises.

Deep learning refers to algorithms — step-by-step data-crunching recipes — for teaching machines to understand “unstructured data.” That’s another way of saying information that lives outside of spreadsheets and databases. For example: images, speech and video.

So, how does deep learning work? A great way to understand it is to look at NVIDIA DRIVE, a new autopilot car computer. When paired with computer vision technology — powered by NVIDIA Tegra processors — DRIVE gives vehicles a level of self-awareness. To show what DRIVE can do, NVIDIA engineers stuck video cameras onto their cars to capture 40 hours of video. They then used Amazon Mechanical Turk to have frames manually tagged by people to categorize about 68,000 objects in the footage.

Training Artificial Brains

Our engineers then fed these images to servers equipped with GPUs (graphics processing units) that form an artificial neural network. It’s a process computer scientists call “training.” It lets a neural network learn to see patterns and recognize objects.

Training artificial neural networks is a little like how children learn. Parents, friends and punch-happy older siblings identify objects in the world for the child. The child’s brain then learns how to identify these objects in a broad array of situations.

This is where the “deep” in deep learning comes in. With deep learning, a neural network learns many levels of abstraction. They range from simple concepts to complex ones. Each layer categorizes some kind of information. The neural network then refines it and passes it along to the next.

Deep learning stacks these layers. This lets a machine learn what computer scientists call a “hierarchical representation.” So, the first layer might look for edges. The next layer looks for collections of edges that form angles. The next might look for patterns of edges. After many layers the neural network learns the concept of, say, a pedestrian crossing the street.

GPUs are great for neural nets, which sort unstructured data like images. Once a system is “trained,” that learning can be used in applications like self-driving cars.

Precision Vision

Deep learning is a technology that’s arriving just in time to advance computer vision. Low-cost cameras and sensors are giving cars the ability to suck in huge amounts of information. Computer vision technology turns that data into 3D maps that vehicles can use to navigate the world around them.

Deep learning takes those automated navigation capabilities to another level. The NVIDIA DRIVE system takes advantage of the models that neural networks create. It lets the car understand the world the way human drivers do.

As a result of this understanding, DRIVE can tease out important driving information fast. It can pick out different kinds of vehicles. It can discern a police car from a taxi; an ambulance from a delivery truck; or a parked car from a car that is about to pull out into traffic. That capability isn’t limited to vehicles. NVIDIA DRIVE can identify everything from cyclists on the sidewalk to absent-minded pedestrians.

Deep learning can even categorize images that challenge human eyes. Even in bad weather, DRIVE can read flickering electronic signs, or spot brake lights … or recognize a Volkswagen Beetle.

Trust us: Never play Punch Buggy with a GPU.

Danny Shapiro is NVIDIA’s senior director of automotive, focusing on solutions that enable faster and better design of automobiles, as well as in-vehicle solutions for infotainment, navigation and driver assistance. A version of this article originally appeared on blogs.nvidia.com.

More Info:

• NVIDIA