In winter of 2011, Daniel Yamins, postdoctoral researcher in computational neuroscience at the Massachusetts Institute of Technology, sometimes worked after midnight on his machine vision project. He painstakingly designed a system that could recognize objects in images, regardless of variations in size, position, and other properties – which humans do with ease. The system was a deep neural network, a type of computing device inspired by the neurological wiring of living brains.
“I vividly remember when we found a neural network that actually solved the task,” he says. It was 2 a.m., a little too early to wake up his advisor, James DiCarlo, or other colleagues, so an excited Yamins strolled through the cold Cambridge air. “I was really excited,” he says.
This would have counted as a remarkable achievement in artificial intelligence alone, one of many that would make neural networks the darlings of AI technology over the next several years. But that was not the main goal of Yamins and his colleagues. For them and for other neuroscientists, it was a pivotal moment in the development of computer models for brain function.
DiCarlo and Yamins, who now run his own lab at Stanford University, are part of a coterie of neuroscientists using deep neural networks to make sense of the architecture of the brain. In particular, scientists have struggled to understand the reasons for specializations within the brain for various tasks. They wondered not only why different parts of the brain do different things, but also why the differences can be so specific: why, for example, does the brain have an area for recognizing objects in general but also for them. faces in particular? Deep neural networks show that such specializations can be the most effective way to solve problems.
Likewise, researchers have shown that the deep networks most proficient at classifying simulated speech, music, and smells have architectures that appear to parallel the auditory and olfactory systems of the brain. Such parallels also appear in deep networks that can look at a 2D scene and infer the underlying properties of the 3D objects it contains, which helps explain how biological perception can be both fast and incredibly rich. All of these results suggest that the structures of living neural systems embody certain optimal solutions to the tasks they have taken on.
These successes are all the more unexpected in that neuroscientists have long been skeptical of comparisons between brains and deep neural networks, the functioning of which can be impenetrable. “Honestly, no one in my lab was doing anything with deep nets [until recently]MIT neuroscientist Nancy Kanwisher said. “Today, most of them train them regularly.”
Deep nets and vision
Artificial neural networks are built with interconnecting components called perceptrons, which are simplified digital models of biological neurons. Networks have at least two layers of perceptrons, one for the input layer and one for the output. Sandwich one or more “hidden” layers between the inlet and outlet and you get a “deep” neural network; the higher the number of hidden layers, the deeper the network.
Deep nets can be formed to identify patterns in the data, such as patterns representing images of cats or dogs. The training involves using an algorithm to iteratively adjust the strength of the connections between perceptrons, so that the network learns to associate a given input (the pixels of an image) with the correct tag (cat or dog). Once formed, the deep net should ideally be able to rank an entry that it has never seen before.
In their general structure and function, deep networks vaguely aspire to mimic brains, in which the adjusted strengths of connections between neurons reflect learned associations. Neuroscientists have often pointed out important limitations in this comparison: Individual neurons can process information more widely than “dumb” perceptrons, for example, and deep threads frequently rely on a kind of communication between perceptrons called retropropagation which does not appear to occur in nervous systems. Nonetheless, for computational neuroscientists, deep networks have sometimes seemed to be the best option available for modeling parts of the brain.