The Neocognitron, Perhaps the Earliest Multilayered Artificial Neural Network

The Neocognitron, a hierarchical multilayered artificial neural network which acquires the ability to recognize visual patterns through learning, may be one of the earliest examples of what was later called "deep learning." It was invented in 1979 by Kunihiko Fukushima while at NHK Science & Technical Research Laboratories (STRL, NHK放送技術研究所, NHK Hōsō Gijutsu Kenkyūjo), headquartered in Setagaya, Tokyo.  The Neocognitron was used for handwritten character recognition and other pattern recognition tasks.

"The extension of the neocognitron is still continuing. By the introduction of top-down connections and new learning methods, various kinds of neural networks have been developed. When two or more patterns are presented simultaneously, the "Selective Attention Model " can segment and recognize individual patterns in tern by switching its attention. Even if a pattern is partially occluded by other objects, we human beings can often recognize the occluded pattern. An extended neocognitron can now have such human-like ability and can, not only recognize occluded patterns, but also restore them by completing occluded contours" (http://personalpage.flsi.or.jp/fukushima/index-e.html.  accessed 11-10-2014).

K. Fukushima,"Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position," Biological Cybernetics, 36 (1980) 93-202.