I really enjoyed this article. It's the first attempt I've seen to assess deep learning toward the integrated end of human level cognition or AGI.
I found one point especially noteworthy:
"
So even though a deep learning model can be interpreted as a kind of program, inversely most programs cannot be expressed as deep learning models—for most tasks, either there exists no corresponding practically-sized deep neural network that solves the task, or even if there exists one, it may not be learnable, i.e. the corresponding geometric transform may be far too complex, or there may not be appropriate data available to learn it.
Scaling up current deep learning techniques by stacking more layers and using more training data can only superficially palliate some of these issues. It will not solve the more fundamental problem that deep learning models are very limited in what they can represent, and that most of the programs that one may wish to learn cannot be expressed as a continuous geometric morphing of a data manifold.
"
What he seems to be suggesting is that a human level cognition built from deep nets will not be a single unified end-to-end "mind" but a conglomeration of many nets, each with different roles, i.e., a confederation or "society" of deep nets.
I suspect Minsky would have agreed, and then suggested that the interesting part is how one defines, instantiates, and then interconnects the components of this society.
I found one point especially noteworthy: " So even though a deep learning model can be interpreted as a kind of program, inversely most programs cannot be expressed as deep learning models—for most tasks, either there exists no corresponding practically-sized deep neural network that solves the task, or even if there exists one, it may not be learnable, i.e. the corresponding geometric transform may be far too complex, or there may not be appropriate data available to learn it.
Scaling up current deep learning techniques by stacking more layers and using more training data can only superficially palliate some of these issues. It will not solve the more fundamental problem that deep learning models are very limited in what they can represent, and that most of the programs that one may wish to learn cannot be expressed as a continuous geometric morphing of a data manifold. "
What he seems to be suggesting is that a human level cognition built from deep nets will not be a single unified end-to-end "mind" but a conglomeration of many nets, each with different roles, i.e., a confederation or "society" of deep nets.
I suspect Minsky would have agreed, and then suggested that the interesting part is how one defines, instantiates, and then interconnects the components of this society.