If this article is correct about limitations, couldn't one simply include a Turing machine model into the process to train algorithms?
Some ideas:
- The vectors are Turing tapes, or
- Each point in a tape is a DNN, or
- The "tape" is actually a "tree" each point in the tape
is actually a branch point of a tree with probabilities going each way, and the DNN model can "prune this tree" to refine the set of "spanning trees" / programs.
Or, hehe, maybe I'm leading people off track. I know absolutely nothing about DNN ( except I remember some classes on gradient descent and SVMs from bioinformatics ).
You can bolt all kinds of funny structures into some DNN system, but if the system doesn't have well behaved gradients (or if it isn't even differentiable) it won't train.
Some ideas:
- The vectors are Turing tapes, or
- Each point in a tape is a DNN, or
- The "tape" is actually a "tree" each point in the tape is actually a branch point of a tree with probabilities going each way, and the DNN model can "prune this tree" to refine the set of "spanning trees" / programs.
Or, hehe, maybe I'm leading people off track. I know absolutely nothing about DNN ( except I remember some classes on gradient descent and SVMs from bioinformatics ).