Heh, I still can't figure out what an HTM is exactly, but I think it's sort of similar to a recurrent convolutional network. It handles learning completely differently (allegedly in a more biologically plausible way).
Personally, I think there are two big issues with what numenta was doing: they diverged too far from mainline neural net research (which, when they started in ~2005 was just before things started getting interesting--it was also a time when "mainline neural net research" was widely assumed to be at a dead end) and they tried too hard to come up with something that was biologically plausible rather than mathematically expedient. Sort of how airplanes need wings but don't need feathers. As far as I am aware, HTMs really just don't work well in practice (and by that, I mean they are not anywhere near competitive with any of the architectures I listed above).
From what I remember, I think HTM learning is isomorphic to that of a recurrent conv net. You'll get roughly the same change given the same training data, but with a parallel-agent approach to learning, instead of the kind of monolithic computation you can throw a GPU at. (In other words, you flip the "data" and "instruction" streams from SIMD to get Multiple-Instruction-Same-Data processes.)
You could, in other words, see a recurrent conv net as an opmitization of an HTM given a von Neumann architecture, or the reverse -- an HTM as an optimization of a recurrent conv net given a biological substrate (where it's much less costly to build tons of crummy processors and link them into an arbitrary graph, than it is to build a single fast processor.)
Again, though, I'm not an ML person, so I might be way off.
2006 is commonly cited as the year when "deep learning" started becoming practical. Pretty sure it was Hinton's group, but they used greedy unsupervised pre-training to get a good initialization of the weights, then followed by supervised finetuning of said weights. That result kicked off a lot of renewed interest in NNs, which then led to using GPUs for a 40x speedup, which then led to many more impressive results (and they just keep coming). It turns out the unsupervised pre-training isn't even necessary, go figure...
Personally, I think there are two big issues with what numenta was doing: they diverged too far from mainline neural net research (which, when they started in ~2005 was just before things started getting interesting--it was also a time when "mainline neural net research" was widely assumed to be at a dead end) and they tried too hard to come up with something that was biologically plausible rather than mathematically expedient. Sort of how airplanes need wings but don't need feathers. As far as I am aware, HTMs really just don't work well in practice (and by that, I mean they are not anywhere near competitive with any of the architectures I listed above).