I don't mean to be too negative here, but this is hardly a new development, so can someone clarify the novelty in this paper? Neural nets have been extensively demonstrated in memristor-based architectures [1] and several memristor-based training architectures have previously been proposed and tested [2]. The abstract's claim that "no model for such a network has been proposed so far" is prima facie blatantly false.
In any case, I have yet to see a conclusive, publicly explained solution to the significant system-level problems with memristor-based neural architectures, or indeed any analog neural architecture. The best claimed digital architectures are around ~250 fJ per multiply-and-accumulate (MAC) [Groq], and these generally involve 8-bit multiplication, which is extremely expensive in the analog ___domain thanks to the exponential scaling of power with precision levels. Even if you set aside the monstrous fabrication and device-level variance issues with memristors, DAC and ADC consume tens of pJ per sample in the realistic IP blocks that are commercially available. Although only one pair of DAC and ADC operations is required per dot product, this is still 40 fJ per MAC from DAC and ADC alone, assuming a 256x256 matrix multiplication and not taking other system-level issues into account. This limits memristors to a 5x over current digital architectures, and as nodes shrink, by the time memristors come out, this will be around a 3x. While a 3x is considerable, I don't think it justifies the moonshot-level deep tech risk that memristors will continue to represent. Many hardware companies [Tabula...] have failed attempting to reach something like a 3x in the main figure-of-merit, only to find that system-level issues get them a 1x instead. Besides, I'm sure digital architectures have more than 3x room for improvement- plenty of tricks left for digital!
I'm hoping for a breakthrough, because I am fundamentally an optimist, but memristors have been failing to deliver since 2008.
There's a pattern common to many unconventional storage and computational technologies: they stay behind the state of the art in the mainstream competitor, keeping up with it for a while but never catching up. Things will probably change if silicon stops improving.
You can go to Groq.com -- that startup claims to have 125fj per flop (and each mac is two flops thanks to marketing logic). Started by 8 out of 10 founding TPU team members.
In any case, I have yet to see a conclusive, publicly explained solution to the significant system-level problems with memristor-based neural architectures, or indeed any analog neural architecture. The best claimed digital architectures are around ~250 fJ per multiply-and-accumulate (MAC) [Groq], and these generally involve 8-bit multiplication, which is extremely expensive in the analog ___domain thanks to the exponential scaling of power with precision levels. Even if you set aside the monstrous fabrication and device-level variance issues with memristors, DAC and ADC consume tens of pJ per sample in the realistic IP blocks that are commercially available. Although only one pair of DAC and ADC operations is required per dot product, this is still 40 fJ per MAC from DAC and ADC alone, assuming a 256x256 matrix multiplication and not taking other system-level issues into account. This limits memristors to a 5x over current digital architectures, and as nodes shrink, by the time memristors come out, this will be around a 3x. While a 3x is considerable, I don't think it justifies the moonshot-level deep tech risk that memristors will continue to represent. Many hardware companies [Tabula...] have failed attempting to reach something like a 3x in the main figure-of-merit, only to find that system-level issues get them a 1x instead. Besides, I'm sure digital architectures have more than 3x room for improvement- plenty of tricks left for digital!
I'm hoping for a breakthrough, because I am fundamentally an optimist, but memristors have been failing to deliver since 2008.
[1] http://www.cs.utah.edu/~rajeev/pubs/isca16-old.pdf [2] https://ieeexplore.ieee.org/document/7010034/