I think this is conflating the stored program design and the Von Neumann machine. The storage of both programs and memory in the same address space was novel, but other options for storing programs exist (see especially Harvard Architecture). You can even make a case that "stored program" as an idea predates electronics and computers. Both Leibniz and Babbage explored the idea of a programmable computing device, and the Jacquard loom used ordered sets of cards that most people would recognize as a program.
Babbage didn't want to put the program and data in the same storage device. The program was a chain of cards; the data store was rows of number wheels mounted on a big cylinder for addressing purposes. Probably the closest production machine to Babbage's machine was the IBM Card Programmed Calculator, which we would not consider a computer today.
All thinking about this issue has to face the fact that there were no good memory elements before magnetic core. Delay lines were slow and sequential, drums were slower and sequential, and Williams tubes were faster, random access, but expensive per bit. Core memory wasn't cheap; it was about a million dollars a megabyte in 1970. "Just put the program in main memory" wasn't a good option until halfway decent memory hardware was developed.
You can even make a case that "stored program" as an idea predates electronics and computers.
You could but then you'd be conflating something on the other end of the spectrum - turning 'programmable' into 'stored program'. There is no sensible way in which a Jacquard loom is a 'stored program' device.
That's a fair point --- I'm certainly stretching to try to extend 'configurable' into 'stored programs'. I do still think that the shift to storing instructions in electronic memory (once said memories became available) was less revolutionary than evolutionary. Von Neumann's innovation (or at least the idea he was first to systematically describe) of treating instructions and data as the same type of thing was probably a little more of a leap, but I think all the implications of that weren't realized until later.
