So a related question becomes, can you learn to understand and thus predict physics (the way a child does - I'm not talking about quantum mechanics) from literature only, without interacting in space?
Yes, the child doesn't learn physics he or she learns motor control of its body. Baby learning to talk is more about the brain learning to control its body through throwing its neurological system.
Look at animal kingdom some animals are walking about 5 minutes of being born and running in hours.
Giving the machine a camera and wheels is basically embedding it in the real world. It goes against the spirit of my question, and since I was asking it skeptically it's actually the thing I intuitively (without any relevant expertise!) expect to be required.