I think the key ingredient is to be reinforcement learning, and more importantly, agents being embedded in the external world.

Regarding the "internal world", we already see the development of AI mechanisms for attention, short term memory (references to concepts recently used), episodic memory (autobiographic) and semantic memory (ontologies).