> it’s like something out of a sci fi movie where humans encounter an alien species that thinks in a way that humans cannot even comprehend due to biological limitations.
I've increasingly felt this since GPT2 wrote that news piece about unicorns back in 2019. These models are still so mysterious, when you think about it. They can often solve decently complex math problems, but routinely fail at counting. Many have learned surprising skills like chess, but only when prompted in very specific ways. Their emergent abilities constantly surprise us and we have no idea how they really work internally.
So the idea that they reason using something other than human language feels unsurprising, but only because everything about it is surprising.
I've increasingly felt this since GPT2 wrote that news piece about unicorns back in 2019. These models are still so mysterious, when you think about it. They can often solve decently complex math problems, but routinely fail at counting. Many have learned surprising skills like chess, but only when prompted in very specific ways. Their emergent abilities constantly surprise us and we have no idea how they really work internally.
So the idea that they reason using something other than human language feels unsurprising, but only because everything about it is surprising.