> This is also a chess puzzle marked as very hard that a person who is good at chess should give themselves fifteen minutes to solve. The author of the chess.com blog containing this puzzle only solved about half of them!
I am human and I solved this before opening the blog post, because I've seen this problem 100 times before with this exact description. I don't understand why an LLM wouldn't have done the same, because pattern matching off things you saw on the internet is IIUC the main way LLMs work.
(I am good at chess, but not world class. This is not a difficult mate in 2 problem: if I hadn't seen it, it would take a minute or so to solve, some composed 2-movers might take me 5 minutes).
I just tried ChatGPT free with the prompt "There's a mate-in-two composed by Paul Morphy. What's the key move?". It searches and finds it immediately. But if I ask it not to search the internet, its response is incoherent (syntactically valid English and knows the names of the chess pieces, but otherwise hallucinated).
I am human and I solved this before opening the blog post, because I've seen this problem 100 times before with this exact description. I don't understand why an LLM wouldn't have done the same, because pattern matching off things you saw on the internet is IIUC the main way LLMs work.
(I am good at chess, but not world class. This is not a difficult mate in 2 problem: if I hadn't seen it, it would take a minute or so to solve, some composed 2-movers might take me 5 minutes).