Hacker News new | past | comments | ask | show | jobs | submit login

LLMs can count words (and letters) just fine if you train them to do so.

Consider the fact that GPT-4 can generate valid XML (meaning balanced tags, quotes etc) in base64-encoded form. Without CoT, just direct output.




That's GPT-4, which you wouldn't use for in-line suggestions because it's too slow.

I don't know what model Copilot uses these days, but it constantly makes bracket mistakes in Python.


You don't need a GPT-4-sized model to count brackets. You just need to make sure that your training data includes enough cases like that for NN to learn it. My point is that GPT-4 can do much more complicated things than that, so there's nothing specific about LMs that preclude them from doing this kind of stuff right.




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: