Fascinating stuff! I'm not familiar with these new reasoning based models, I wonder how much of the reasoning ability comes from the weights vs the architecture of the model itself vs the system prompt.
Both “Principle 1” and “Principle 2” in the article are essentially LLM-focused details of basic principles in ML that have been known since before I (and probably you, if you’re still working age) were born.
I thought the same thing. It’s usually content that’s well outside my areas of familiarity, often even outside my areas of interest. But I usually find his writing interesting enough to read through anyway, and clear enough that I can usually follow it even without familiarity with the subject matter.
I had the same thought too. I wonder if this his role at Microsoft now? Kind of a human institutional knowledge repository, plus a kind of brand ambassador to the developer community, plus mentor to younger engineers, plus chronicler.
I hope he keeps going, no doubt he could choose to finish up whenever he wants to.
I actually started a short blog series about a similar problem where a friend had blown away /bin and a bunch of other stuff, but/lib was still there. Unfortunately it didn't end up getting anywhere because even though I was able to drop executables on the machine with echo and make them executable with a .so from lib I wasn't able to get back to root permissions as sudo and everything had been blown away and I didn't think I'd have great luck trying to find a zero-day in the kernel. It was still a lot of fun though.
See also: the undergrad who found a breakthrough in hash tables recently and wasn't put off by a long-standing conjecture about what the bound on their performance was because he simply wasn't aware of it
I just started following a show on twitch called the litigation disaster tourism hour which is a lawyer commenting on various cases, and currently covering the WordPress stuff. Seems like it might be up his alley if somebody asks in his chat, though I generally can't catch it live to do that because of time zones
reply