There would seem to be at least two independent problems here:
1. How can you recognize that someone is actually a better programmer than you are, as opposed to someone who can bullshit beyond your ability to detect it? (It may be easier to detect that someone is a worse programmer than you are.)
2. Once you've found someone who you're convinced is a better programmer, how do you convince them to join a company where they'd be the most skilled programmer (and thus have nobody to learn from)? You might need to offer them extra compensation and a better job title than they'd be able to get elsewhere.
Also, I'd question whether you can actually rank programmers on a linear scale of better/worse. The best programmer I ever hired was better than me in some ways (could build complex systems faster than I could) but worse than me in other ways (the systems he built were less maintainable).
I don't really have good answers to these questions, but hopefully this might clarify some of the issues and help get the conversation going.
> 1. How can you recognize that someone is actually a better programmer than you are, as opposed to someone who can bullshit beyond your ability to detect it? (It may be easier to detect that someone is a worse programmer than you are.)
This question comes up a lot, and I'm not sure that it's really as hard of a problem as it's often made out to be. Especially if you're having a candidate write algorithmic code on a whiteboard, I think there are ways that you could judge whether they're better than you:
1. They solve your problem faster than you did (the first time). Of course this assumes you have a novel problem (i.e. they haven't practiced it before), etc.
2. They solve your problem and in the process of doing so teach you something unique. Maybe their solution is faster or simpler than yours. Maybe they catch some corner case you didn't see.
Another way to look at this is that if you make up your own (difficult) interview question, and then use it repeatedly, you will eventually become an expert on it. As you watch other people solve it, you'll learn various tricks for solving it, and will see it from many different angles. You'll also become aware of common pitfalls.
So when someone walks in the door and has never seen your problem before in their life, you have them at a disadvantage. Now if they give a solution that's as good or better than yours, what does that mean? It could just be that they're solved similar problems before, but maybe they're better than you.
This approach does assume that you are good enough to create a novel question, which in and of itself is a difficult thing to do...
[EDIT] One thing I left out is that while I believe it's possible to say "this candidate is probably better than me," there's an upper bound on your ability to see exactly how much better they are. In other words, it might be difficult to tell the difference between someone moderately better than you and someone vastly better than you.
"This question comes up a lot, and I'm not sure that it's really as hard of a problem as it's often made out to be."
The speed or novelty of a solution to an interviewers pet problem provides no insight at all into how the candidate will perform on the job.
At my last company we would re-interview a sample of our staff using the same interview questions we were going to ask externals in order to gauge the quality of the questions. Pet problems provided the least value out of all the interview techniques we tried.
"In other words, it might be difficult to tell the difference between someone moderately better than you and someone vastly better than you."
It is very difficult to determine if somebody is better, and how much better they are, than you when you are close in skill level.
On the other hand if somebody is 5x better than you it is easy to determine. You are right though in that it is impossible to tell the difference between somebody who is 5x better than you and 10x better.
Yeah, thanks for clearing that up for me. Obviously, you can't rank 'programmers' on a linear scale, but you can rank programmers that are suitable for your type of work on a linear scale. Also, all of the best programmers have something in common: they're all very intelligent people.
Question #1 is somewhat out of the scope of this question tho, but I'm really interested in seeing thoughts on question #2.
Another problem I wrestle with is how to hire someone with skills way outside your own skillset and outside anyone else's skillset in the company. For instance I'm in a situation where I can tell the people that have set up the infrastructure are not capable of doing it right because the infrastructure stinks by any outside measure. But without knowledge of how to do it myself, how am I supposed to hire anyone better?
a) Gain enough knowledge to make a more informed hiring decision. You don't typically need as much knowledge as you think.
b) Find someone who is knowledgable about the role and ask for help. If you know a company with a great infrastructure person, it is very likely that he would be willing to help filter out a few candidates for your company. At the very least he could explain what he does on an average day and what he looks for in a candidate.
Common, dramatized problem. Hiring can be only occasionally perfect.
"Will burns the proof. Lambeau tries to save it.
LAMBEAU: You're right, Will. I can't do this proof. But you can, and when it comes to that it's only about ... it's just a handful of people in the world who can tell the difference between you and me. But I'm one of them.
WILL: Sorry.
LAMBEAU: Yeah, so am I. Most days I wish I never met you. Because then I could sleep at night, and I wouldn't have to walk around with the knowledge that there's someone like you out there ... . And I didn't have to watch you throw it all away ..."
1. How can you recognize that someone is actually a better programmer than you are, as opposed to someone who can bullshit beyond your ability to detect it? (It may be easier to detect that someone is a worse programmer than you are.)
2. Once you've found someone who you're convinced is a better programmer, how do you convince them to join a company where they'd be the most skilled programmer (and thus have nobody to learn from)? You might need to offer them extra compensation and a better job title than they'd be able to get elsewhere.
Also, I'd question whether you can actually rank programmers on a linear scale of better/worse. The best programmer I ever hired was better than me in some ways (could build complex systems faster than I could) but worse than me in other ways (the systems he built were less maintainable).
I don't really have good answers to these questions, but hopefully this might clarify some of the issues and help get the conversation going.