Even more disconcerting, it can sometimes be very hard to tell the competent from the incompetent until you have them in the door working for you.
I once worked with a guy (not in an engineering discipline) who could talk circles around much more competent people. He knew all the buzz words, and even knew most of the topics about one level deep. When he spoke, he spoke with an absolute air of authority on whatever subject he was talking about. He managed to convince division manager after division manager (he jumped around a lot and it was a very large company) that past failures were just because of circumstances out of his control.
However, he didn't actually know anything and every project he was put in charge of either failed disastrously, or was caught from the brink at the last minute by colleagues who worked double shifts to do the project (which of course allowed him to point to those same colleagues as incompetents that he had to deal with on his project and hence the reason for failure).
The really sad thing is, I could never figure out if he knew he was grossly incompetent and just spinning things to keep a job far above the level he should have had, or actually believed the yarns he wove about what he knew.
I'm curious: is this not solved by a solid technical interview?
I would have guessed that if someone impresses HR, you could find out if they really know their stuff by getting a KNOWN competent person to ask them lots of hard technical questions, make them solve problems on the spot, etc.
A friend recently gave me a great example of how to do open-ended questions and probe someone's knowledge: ask a question like "when you type in a web address and hit enter, what happens?" The answer can range from "your computer loads the page" to delving into DNS, IP routing, HTTP headers, server load management, server-side processing, sessions, database access, cookies, browser rendering, etc etc. If the candidate gives a simple answer, you can continue to probe different areas to see how much they know in each one.
The trick is that the interviewer has to be competent enough to tell what the candidate knows and whether he/she will BS rather than admit not knowing something (which is also important to find out).
If you don't already have anyone competent enough to conduct the interview - maybe you're hiring your first programmer - you could temporarily hire someone who comes highly recommended to help you with the interview process.
So, was this guy vetted like this on technical merits, or did he just talk impressively to non-technical people? I'm curious to know how he slipped through.
It wasn't exactly a technical field. It's amazing how many fields outside of development are very nearly impossible to test in a meaningful way. For example, hiring a sales vp, if done at the same level of problem solving as most tech interviews I've seen, would have to result in a 30% sales increase for the company. You basically have to just see what they say, if they sound like they can tie their own shoes, don't suffer from short term memory loss and have a pulse you usually give them a try and just fire them if they don't work out.
That's why there are so many hangups during the interview process that have nothing to do with a person's actual skill in a given area..like font choice on a resume, or little tips like how to sit in a chair or whatever. Those are all little tells that people look for in an interview, but candidates also know this, so they try to game the system by appearing confident, showing up precisely 15 minutes early with a binder full of extra copies of their resume, just the right amount of hair gel, etc. Because by and large, in fields outside of technical ones, that's how you get a job.
> So, was this guy vetted like this on technical merits, or did he just talk impressively to non-technical people? I'm curious to know how he slipped through.
But yeah, this is actually how he kept slipping through. His interviewers were hiring him for expertise in his field that they themselves did not have. Pretty much it was bureaucrats and administrators hiring somebody for a particular skill requirement. They simply didn't know how to dig more than a level or two deep. In the same way that a person can game the hiring system like I mentioned above, he could do the same for his field. Somebody who was actually in the field would have to spend a little time to figure out that he didn't know the equivalent that computers need electricity to operate. But he could talk for hours about how for example, a browser operates with DNS servers to resolve IP addresses -- and do it with authority (again, as a very rough equivalent).
You could even ask him to produce a plan of action for the next six months and he could produce a pretty good outline, good enough for an interview, but ask him to fill in the details and he couldn't.
> His interviewers were hiring him for expertise in his field that they themselves did not have. Pretty much it was bureaucrats and administrators hiring somebody for a particular skill requirement.
This is what keeps me coming back to small companies and startups. Any company with an HR department big enough that they hire using a series of bullet points pretty much guarantees I will have brain-dead coworkers.
1. He noted that this wasn't in an engineering field (which I would presume to include programming :) ).
2. A technical interview is hardly a panacea against bad hiring decisions. It's just the least crappy way to hire people that anyone's found (that I know of).
This happens often in music, where subjectivity comes into play. What's not immediately obvious: programmers use subjective measures of quality all the time. We have to do so to get our work done in a timely fashion.
Those who want to be truly great should be humble. They should also check themselves as much as possible with objective measures and feedback from trusted and respected fellow practitioners.
Come to think of it, I suspect this phenomenon affects the implementation of programming languages. A lot of second tier CS talents implement languages because they have enough ego to get through the ordeal.
Yep it's whether you are an active information-seeker or not: If you never question your current model of thought it is a lot harder to learn, you'll mostly learn things that match what you know and the ideas and perspectives you can apply to a problem will be fairly limited.
Contrast with a competent person, that have a knack for picking up new stuff, they are used to learning stuff so they know that they don't know everything.
You are right, I should not have put the "why" there (and can't change it anymore). If someone with editing abilities sees this, could you please change the title to "The Dunning–Kruger effect", or what you think is more appropriate. Thanks.
The skill you're looking for here is called meta-knowledge. It is knowledge of your level of knowledge about a given topic.
It is a teachable trait, arguably what very selective schools with hard grading curves teaching difficult things teach people to develop (as you have to determine what you don't know to pass/excel/etc there).
I've always personally believed the test: "What would you need to get going on these 5 projects?" Is a great question. If talking about researching X Y or Z don't come out of the candidate's mouth (and it's not an old solved problem, like self-contained embedded C code), you mark them down as less meta-knowledgeable. (Doesn't mean unhireable, but you don't want them in highly independent positions or constantly learning new things; people with low meta knowledge appear to actually be quite happy in places that high meta people hate, such as long term maintenance programming).
In software particularly, study of estimation (and it's continual use) can teach people the practice of evaluating all risks, including metaknowledge. I like the book: http://www.amazon.com/Software-Estimation-Demystifying-Pract... (non-aff link) for getting people going with the practice.
“There are, in effect, two things: to know and to believe one knows. To know is science. To believe one knows is ignorance.”
"Their misconception is fueled by the fact that they are completely unaware of their own ignorance. This unawareness or unwillingness to admit their ignorance subconsciously prods them to make (wrong) assumptions in order to fill the void in their knowledge."
"Ignorance more frequently begets confidence than does knowledge."
"What we do not understand we do not possess."
"We're even wrong about which mistakes we're making."
I once worked with a guy (not in an engineering discipline) who could talk circles around much more competent people. He knew all the buzz words, and even knew most of the topics about one level deep. When he spoke, he spoke with an absolute air of authority on whatever subject he was talking about. He managed to convince division manager after division manager (he jumped around a lot and it was a very large company) that past failures were just because of circumstances out of his control.
However, he didn't actually know anything and every project he was put in charge of either failed disastrously, or was caught from the brink at the last minute by colleagues who worked double shifts to do the project (which of course allowed him to point to those same colleagues as incompetents that he had to deal with on his project and hence the reason for failure).
The really sad thing is, I could never figure out if he knew he was grossly incompetent and just spinning things to keep a job far above the level he should have had, or actually believed the yarns he wove about what he knew.