I think that The Peter Principle is a fairly well-recognized, if not universally accepted, phenomena, but up until recently I had no idea that there is a real, well-studied psychological tendency that drives it: the Dunning-Kruger effect!

It's something of a staple in popular culture to represent the most ignorant among us as the most confident (think Dilbert's pointy-haired boss, or heck, just about anyone's boss in any movie) and I think that most of us realize that there is significant value in knowing enough about things to know that you don't know everything. Those who don't know enough to know they should know more have all sorts of confidence, and confidence is often compelling in and of itself. This, in a nutshell, describes Dunning-Kruger. And here I had always thought it was just one of those amusing ironies we only pick up on when true, like Murphy's Law.

This is especially dangerous in a field as complex as information technology, but it's also especially prevalent. The reason is that frequently, non-technical, or at least non-specialist, managers are responsible for promoting technical talent. When you don't know anything about the details of the position you are promoting to, how do you select who to promote to it? Frequently, managers seem to pick the candidate who seems most confident.

I don't know that there is an easy solution to guard against this, although Rob makes some suggestions in his article linked above. I think the best thing may just be to be aware of the tendency and to consider that the most confidence may well mean the least competence, and if all other things are equal, pick the guy that doesn't seem to know what he's doing.