Ignorance More Frequently Begets Confidence Than Does Knowledge*

There's an interesting study (PDF) about how difficulties in recognizing one's own incompetence lead to inflated self-assessments by Justin Kruger and David Dunning. This is known as the Dunning-Kruger effect. The paper is barely 16 pages in length.

Here are the essential conclusions:

1. Incompetent individuals tend to overestimate their own level of skill.
2. Incompetent individuals fail to recognize genuine skill in others.
3. Incompetent individuals fail to recognize the extremity of their inadequacy.
4. If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill.

From the study:

Perhaps more controversial is the third point, the one that is the focus of this article. We argue that when people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it.

...

In short, the same knowledge that underlies the ability to produce correct judgment is also the knowledge that underlies the ability to recognize correct judgment. To lack the former is to be deficient in the latter.

In a self-deprecating concluding paragraph the authors note:

In sum, we present this article as an exploration into why people tend to hold overly optimistic and miscalibrated views about themselves. We propose that those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it. Although we feel we have done a competent job in making a strong case for this analysis, studying it empirically, and drawing out relevant implications, our thesis leaves us with one haunting worry that we cannot vanquish. That worry is that this article may contain faulty logic, methodological errors, or poor communication. Let us assure our readers that to the extent this article is imperfect, it is not a sin we have committed knowingly.

I see this all the time--even in me.

* Charles Darwin