What you don’t know: The role played by errors of omission in imperfect self-assessments☆
Section snippets
Relation to previous work on judgments of skill
A careful reader may wonder about the relation of this analysis to other recent research suggesting that people are at times not in a position to evaluate their skills adequately because they lack the requisite knowledge or information. In particular, Dunning et al., 2003, Kruger and Dunning, 1999 argued that incompetent individuals should not be expected to recognize their poor performances because their deficits present them with two problems. First, their deficits prevent them from reaching
Predictions
Thus, we made five specific predictions about the links between peoples’ self-evaluations, errors of omission, and actual performance.
The Studies
We explored all five predictions by presenting participants with tasks that had multiple solutions, such as a popular word game (Studies 1 and 5), a visual search task (Study 2), a grammar exercise (Study 3), or a methodological critique of scientific research (Study 4). In Studies 1–4 we then asked participants to assess their general ability to perform the task as well as their specific performance. We predicted that participants’ self-evaluations would largely reflect the number of solutions
Study 1: Word play
In Study 1, we wanted to explore people’s self-evaluations on a task that had a large number of solutions that would not be immediately evident to them. However, it was important that the task have a finite set of solutions so that we could test our prediction related to their actual performance. We used the board game Boggle in which one is given a 4 × 4 array of alphabetical letters and asked to find as many words as possible, within a short time limit, by chaining adjacent letters together.
We
Study 2: Visual search
In Study 2, we explored self-evaluations on a visual task that did not have a large number of solutions but that contained solutions that would not be immediately evident to participants. Again, it was important that the task have a finite set of solutions so that we could test our prediction related to their actual performance. We used caricatures from a collection of Al Hirschfeld’s works in The world of Hirschfeld (1970). In order to add something different to his caricatures of celebrities,
Study 3: Grammatical correction
In Study 3, we explored self-evaluations on another task involving errors of omission. It could be argued that the tasks used in both Studies 1 and 2 were somewhat unfamiliar to most participants. In Study 3 we wanted to use a task that participants would definitely have had experience with, finding grammatical errors in text. We took a report from the National Commission on Excellence and Education and inserted grammatical errors throughout the four pages of text. We were careful to only
Study 4: Research methods
In Study 4, we explored self-evaluations on a task that psychologists and researchers encounter every day—searching for methodological flaws in experimental designs. We created four descriptions of hypothetical experiments and inserted methodological errors in each experiment. We then recruited advanced graduate student participants via email to identify and describe the errors. After completing the task, participants were asked about their methodological knowledge and performance at the task.
Meta-analysis: Studies 1–4
The studies so far provided results that were generally supportive of our analysis concerning the role of omission errors in self-evaluations. However, not all critical statistical comparisons are consistent across all studies with our hypotheses. We did uncover a few surprises. Thus, to provide a sharper overall portrait of our results so far, we decided to conduct a meta-analysis to examine which of our predictions had received overall support, and at what level of significance. This analysis
Study 5: Implications for behavior
In Study 5, we examined potential implications for behavior. Participants again had the opportunity to play the popular board game Boggle, and then were asked to bet on the quality of their performance with money given to them by the experimenter. Roughly half the participants were given information about the total number of solutions present in the puzzle prior to betting and the other half were not. Our first question was whether participants not informed of their errors of omission would
General discussion
In mid-2002, soon after the tragic events of September 11, 2001, Secretary of Defense Donald Rumsfeld was asked about the state of the U.S. government’s war on terrorism. As he discussed the range of threats against the U.S., he remarked that:
There are things that we know that we know. There are [also] known unknowns—that is to say, there are things that we now know we don’t know. But there are also unknown unknowns. There are things we do not know we don’t know. So when we do the best we can
The unknown unknowns of everyday life
In this paper, we proposed that similar “unknown unknowns” present an intrinsic difficulty for people as they assess their skills and talents in everyday life. People may be aware of solutions they have found to problems, and might at times have some dim awareness of the solutions they did not identify, but by definition they are not generally aware of their gaps in knowledge, what they “don’t know,” their unknown unknowns, or their errors of omission. We argued that peoples’ evaluations of
Ill-defined problems
All of the studies in this paper showed how the inability to recognize errors of omission led to imperfect performance evaluations, but these studies also may have underestimated the magnitude and pervasiveness of this problem in real life. In real life, the problem is not that people are not aware that they make errors of omission, it is that often such errors are impossible to define and enumerate.
Errors of omission can be impossible to enumerate because of the important distinction between
Familiarity
However, a reader could disagree with our observations about the difficulties that omission errors create for self-evaluation. Perhaps at completely novel tasks—Boggle task, for example—people may not be aware of their errors of omission. Maybe if our tasks were more familiar, and our participants had more experience with them, they would have had a more accurate sense of all the omission errors they were making. Perhaps this is true: Participants’ initial self-evaluations along grammar skills,
Lack of feedback
Although the focus of this paper is on why people are poor at assessing their abilities and performances, we do not mean to imply that people never become better self-evaluators. If lack of awareness concerning omission errors leads to poor self-evaluation, then feedback from others about one’s omissions should improve assessments of ability and performance. There is research outside of social psychology that suggests that this might be the case.
In organizational settings, benchmarking is a
Concluding remarks
In sum, this paper is an exploration into why people make imperfect self-assessments of their performances and abilities. We assert that people in many areas of life cannot be expected to reach accurate views because they face significant informational barriers. We proposed and showed across five studies that people largely possessed little or no magical insight into their errors of omission, even when they admitted that such errors were relevant to their performance, and were motivated to
References (44)
- et al.
Newcomer socialization and stress: Formal peer relationships as a source of support
Journal of Vocational Behavior
(1999) - et al.
Do those who know more also know more about how much they know?
Organizational Behavior and Human Decision Processes
(1977) The relation of a work-based mentoring program to the academic performance and behavior of African American students
Journal of Vocational Behavior
(2001)The structure of ill structured problems
Artificial Intelligence
(1973)Global self-evaluation as determined by the desirability and controllability of trait adjectives
Journal of Personality and Social Psychology
(1985)Relations between metacognition and instruction: Development of metacognitive awareness via instruction of reflective research skills in an academic context for middle school students
Dissertation Abstracts International Section A: Humanities and Social Sciences
(1996)- et al.
General practitioner training in palliative care: An experiential approach
Journal of Palliative Care
(2000) - Boisvert, H. (2001). Using benchmarking to strategically deploy the finance function profiling. CMA Management, May,...
Understanding your industry inside and out
Pulp and Paper
(2002)- College Board (1976–1977). Student descriptive questionnaire. Princeton, NJ: Educational Testing...
The overconfidence effect in social prediction
Journal of Personality and Social Psychology
Why people fail to recognize their own incompetence
Current Directions in Psychological Science
Ambiguity and self evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability
Journal of Personality and Social Psychology
Depression, realism, and the overconfidence effect: Are the sadder wiser when predicting future actions and events?
Journal of Personality and Social Psychology
Feeling “holier than thou”: Are self-serving assessments produced by errors in self or social prediction?
Journal of Personality and Social Psychology
Student self assessment in higher education: A meta-analysis
Review of Educational Research
Effects of comparative performance information on the accuracy of self-ratings and agreement between self- and supervisor ratings
Journal of Applied Psychology
Word freak: Heartbreak, triumph, genius, and obsession in the world of competitive scrabble players
Mentoring urban Black students: Implications for academic achievement, ethnic/racial identity development, racial socialization, and academic disidentification
Dissertation Abstracts International: Section B: The Sciences and Engineering
Does mentoring work? An impact study of the Big Brothers Big Sisters Program
Evaluation Review
Cited by (56)
Differences is students' performance, (mis)calibration, and confidence judgments depending on their preferred sitting position in class
2024, Personality and Individual DifferencesThe psychology of esports players’ ELO Hell: Motivated bias in League of Legends and its impact on players’ overestimation of skill
2023, Computers in Human Behavior‘It is not fair that you do not know we have problems’: Perceptual distance and the consequences of male leaders' conflict avoidance behaviours
2018, European Management JournalCitation Excerpt :People anticipate how one is perceived by others by drawing conclusions about themselves from external cues, including the observations of others (Tice & Wallace, 2003). However, one challenge is that people do not fully know what is unknown about themselves, primarily because of ill-defined problems from ‘unknown unknowns’ (Caputo & Dunning, 2005). Accordingly, some urge leaders to show humility as well as recognise that people can only access selected parts of their own reality (Diddams & Chang, 2012).
The Dunning-Kruger Effect in a workplace computing setting
2017, Computers in Human BehaviorCitation Excerpt :Once participants were made aware of the errors they had missed, they used this information and lowered both their general and task-specific self-evaluation ratings. In studies such as Caputo and Dunning's (2005), all participants had a moderate-to-high level of expertise in the area that was the subject of the evaluation and were, therefore, willing to expand their expertise by adding the missing information. This is not always the case, however, especially for people with little or no expertise in a domain (Carter & Dunning, 2008; Kruger & Dunning, 1999).
Motivation and Feelings of Competence Among University Students in Introductory Physics
2023, Research in Science EducationAn examination of the effect of feedback on meta-ignorance of mental illness public stigma
2023, Metacognition and Learning
- ☆
Many thanks to Dennis Regan, Melissa Ferguson, and the Dunning Lab for assistance with material design. We also thank Amy Tsai, David Chen, and Tarik Scott for all their assistance in data collection. This research was supported financially by National Institute of Mental Health Grant RO1 56072. Portions of this research were presented at the annual meeting of the Society for Personality and Social Psychology, Savannah, January, 2002, and the annual meeting of the Society for Personality and Social Psychology, Austin, 2004.