What you don’t know: The role played by errors of omission in imperfect self-assessments

https://doi.org/10.1016/j.jesp.2004.09.006Get rights and content

Abstract

Perceptions of ability often bear little relationship to objective performance. We suggest that people fail to judge their ability more accurately because they have little or no insight into their errors of omission (i.e., solutions they could have generated to problems but missed), although they can be perfectly aware of solutions found. Across five studies with tasks involving, for example, word games and research methodology, we found that participants gave weight to the number of solutions found when making self-evaluations, but not to solutions missed. When given explicit information about these errors of omission, participants gave them just as much weight as they did solutions found, and thus provided more accurate self-evaluations.

Section snippets

Relation to previous work on judgments of skill

A careful reader may wonder about the relation of this analysis to other recent research suggesting that people are at times not in a position to evaluate their skills adequately because they lack the requisite knowledge or information. In particular, Dunning et al., 2003, Kruger and Dunning, 1999 argued that incompetent individuals should not be expected to recognize their poor performances because their deficits present them with two problems. First, their deficits prevent them from reaching

Predictions

Thus, we made five specific predictions about the links between peoples’ self-evaluations, errors of omission, and actual performance.

The Studies

We explored all five predictions by presenting participants with tasks that had multiple solutions, such as a popular word game (Studies 1 and 5), a visual search task (Study 2), a grammar exercise (Study 3), or a methodological critique of scientific research (Study 4). In Studies 1–4 we then asked participants to assess their general ability to perform the task as well as their specific performance. We predicted that participants’ self-evaluations would largely reflect the number of solutions

Study 1: Word play

In Study 1, we wanted to explore people’s self-evaluations on a task that had a large number of solutions that would not be immediately evident to them. However, it was important that the task have a finite set of solutions so that we could test our prediction related to their actual performance. We used the board game Boggle in which one is given a 4 × 4 array of alphabetical letters and asked to find as many words as possible, within a short time limit, by chaining adjacent letters together.

We

Study 2: Visual search

In Study 2, we explored self-evaluations on a visual task that did not have a large number of solutions but that contained solutions that would not be immediately evident to participants. Again, it was important that the task have a finite set of solutions so that we could test our prediction related to their actual performance. We used caricatures from a collection of Al Hirschfeld’s works in The world of Hirschfeld (1970). In order to add something different to his caricatures of celebrities,

Study 3: Grammatical correction

In Study 3, we explored self-evaluations on another task involving errors of omission. It could be argued that the tasks used in both Studies 1 and 2 were somewhat unfamiliar to most participants. In Study 3 we wanted to use a task that participants would definitely have had experience with, finding grammatical errors in text. We took a report from the National Commission on Excellence and Education and inserted grammatical errors throughout the four pages of text. We were careful to only

Study 4: Research methods

In Study 4, we explored self-evaluations on a task that psychologists and researchers encounter every day—searching for methodological flaws in experimental designs. We created four descriptions of hypothetical experiments and inserted methodological errors in each experiment. We then recruited advanced graduate student participants via email to identify and describe the errors. After completing the task, participants were asked about their methodological knowledge and performance at the task.

Meta-analysis: Studies 1–4

The studies so far provided results that were generally supportive of our analysis concerning the role of omission errors in self-evaluations. However, not all critical statistical comparisons are consistent across all studies with our hypotheses. We did uncover a few surprises. Thus, to provide a sharper overall portrait of our results so far, we decided to conduct a meta-analysis to examine which of our predictions had received overall support, and at what level of significance. This analysis

Study 5: Implications for behavior

In Study 5, we examined potential implications for behavior. Participants again had the opportunity to play the popular board game Boggle, and then were asked to bet on the quality of their performance with money given to them by the experimenter. Roughly half the participants were given information about the total number of solutions present in the puzzle prior to betting and the other half were not. Our first question was whether participants not informed of their errors of omission would

General discussion

In mid-2002, soon after the tragic events of September 11, 2001, Secretary of Defense Donald Rumsfeld was asked about the state of the U.S. government’s war on terrorism. As he discussed the range of threats against the U.S., he remarked that:

There are things that we know that we know. There are [also] known unknowns—that is to say, there are things that we now know we don’t know. But there are also unknown unknowns. There are things we do not know we don’t know. So when we do the best we can

The unknown unknowns of everyday life

In this paper, we proposed that similar “unknown unknowns” present an intrinsic difficulty for people as they assess their skills and talents in everyday life. People may be aware of solutions they have found to problems, and might at times have some dim awareness of the solutions they did not identify, but by definition they are not generally aware of their gaps in knowledge, what they “don’t know,” their unknown unknowns, or their errors of omission. We argued that peoples’ evaluations of

Ill-defined problems

All of the studies in this paper showed how the inability to recognize errors of omission led to imperfect performance evaluations, but these studies also may have underestimated the magnitude and pervasiveness of this problem in real life. In real life, the problem is not that people are not aware that they make errors of omission, it is that often such errors are impossible to define and enumerate.

Errors of omission can be impossible to enumerate because of the important distinction between

Familiarity

However, a reader could disagree with our observations about the difficulties that omission errors create for self-evaluation. Perhaps at completely novel tasks—Boggle task, for example—people may not be aware of their errors of omission. Maybe if our tasks were more familiar, and our participants had more experience with them, they would have had a more accurate sense of all the omission errors they were making. Perhaps this is true: Participants’ initial self-evaluations along grammar skills,

Lack of feedback

Although the focus of this paper is on why people are poor at assessing their abilities and performances, we do not mean to imply that people never become better self-evaluators. If lack of awareness concerning omission errors leads to poor self-evaluation, then feedback from others about one’s omissions should improve assessments of ability and performance. There is research outside of social psychology that suggests that this might be the case.

In organizational settings, benchmarking is a

Concluding remarks

In sum, this paper is an exploration into why people make imperfect self-assessments of their performances and abilities. We assert that people in many areas of life cannot be expected to reach accurate views because they face significant informational barriers. We proposed and showed across five studies that people largely possessed little or no magical insight into their errors of omission, even when they admitted that such errors were relevant to their performance, and were motivated to

References (44)

  • Department of Education (1983). An open letter to the American people. A nation at risk: The Imperative for Educational...
  • D. Dunning et al.

    The overconfidence effect in social prediction

    Journal of Personality and Social Psychology

    (1990)
  • D. Dunning et al.

    Why people fail to recognize their own incompetence

    Current Directions in Psychological Science

    (2003)
  • D. Dunning et al.

    Ambiguity and self evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability

    Journal of Personality and Social Psychology

    (1989)
  • D. Dunning et al.

    Depression, realism, and the overconfidence effect: Are the sadder wiser when predicting future actions and events?

    Journal of Personality and Social Psychology

    (1991)
  • N. Epley et al.

    Feeling “holier than thou”: Are self-serving assessments produced by errors in self or social prediction?

    Journal of Personality and Social Psychology

    (2000)
  • N. Falchikov et al.

    Student self assessment in higher education: A meta-analysis

    Review of Educational Research

    (1989)
  • J. Farh et al.

    Effects of comparative performance information on the accuracy of self-ratings and agreement between self- and supervisor ratings

    Journal of Applied Psychology

    (1989)
  • S. Fatsis

    Word freak: Heartbreak, triumph, genius, and obsession in the world of competitive scrabble players

    (2002)
  • D.M. Gordon

    Mentoring urban Black students: Implications for academic achievement, ethnic/racial identity development, racial socialization, and academic disidentification

    Dissertation Abstracts International: Section B: The Sciences and Engineering

    (2000)
  • Green, T. (2002). The advantages of mentoring. Florida Alliance of Virtual Assistants, May, Retrieved on August 28,...
  • J.B. Grossman et al.

    Does mentoring work? An impact study of the Big Brothers Big Sisters Program

    Evaluation Review

    (1998)
  • Cited by (56)

    • ‘It is not fair that you do not know we have problems’: Perceptual distance and the consequences of male leaders' conflict avoidance behaviours

      2018, European Management Journal
      Citation Excerpt :

      People anticipate how one is perceived by others by drawing conclusions about themselves from external cues, including the observations of others (Tice & Wallace, 2003). However, one challenge is that people do not fully know what is unknown about themselves, primarily because of ill-defined problems from ‘unknown unknowns’ (Caputo & Dunning, 2005). Accordingly, some urge leaders to show humility as well as recognise that people can only access selected parts of their own reality (Diddams & Chang, 2012).

    • The Dunning-Kruger Effect in a workplace computing setting

      2017, Computers in Human Behavior
      Citation Excerpt :

      Once participants were made aware of the errors they had missed, they used this information and lowered both their general and task-specific self-evaluation ratings. In studies such as Caputo and Dunning's (2005), all participants had a moderate-to-high level of expertise in the area that was the subject of the evaluation and were, therefore, willing to expand their expertise by adding the missing information. This is not always the case, however, especially for people with little or no expertise in a domain (Carter & Dunning, 2008; Kruger & Dunning, 1999).

    View all citing articles on Scopus

    Many thanks to Dennis Regan, Melissa Ferguson, and the Dunning Lab for assistance with material design. We also thank Amy Tsai, David Chen, and Tarik Scott for all their assistance in data collection. This research was supported financially by National Institute of Mental Health Grant RO1 56072. Portions of this research were presented at the annual meeting of the Society for Personality and Social Psychology, Savannah, January, 2002, and the annual meeting of the Society for Personality and Social Psychology, Austin, 2004.

    View full text