«Informant reports: A cheap, fast, and easy method for personality assessment Simine Vazire * Department of Psychology, The University of Texas at ...»
Journal of Research in Personality 40 (2006) 472–481
Informant reports: A cheap, fast, and easy
method for personality assessment
Simine Vazire *
Department of Psychology, The University of Texas at Austin, Austin, TX 78712, USA
Available online 21 November 2005
Despite widespread agreement that multi-method assessments are optimal in personality research,
the literature is dominated by a single method: self-reports. This pattern seems to be based, at least in part, on widely held preconceptions about the costs of non-self-report methods, such as informant methods. Researchers seem to believe that informant methods are: (a) time-consuming, (b) expen- sive, (c) ineﬀective (i.e., informants will not cooperate), and (d) particularly vulnerable to faking or invalid responses. This article evaluates the validity of these preconceptions in light of recent advances in Internet technology, and proposes some strategies for making informant methods more eﬀective. Drawing on data from three separate studies, I demonstrate that, using these strategies, informant reports can be collected with minimal eﬀort and few monetary costs. In addition, infor- mants are generally very willing to cooperate (e.g., response rates of 76–95%) and provide valid data (in terms of strong consensus and self-other agreement). Informant reports represent a mostly untapped resource that researchers can use to improve the validity of personality assessments and to address new questions that cannot be examined with self-reports alone.
Ó 2005 Elsevier Inc. All rights reserved.
Keywords: Methodology; Personality assessment; Peer evaluation; Informant reports; Personality traits
1. Introduction Assessing personality is a challenging task that cannot be accomplished thoroughly with a single method. Yet personality researchers frequently do just that, relying exclusively on * Fax: +1 512 471 5935.
E-mail address: firstname.lastname@example.org 0092-6566/$ - see front matter Ó 2005 Elsevier Inc. All rights reserved.
doi:10.1016/j.jrp.2005.03.003 S. Vazire / Journal of Research in Personality 40 (2006) 472–481 473 self-reports. Indeed, an analysis of all studies published in the Journal of Research in Person- ality (JRP) in 2003 revealed that of the 45 studies in which personality was assessed, 44 of them (98%) used self-reports and for 31 of these (70%) this was the only measure collected.
In contrast, only 24% of the JRP studies collected informant reports (i.e., ratings of the tar- gets by well-acquainted others, such as friends, spouses, or co-workers). The purpose of this paper is to encourage researchers to increase the number of methods they routinely use by adding informant reports to their battery of research instruments.
In addition to collecting informant reports for the sake of multi-method assessment, many researchers have provided compelling arguments for exploiting this rich source of information (e.g., Costa & McCrae, 1988; Craik, 1986, 1996, 2000; Hofstee, 1994; Hogan, 1998; John & Robins, 1993; Kenny, 1994; Kolar, Funder, & Colvin, 1996;McCrae, Stone, Fagan, & Costa, 1998; Oltmanns, Turkheimer, & Strauss, 1998; Paulhus & Morgan, 1997;
Paulhus, 2005; Watson, Hubbard, & Wiese, 2000). In fact, informant reports are an ideal complement to self-reports because self-reports provide a view of personality from the inside (i.e., identity) whereas informant reports provide a view of personality from the outside (i.e., reputation; Hogan, 1998). Perhaps the most important feature of informant reports is that, unlike self-reports, they can be aggregated across observers to obtain a more reliable assessment of personality (Block, 1961; Hofstee, 1994).
In light of these obvious and compelling beneﬁts, Why are informants not more widely used? Given that the beneﬁts are evident, the reasons must lie in the costs, or in researchersÕ preconceptions of the costs. What are the perceived costs of collecting informant reports? It is diﬃcult to address this question empirically, but having spoken to numerous researchers about this issue, it seems that the reluctance to use informant reports is driven by four widely held preconceptions (Table 1). First, researchers seem to believe that collecting informant reports is time consuming. Second, many researchers cannot aﬀord the monetary costs they assume would be associated with collecting informant reports.
Third, researchers anticipate low cooperation on the part of informants. Fourth, researchers worry that because informants frequently complete their ratings outside of the laboratory (e.g., from home), the lack of control over the ratings will adversely aﬀect the quality of the data, perhaps even leading to fake responses. In my experience, all four of these beliefs are unfounded. Informant methods are much easier and more successful than many researchers believe.
After learning how easy the procedure is and how willing informants are to cooperate, many initially skeptical colleagues have incorporated informant reports into their studies.
Therefore, I suspect that if more researchers were convinced that their preconceptions are groundless, many of them would collect informant reports and personality research would be less one-dimensional. This would not only improve the validity of personality research, but also allow researchers to address new questions that cannot be examined with self-reports alone. In this paper I evaluate these preconceptions in light of new technological advances, describe the strategies I have learned for addressing them, and present results from three studies that implemented these strategies.
2. Four preconceptions: Evaluation and strategies
2.1. Preconception 1: Informant reports are time consuming Perhaps the biggest reason for our ﬁeldÕs over-reliance on self-reports is that they are seen as far more convenient than any other method. A corollary of this belief is that any other method, including informant reports, would be a strain on either the researchers or the participants. However, recent technological advances have increased the practicality and eﬃciency of many methods of data collection, including informant reports.
Speciﬁcally, the proliferation of the Internet and e-mail has opened up a new avenue for collecting informant reports. According to a survey conducted in 2003, 54.6% of American households have access to the Internet, and 75% of individuals without Internet access at home still use the Internet at least once a week (U.S. Department of Commerce, 2004).
Researchers can take advantage of this trend by administering informant questionnaires on the Internet. Creating a web questionnaire can be done in under an hour with the help of free websites specializing in survey construction or by using special software or programming guides (e.g., Fraley, 2004). After the web questionnaire has been created, researchers can simply send an e-mail to the informants with a link to the questionnaire and a unique ID number.
This simple procedure saves the researcher a lot of time compared to the traditional postal mailings or the even more time-consuming process of asking informants to come into the laboratory. In addition to eliminating the need for making photocopies and stuﬀing envelopes, Internet questionnaires also save time by eliminating the need for data entry, and making it easy for the researcher to keep track of informantsÕ participation.
Another important beneﬁt of Internet questionnaires is that they require less eﬀort on the part of informants than do traditional methods. Clicking on a link in an e-mail and completing an Internet questionnaire takes less eﬀort than opening a letter, completing a paper-and-pencil questionnaire, and mailing the questionnaire back to the researcher.
Researchers may worry that Internet questionnaires introduce new problems that are not present when using paper-and-pencil questionnaires, but many of these concerns are also unfounded (Gosling, Vazire, Srivastava, & John, 2004). Several researchers have expressed the speciﬁc concern that Institutional Review Boards (IRBs) would require them to obtain written consent from the informants, which cannot be administered over the Internet. In my experience, which has been corroborated by other researchers, IRBs do not require written consent for Internet questionnaires as long as the subject matter is not sensitive or private. Informants can simply give their consent by clicking a button.
Another common concern is that not all potential informants will have e-mail addresses and access to the Internet. Although this seems like a valid concern, in my own research I have rarely encountered this problem, despite having solicited over 700 informants from S. Vazire / Journal of Research in Personality 40 (2006) 472–481 475 participants, including groups of informants potentially less likely to have Internet access (e.g., older adults). For example, in one study I speciﬁcally asked each participant to nominate a parent as an informant. College studentsÕ parents are likely to fall into the age group with the least Internet access (30% of adults 50 years old and over had access to the Internet in 2000, compared with 55% for 25–49-year-olds; U.S. Department of Commerce (2000)). Nevertheless, 65 out of the 80 participants provided a parent as an informant, indicating that at least 81% of participants had a parent with Internet access.
Furthermore, 60 of the parents (92%) completed the ratings. Thus, lack of access to the Internet does not seem to be a major obstacle to collecting informant reports. In fact, we even collected informant reports from one grandmother and one great-grandmother, the so-called silver surfers.
2.2. Preconception 2: Informant reports are expensive to collect
Many personality researchers may be concerned that recruiting informants will cost more money than they can aﬀord. Part of these anticipated costs are associated with the postal mailing, which can be avoided by using the Internet, as described above. The rest of these anticipated costs are associated with compensating the informants. There is a simple and eﬀective solution for this: do not compensate the informants.
Researchers seem to believe that compensating informants will increase response rates.
This may be true, but there are two reasons not to compensate informants, in addition to saving money. First, as I demonstrate below, it is relatively easy to obtain very high response rates without compensating informants. Second, compensating informants introduces an incentive for participants to cheat (i.e., ﬁll out their own informant reports) and for informants to provide non-serious responses simply to get the reward. Thus, I recommend that researchers avoid compensating informants unless they are asking the informants to complete a diﬃcult or time-consuming task. This has the added beneﬁt of eliminating the monetary costs of collecting informant reports.
2.3. Preconception 3: Informants will not cooperate
As I alluded to above, many researchers assume that informants will be unwilling to cooperate unless they are compensated or otherwise given an incentive. As my own results will demonstrate, this skepticism is unfounded. Using a few very simple strategies, I have obtained response rates of 76–95% from informants.
I suspect response rates among informants are high for several reasons. First, completing a rating of someone one knows well is intrinsically interesting—it gives the informant the opportunity to reﬂect on their relationship with that person. Informants may think, often correctly, that they have some special insight into the target they are rating that will help the researcher understand the person better. In fact, I have sometimes gotten followup e-mails from informants who wanted to provide a more detailed description of the person they rated. Furthermore, a typical informant report consisting of 100 items only takes about 10 min to complete, so completing the ratings is not a signiﬁcant burden on informants.
In addition, I use a few simple strategies to boost response rates. These strategies, described in detail in Section 3.1, include taking steps to avoid spam ﬁlters when contactS. Vazire / Journal of Research in Personality 40 (2006) 472–481 ing informants, sending follow-up e-mails to non-responders, and asking participants to remind informants themselves. These common-sense strategies are relatively eﬀortless and are surprisingly eﬀective, suggesting that personality researchers may be overly pessimistic about peopleÕs willingness to cooperate with their demands.
2.4. Preconception 4: Informant reports are not valid (because of dishonesty or faking) There are several ways in which the quality of informant data could be jeopardized.
Surprisingly, the most common concern I have heard from researchers is that participants may fake their own informant reports. To evaluate this concern, we must ask ourselves two questions: Would it be possible for participants to do this? And, are participants motivated to do this?
Using the strategies I have proposed here, it would be very diﬃcult for a participant to fake their own informant reports. Instead of providing real e-mail addresses for each of the two or three informants, participants would have to provide two or three of their own e-mail addresses without it being obvious to researchers that the addresses are their own.
Furthermore, participants have nothing to gain from doing this, unless there is some compensation to the participant that is contingent on the informantsÕ cooperation. For this reason, as well as in fairness to participants, any compensation to the participant should not be contingent on the cooperation of the informants. Hopefully this lays to rest any fears researchers have about participants cheating on their informant reports.