Implicit Association Testing

Description

From the Free Speech Union:

"What is the Implicit Association Test?

The Implicit Association Test (IAT) is a psychological tool that claims to measure ‘implicit bias’. The test was introduced by psychologists Anthony Greenwald, Mahzarin Banaji, and Brian Nosek in 1998.1 The race IAT is the best known and the one described here. It claims to measure implicit bias, or unconscious prejudice, between racial groups.

What is the significance of the Implicit Association Test?

At the launch of the test, the creators of the IAT stated that they had discovered that 90-95% of IAT participants show an ‘implicit preference’ for white over black faces, including a significant number of black participants.2\ In the terms of the test, this means that they are faster to associate good words with white faces compared to black faces, and slower to associate bad words with white faces compared to black faces. Individuals who hold egalitarian beliefs about race are frequently shocked that the test appears to reveal racial prejudices they were not conscious of. The creators of the IAT have argued that the results explain the persistence of racial disparities in a society where explicitly racist attitudes and behaviours have declined. The apparent preference for white faces over black is taken as evidence that racist attitudes are being perpetuated unconsciously, even by well-meaning people who regard themselves as supporters of racial equality.3 It was initially stated that the IAT could predict discriminatory behaviour at the individual level, suggesting that people with a high IAT score were more likely to behave in racially prejudiced ways.

Mahzarin Banaji and Anthony Greenwald stated that:

‘[T]he automatic White preference expressed on the Race IAT is now established as signalling discriminatory behavior. It predicts discriminatory behavior even among research participants who earnestly (and, we believe, honestly) espouse egalitarian beliefs.’4 The test’s creators suggested that implicit bias influenced important real-world outcomes, such as whether white police officers were more likely to shoot at black suspects.5 This narrative was enthusiastically taken up and replicated by the media, even though there was little evidence to support it at the time.6 A second generation of research has cast considerable doubt on the initial claims made about the IAT and its validity as a psychological tool is questionable.7 The creators of the IAT have modified some of their earlier claims.8 The test has proved a very poor predictor of real-world behaviour, discrediting their initial assertion that IAT scores can determine an individual’s propensity to behave in racially prejudiced ways. Nonetheless, the narrative of unconscious, implicit racial prejudice as a major driver of real-world racial inequality has become commonplace."

(https://freespeechunion.org/wp-content/uploads/2020/09/Unconscious-Bias-Training.pdf)

Characteristics

How does Implicit Association Testing work?

Free Speech Union:

"In practice, the IAT takes place on a computer across the following four rounds:1. To begin with, participants are asked to hit ‘I’ on the keyboard when a positive word shows on screen and ‘E’ when a negative word shows.2. Next, participants are prompted to hit ‘I’ when they see a black African face and ‘E’ when they see a white European face.3. In the third-round participants must press ‘I’ for a good word or a black face and ‘E’ for a bad word or a white face.4. In the last round, this is reversed so that participants press ‘I’ for a bad word or a black face and ‘E’ for a good word or a white face.9 The IAT calculates your reaction speed when associating concepts (positive or negative words) and attributes (black or white faces).10 In other words, the race IAT is measuring how fast or slow you are to associate positive or negative words with different racial groups.

The IAT feeds reaction times into an algorithm that categorises participants as having either a ‘slight,’ ‘moderate’ or ‘strong’ preference for white faces over black or vice versa. So a ‘strong preference’ for white faces would be regarded as evidence of high levels of implicit anti-black bias. The test can also show no preference."

(https://freespeechunion.org/wp-content/uploads/2020/09/Unconscious-Bias-Training.pdf)

Research Findings

Free Speech Union:

Is the IAT a valid psychological test?

Psychological tests must meet certain scientific standards in order to be considered legitimate and useful. The IAT falls well short of these standards in three key areas: reliability, validity and objectivity.12 Realistically, this calls into question the test’s ability to accurately measure implicit racial bias. Jussim et al. summarise this shortcoming as follows:‘There are many scientific gaps in understanding the IAT: it has dubious construct validity and low test-retest reliability; it is subject to various other psychometric oddities; its bias effect has often been computed in a way that appears to exaggerate its size; it has been almost universally misinterpreted and misrepresented as measuring “implicit bias” when, by Greenwald’s (2017) own definition, it does not do so. Its predictive validity has often been found to be modest and is, at best, controversial. Even if we ignore all that, its ability to account for inequality in the present is likely to be limited.’13You can read more about whether it is ethical to use the findings of a discredited psychological test in the context of workplace Unconscious Bias Training (UBT) here. Reliability: Test-retest reliability describes whether a given measure reliably produces the same results over multiple tests. A test which produces different results every time it is taken has low test-retest reliability and one which produces the same results every time has high test-retest reliability.

Science reporter Jesse Signal elaborates: ‘A tape measure has high test-retest reliability because if you measure someone’s height, wait two weeks, and measure it again, you’ll get very similar results. The measurement procedure of grabbing an ice cube from your freezer and seeing how many ice cubes tall your friend is would have much lower test-retest reliability, because different ice cubes might be of different sizes; it’s easier to make errors when counting how many ice cubes tall your friend is.’

The IAT is well below the threshold of acceptable reliability for a psychological test. Participants often get very different scores when they take the test multiple times. This means the IAT is not a reliable psychological measure, calling into question its use in real world environments. Validity: Validity describes how well a test measures what it claims to be measuring. A psychological test with high validity should accurately predict related behaviours.17 For example, a depression test that consistently diagnosed people without depression as depressive would have low validity because it would clearly be measuring something other than depression.The IAT claims to measure unconscious racial prejudice but, as you can read here and here, there is considerable doubt that this is in fact what it is measuring. This suggests that the IAT has poor validity as a psychological test.

• Objectivity: The findings of a psychological test should reflect the objective properties of what is being tested for, not the subjective values of the tester.19

The findings of the IAT are particularly vulnerable to politicised interpretation which decreases validity.20The IAT algorithm classifies rates of reaction speed as evidence of slight, moderate or strong implicit bias according to thresholds set by the creators of the test. These thresholds were not determined by real world behaviours objectively observed to be shared by participants with similar IAT scores. Rather, they reflect subjective judgements by the test’s creators about the significance of differences in reaction times when associating concepts (positive or negative words) and attributes (black or white faces).We can best understand the impact of this on the IAT’s objectivity as a psychological test by examining changes made to the algorithm thresholds by the test’s creators. One such change caused a 20% drop in participants registering a strong anti-black bias. The IAT’s creators have never adequately explained their reasons for altering the algorithm or precisely why it was changed but in the words of Mitchell and Tetlock:‘This change... was not due to a sudden societal shift, nor due to the findings of any studies linking particular bands of IAT scores to particular behaviors. This change was due solely to the researchers’ change in definitions.’

A psychological test that is so heavily influenced by the subjective values of its creators has questionable validity. To quote Mitchell and Tetlock again, when someone gets an IAT score indicating strong anti-black implicit bias:‘All that really means is that the test-taker’s relative reaction times, as measured in milliseconds, were above a threshold arbitrarily set by the test designers (i.e., the bias label is shorthand for reaction time differences – it is not shorthand for bias on anything other than the test.)’21

Can the Implicit Association Test really measure unconscious prejudice?

Several factors cast doubt on the claim that the IAT is measuring something that can be defined as unconscious. The difference between conscious (explicit) and unconscious (implicit) thought is unclear: There is no consensus in psychology and neuroscience about precisely how explicit and implicit thoughts are divided from each other. Luvell Anderson describes it this way:‘A lot of habits we have work like muscle memory; we do them automatically without thinking about it explicitly. But it’s not necessarily clear to me that those habits are unconscious in any deep or significant way.’22This suggests that the IAT is claiming to measure something (unconscious/implicit thought) that it is presently impossible to define.23 IAT participants can predict their score: IAT participants can often predict how implicitly biased the test will find them to be.24 This means that participants show conscious awareness of something that the IAT claims to be unconscious. Explicit measures are as accurate as implicit ones: Explicit self-report measures of racial prejudice ask participants to rank their agreement with statements like ‘Blacks are getting too demanding in their push for equal rights.’ Self-report is regarded as a poor measure of someone’s actual views because participants can choose to answer in a way that doesn’t reflect their true beliefs. The IAT was hailed as a revolutionary tool of psychological analysis in part because it claimed to overcome this difficulty and reveal participant’s hidden attitudes.25 However, meta-analysis shows that the IAT is no better at predicting racist behaviour than explicit self-report measures.

This again suggests that the IAT is not measuring something that can be defined as unconscious in any meaningful way.

Is the Implicit Association Test really measuring implicit bias against racial groups?

It’s important to make clear that our behaviour is influenced by associations we are not consciously aware of. However, we do not yet know how to accurately define or measure unconscious associations, or what they signify. Additionally, human beings do demonstrate in-group favouritism, even when in-groups are defined entirely arbitrarily.27 But again, there is no consensus about what this signifies. Even if we accept that the IAT is measuring implicit thought, the idea that this signifies unrecognised racial prejudice is not universally accepted or well evidenced.28Before continuing, it is worth reiterating that the IAT performs very poorly in terms of validity, reliability and objectivity, as described here. To quote Jesse Signal’s summary: ‘The IAT falls far short of the quality-control standards normally expected of psychological instruments.’29 It is entirely legitimate to question the ability of the IAT to measure any psychological phenomenon, including implicit racial bias.That being said, some of the theories about what the IAT may really be measuring are described below. Reaction Times: Fundamentally, the IAT is a reaction speed test. Are differences in reaction time evidence of implicit racial prejudice? As Jussim et al. put it, ‘Difference in reaction times is not discrimination.’30 Cognitive Ability: Because the IAT equates speed at completing a cognitive task with low levels of implicit bias, participants with slower cognitive processing speeds will be categorised as having high levels of unconscious racial prejudice. People who are good at tasks like crosswords puzzles or video games and consequently have faster cognitive processing speeds, may be scored as having lower levels of implicit bias for this reason alone. Cognitive processing speed declines with age, meaning that IAT scores may vary simply according to how old participants are.31Additionally, a participant can alter their IAT score by completing the test more slowly or quickly and it is possible to improve your score just by practising.32

In effect, rather than testing your level of implicit racial bias, the IAT may simply be measuring how good you are at completing cognitive tasks like the IAT. Stereotype or Base Rate Familiarity: It has been suggested that the IAT may measure familiarity with cultural stereotypes or averages, sometimes called ‘base rates.’ Studies show that base rates have a considerable impact on IAT scores.33 This means that someone may be faster to associate negative concepts with racial attributes simply because they are familiar with the base rate of racist stereotypes in wider society. This says nothing about whether a participant believes such stereotypes to be accurate or justifiable. Jussim et al. explain: ‘defining associations of concepts in memory as ‘bias’ imports a subterranean assumption that there is something wrong with those associations in the absence of empirical evidence demonstrating wrongness.’34 Out-group Empathy: A similar theory finds that the IAT may measure empathy for members of victimized groups. This means that a participant who profoundly disagrees with racist stereotypes may be faster to associate negative words with black faces precisely because they feel empathy with victims of racial inequality. The IAT may falsely interpret awareness of and opposition to racism as evidence of implicit racial prejudice.35 This could explain the surprising fact that the IAT finds a quarter of black participants to have anti-black implicit bias. This theory is explored in an article by Arkes and Tetlock entitled ‘Would Jesse Jackson Fail the Implicit Association Test?’36 Stereotype familiarity and out-group empathy are sometimes called the Association Model of implicit bias.37 Context: IAT scores appear to be highly context dependant. Contextual factors are sometimes called the Propositional Model of implicit bias.38 This means people get different scores depending on the context in which the test is administered and explained to them.39 For example, one IAT study communicated to participants that the faces they would be shown during the test were of churchgoers. Instead of the usual finding that a majority of participants showed anti-black implicit bias, the results showed no detectable bias towards either race.40The impact of context on IAT results was creatively demonstrated in a study by Uhlmann, Brescoll and Paluck. The researchers invented two fictional races, Noffians and Fasites. Research participants were divided into two groups: one group was told that the Noffians were privileged and the Fasites were oppressed and another was told the opposite. Participants then took a race IAT where black and white faces were exchanged for Noffians and Fasites. Both groups were faster to associate negative words with the fictional race they had been told was oppressed.41 Jesse Signal comments:‘the experimenters were able to easily induce what the IAT would interpret as “implicit bias” against Noffians simply by forming an association between them and downtroddenness in general.’42The fact that IAT scores are heavily affected by the context in which the test is taken, and the values communicated to the participants casts doubt on the theory that the IAT measures implicit racial bias on an individual level. In the words of Mitchell, the IAT may actually be measuring ‘properties of the task or the environment rather than attributes of test-takers’.43 As explained here, this may be particularly relevant to the use of implicit association testing in the context of unconscious bias training.

Does a high IAT score mean someone is racist?

Simply put, no. A high IAT score does not mean someone is more likely to be racist in real life. As one researcher puts it:‘[a]n individual who supposedly shows high implicit bias on the IAT is no more likely to discriminate in any given situation than an individual who supposedly shows low implicit bias on the IAT.’45Which begs the question: where has the commonly held belief that the IAT can predict racism come from? The premature claims of the test’s creators and favourable media coverage have both contributed to this misconception.46 This is in spite of the fact that the IAT’s creators have publicly stated that the test should not be employed to predict an individual’s level of racial prejudice. The IAT does not predict real-world behaviour: As we would expect of a psychological test that has poor reliability, validity and objectivity, IAT scores correlate too poorly with real world outcomes to predict individual behaviour. On average, participants scored as having strong anti-black implicit bias are no more likely to be racist than participants scored as having low anti-black implicit bias.48 If the IAT measures ‘implicit racism’ but this does not give rise to racist behaviour and attitudes, how meaningful is a high IAT score?

As Jesse Singal puts it:

12‘Can a bias be a bias if it only exists in the context of a very specific test result, but never bubbles out into the real world?’49 Simply asking someone their views is a more accurate measure of individual racial prejudice than the IAT: The IAT has been found to be no more effective at predicting racist behaviour than explicit, self-report measures.50 If you are trying to find out whether a given individual is racist, research suggests that simply asking them their views on race would be more informative than checking their IAT score"

(https://freespeechunion.org/wp-content/uploads/2020/09/Unconscious-Bias-Training.pdf)