In Limitation of the “Four-Fifths Rule” and Statistical Parity Tests for Measuring Fairness, authors Manish Raghavan & Pauline T. Kim critique the use of the four-fifths test and other statistical parity tests (SPTs) in employment decision making. The article discusses how SPTs have been used historically to identify adverse impact as part of a disparate impact discrimination inquiry and how SPTs are being used now, and how that may obscure algorithmic discrimination.
The article is intriguing because it discusses where we have been, where we are, and where we may be going regarding the use of SPTs and algorithmic discrimination. That is especially important because the use of algorithms in employment decision making is likely here to stay. Algorithms can be used to screen job applicants for later evaluation or to rank job applicants for sequential hiring or for other purposes. Concerns regarding the possibility of discrimination through algorithmic use are common. Indeed, various jurisdictions, including New York City, require audits of employment selection algorithms before they are used.
The article provides a good primer on the law regarding SPTs and a discussion of their historical use. The article explains the employment law issues, such as the difference between a finding adverse impact and a finding of disparate impact clearly. It also discusses the problems and possible benefits of using SPTs retrospectively and prospectively, providing a balanced and nuanced approach to using tools that have been used bluntly in the employment context.
As they discuss the uses and misuses of SPTs, the authors suggest how SPTs can be used more judiciously in assessing whether an employment rule has triggered an adverse impact with respect to a known applicant pool. In addition, they propose measures to be used in combination with SPTs to help designers craft algorithms that are fair and nondiscriminatory, rather than algorithms designed in part merely to avoid a finding of adverse impact and possible discrimination claims. The authors’ suggestions regarding the prospective use of algorithms are especially important, as the use of algorithms by employers is unlikely to diminish.
The article is rich with wonderful insights. Three key points stand out.
First, the four-fifths test, the simplest of the SPTs and initial focus of the article, has been used as a rule of thumb to help identify possibly discriminatory rules, but it is not very good at the task. Any employment rule may disqualify a higher percentage of some groups of applicants than other groups of applicants. The core issue is how much adverse impact is sufficient to trigger the need to examine whether the rule might cause an unlawful disparate impact.
The four-fifths test is violated, for example, when the use of a rule yields a selection rate of one racial group is less than 4/5ths of the selection ratio of another racial group. Federal agencies have historically treated the violation of the rule as evidence that an employment rule has had an adverse impact on the under-selected group based on race. That typically triggers further evaluation of whether the rule caused a disparate impact on the group that might violate an employment discrimination statute.
However, the authors note the four-fifths rules is an unsophisticated statistical tool that is both overinclusive and underinclusive in detecting discriminatory conduct. Nonetheless, when used carefully, the rule and other more sophisticated SPTs, can help identify when an employment rule should be scrutinized for possible bias.
Second, the recognition of algorithmic discrimination has led algorithm designers to use the four-fifths test prospectively to attempt to avoid a finding of adverse impact and, they surmise, discrimination. That approach has a number of problems. As noted, the four-fifths test is not an accurate test of adverse impact or discrimination.
In addition, any prospective use of SPTs has a “data-dependence problem” because no data set that is defined prospectively to test the employment rule will necessarily match a subsequent applicant pool. More troubling, concentrating on the results of the prospective use of an SPT can trigger a focus on adverse impact rather than validity, i.e., the algorithm’s ability to identify the strongest set of applicants.
When designers focus insufficiently on an algorithm’s validity, the resulting algorithm may be discriminatory – though it does not violate the four-fifths test – but also may not help the employer make better employment decisions. That is a lose-lose situation.
Third, SPTs and other measures can be used to audit algorithms to lessen discrimination and encourage the accuracy or validity of the underlying algorithm. Though the authors provide recommendations that can help make algorithms better, they caution that even their interventions will not guarantee algorithms will not discriminate.
Anyone who is interested in employment law should read this article. The breadth of coverage makes the article a good read for those who think about employment law and those who practice or seek to practice employment law. It also may be helpful to employers and non-lawyers who care about employee selection processes. For those reasons, it is a Thing I Love Lots.
