New York City Releases Final Rules Implementing Local Law 144 Aimed at Curbing Artificial Intelligence Bias in Employment Decisions

Seyfarth Synopsis: Earlier today, New York City’s Department of Consumer and Workplace Protection (“DCWP”) released the highly anticipated final rules implementing New York City Local Law 144 of 2021 (“Local Law 144”), which regulates the use of automated employment decision tools (“AEDT”). While Local Law 144 took effect on January 1, 2023, DCWP stated in a transmittal email accompanying the final rules that the rules will be effective, and enforcement will begin, on July 5, 2023.

By way of background, DCWP initially released a set of proposed rules implementing Local Law 144 on September 23, 2022 and held a public hearing on November 4, 2022. Due to stakeholder concerns raised during the hearing and public comment period, including those raised by employers, employment agencies, law firms, AEDT developers, and advocacy organizations, DCWP issued revised proposed rules on December 23, 2022. After a January 23, 2023 public hearing on the revised proposed rules, the final rules have now been issued which govern the use of AEDTs in NYC.

According to DCWP, the changes to the final rules:

1. Modify the definition of “machine learning, statistical modeling, data analytics, or artificial intelligence” to expand its scope;
2. Add a requirement that bias audits indicate the number of individuals an AEDT assessed that are not included in the calculations because race/ethnicity and sex data is unknown, and require that number be included in the results summary;
3. Allow an independent auditor to exclude a category that comprises less than 2% of the data being used for the bias audit from the impact ratio calculations;
4. Clarify examples of a bias audit and when an employer or employment agency may rely on a bias audit conducted using the historical data of other employers or employment agencies;
5. Provide examples of when an employer or employment agency may rely on a bias audit conducted with historical data, test data, or historical data from other employers and employment agencies; and
6. Clarify that the number of applicants in a category and scoring rate of a category, if applicable, must be included in the results summary.

These specific changes are discussed in more detail below.

Machine Learning, Statistical Modeling, Data Analytics, Or Artificial Intelligence

The Final Rules modified the definition of “machine learning, statistical modeling, data analytics, or artificial intelligence” to mean a group of mathematical, computer-based techniques that:

1. generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and
2. for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.

The definition eliminates the prior requirement that to be considered “machine learning, statistical modeling, data analytics, or artificial intelligence,” a mathematical computer-based technique must also allow for inputs and parameters to be refined through cross-validation or by using training and testing data.  

Bias Audits, Publication and Data Requirements

The Final Rules clarify that bias audits are required even where an AEDT is not being used in connection with a final hiring decision. Rather, bias audits are also required when used to “screen” at early points in the hiring process. 

Further clarification regarding the information that must be disclosed in connection with the required bias audit was also provided.  Specifically, NYC employers will be required to disclose a summary of the results of its bias audit, which is to include (1) “the source and explanation” of the data used to conduct the bias audit, (2) the date of the last audit, (3) number of applicants or candidates, (4) the selection or scoring rates, (5) the impact ratio by race/ethnicity, sex, and intersectional categories, and (6) the number of individuals with unknown race/ethnicity and sex that were assessed by the tool.  To comply with the “unknown” disclosure, the following example was provided: “The AEDT was also used to assess 250 individuals with an unknown sex race/ethnicity category. Data on those individuals was not included in the calculations above.”

When conducting a bias audit, independent auditors are permitted to exclude a race/ethnicity or sex category that represents less than 2% of the data being used from the required impact ratio calculations. If any category is excluded, the summary of results must include the “justification for the exclusion, as well as the number of applicants and scoring rate or selection rate for the excluded category.”

Further clarification is also provided regarding when historical or test data can be used when conducting a bias audit. The general rule is that a bias audit must rely on historical data.  However, a bias audit may rely on the historical data of other employers or employment agencies only if (1) such employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit or (2) if such employer or employment agency has never used the AEDT at issue.

In instances where insufficient historical data is available, then an employer or employment agency may rely on a bias audit that relied on test data to conduct a statistically significant bias audit. Where test data is utilized, the summary of the results of the bias audit must explain why historical data was not used and describe how the test data used was generated and obtained.

Enforcement Deadline Extended

In an attempt to provide employers and employment agencies with more time to comply with Local Law 144’s final regulations, DCWP announced that it will delay enforcement of the final rules until July 5, 2023. Since this three-month delay does not provide employers with much time to come into compliance, those impacted by this law must be diligent in assessing the tools it uses that may qualify as an AEDT under NYC law, evaluate how the final rules impact their operations, and what next steps need to be taken to mitigate any non-compliance risks.

Workplace Solutions

We encourage you to contact the authors of this article or a member of Seyfarth’s People Analytics team as soon as possible if your organization seeks assistance in complying with Local Law 144’s final rules.