Summary of Key Points

  • An omnibus federal privacy bill with an unusual amount of bipartisan support is currently under congressional review and, if enacted, could dramatically increase oversight of how companies use artificial intelligence (“AI”) in their businesses.
  • This article discusses the bill, which, even if not enacted, provides valuable insights as to future regulation of AI.

On July 20, 2022, the House Energy and Commerce Committee approved the proposed American Data Privacy and Protection Act (ADPPA) by a 53-2 margin.1 The bill would create national standards and safeguards for personal information collected by companies, including protections for marginalized communities affected by potentially discriminatory impacts of algorithms.

Although the bill is currently in limbo, the ADPPA is a step toward a first-of-its-kind comprehensive data privacy law in the United States. It is the culmination of years of failed attempts by lawmakers to install federal legislation addressing the public’s growing concern relating to privacy, data security, artificial intelligence, fair competition, and bias and is considered well overdue by many experts. Although several other federal bills addressing algorithmic decision-making have been introduced in recent years, the ADPPA is the first with significant bipartisan support and momentum, and the first to bundle provisions targeting algorithmic accountability and bias with provisions addressing data privacy and cybersecurity issues.

Scope and Applicability

If enacted, the ADPPA would apply broadly to organizations and businesses operating in the United States. Key definitions in the proposed legislation include those noted below.

Covered entity is defined as an entity that “collects, processes, or transfers covered data and is subject to the Federal Trade Commission Act,” in addition to nonprofit organizations and common carriers. Though the definition is undeniably broad, the ADPPA identifies several different types of entities with additional obligations or exemptions. Covered entities are divided by “impact” (i.e., annual global revenue and number of data subjects affected by the entity’s operations) and “relationship with the data subject” (e.g., direct, third-party, or service provider relationships). By way of example, a “large” entity is defined as one with annual gross revenues of at least $250 million and that has collected covered data on more than 5 million individuals or devices or has collected sensitive covered data of more than 100,000 individuals or devices. “Small” and “medium” entities are still regulated by the ADPPA but are exempt from some substantive provisions under a qualifying exception.

Covered data is defined as “information that identifies or is linked or reasonably linkable to one or more individuals, including derived data and unique identifiers.” Importantly, both employee data and publicly available data are excluded from this definition. Some covered data may further be considered sensitive covered data, which would include government identifiers (such as driver’s license or Social Security numbers) as well as “traditionally” sensitive information related to health, geolocation, financials, log-in credentials, race, and sexual history or identity. Sensitive data may also include unconventional categories, such as television viewing data, intimate images, and “information identifying an individual’s online activities over time or across third-party websites or online services.”

A service provider is defined as “a person or entity that collects, processes, or transfers covered data on behalf of, and at the direction of, a covered entity for the purpose of allowing the service provider to perform a service or function on behalf of, and at the direction of, such covered entity.” Notably, the ADPPA would place direct obligations on service providers, including obligations not found in state privacy laws such as the prohibition of transferring data, except to another service provider, without affirmative express consent.

A third-party collecting entity is defined as “a covered entity whose principal source of revenue is derived from processing or transferring the covered data that the covered entity did not collect directly from the individuals linked or linkable to the covered data.” Third-party collecting entities would be required to provide consumers with notice of their activity and register with the Federal Trade Commission (“FTC”) if they process data pertaining to more than 5,000 individuals or devices that are reasonably linkable to an individual.

Data Privacy Measures

The bill seeks to address privacy threats in a number of ways:

  • Transparency: Covered entities would be required to disclose data collection, retention, usage, and accessibility policies and include contact information, including phone numbers to which customers could direct privacy and data security inquiries for both the covered entities and any other entities within the same corporate structure to which a covered entity transferred covered data. Entities would also have to disclose third-party collecting entities to whom they transfer covered data.
  • Consent: Covered entities would need to obtain express affirmative individual consent (opt-in) from consumers for the collection and use of certain types of sensitive covered data, including biometric information, genetic information, aggregated internet browsing and search history, physical activity information, and precise geolocation information. Covered entities would also be required to provide consumers with an opportunity to opt out of targeted advertising and of the transfer of any covered data to a third party.
  • Data Minimization: The bill would impose data minimization requirements by prohibiting covered entities from collecting, processing, or transferring covered data that is “beyond what is reasonably necessary, proportionate, and limited to” a product or service provided by the covered entity.
  • Data Security: Covered entities would be required to “establish, implement, and maintain reasonable administrative, technical, and physical data practices and procedures to protect and secure covered data against unauthorized access and acquisition.” While “reasonable” data practices would be scalable depending on the size and nature of the covered entity and covered data, covered entities would have to, as a baseline, conduct vulnerability assessments, maintain preventative and corrective action plans, develop data retention plans, conduct employee training and awareness relating to safeguarding data, and designate personnel responsible for implementing these policies.
  • Additional Protections for Children and Minors: If the covered entity knows an individual is under 17, the covered entity would be prohibited from targeting ads to that individual or transferring the individual’s covered data to a third party without express affirmative consent.

Oversight of AI and Algorithmic Decision-Making

Notably with respect to AI, the ADPPA includes a provision—Section 207: Civil Rights and Algorithms—under which covered entities or service providers “may not collect, process, or transfer covered data in a manner that discriminates in or otherwise makes unavailable the equal enjoyment of goods or services on the basis of race, color, national origin, sex, or disability.” The two limited exceptions are a covered entity’s self-testing to prevent or mitigate unlawful discrimination and a covered entity’s efforts to diversify an applicant, participant, or customer pool.

Unlike most existing state privacy laws, Section 207 of the ADPPA would go a step further by requiring companies to evaluate certain artificial intelligence tools and submit those evaluations to regulators.

Which entities are subject to Section 207? Covered entities and service providers that develop algorithms to collect, process, or transfer covered data or publicly available information would be required to conduct algorithm design evaluations prior to deploying the algorithms in interstate commerce. In addition, anylarge data holder that uses an algorithm “that may cause potential harm to an individual,” and uses such algorithm to collect, process, or transfer covered data, would also be required to conduct an algorithm impact assessment on an annual basis.

What is an “algorithm”? The bill defines an “algorithm” as “a computation process that uses machine learning, natural language processing, artificial intelligence techniques, or other computational processing techniques of similar or greater complexity that makes a decision or facilitate human decision-making with respect to covered data, including to determine the provision of products or services or to rank, order, promote, recommend, amplify, or similarly determine the delivery or display of information to an individual.”

What is an “algorithm design evaluation”? According to the proposed bill, covered entities and service providers must evaluate the design, structure, and data inputs of the algorithm to reduce the risk of potential discriminatory impacts. The draft legislation emphasizes that algorithm design evaluations must occur at the design phase, including any training data used to develop the algorithm. The ADPPA would also require the use of an external, independent researcher or auditor to conduct the evaluation to the extent possible. The covered entity or service provider would be required to submit the evaluation to the FTC no later than 30 days after completion of the evaluation and to make it available to Congress upon request.

What is an “algorithm impact assessment”? For large data holders who use algorithms that may cause potential harm to an individual, and that use such algorithms to collect, process, or transfer covered data, an algorithm impact assessment is also required. The draft bill provides a detailed description of these assessments and requires that they include:

  • A detailed description of the design process and methodologies of the algorithm;
  • A statement of the algorithm’s purpose, its proposed uses, and its foreseeable capabilities outside of the articulated proposed use;
  • A detailed description of the data inputs used by the algorithm, including the specific categories of data that will be processed and any data used to train the underlying model;
  • A description of the outputs produced by the algorithm;
  • An assessment of the necessity and proportionality of the algorithm in relation to its purpose, including the reasons an algorithm is superior to a non-automated decision making process; and
  • A detailed description of steps to mitigate potential harms.

Large data holders would be required to submit the impact assessment to the FTC no later than 30 days after completion of the assessment and continue to produce assessments on an annual basis. As with algorithm design evaluations, the proposed legislation would require the use of an external, independent researcher or auditor to conduct the algorithm impact assessment, to the extent possible.

The level of prescriptive detail is sure to be helpful to many potential “covered entities” and “service providers” and may even provide a basis to argue that something of a safe harbor exists if entities have conducted impact assessments that comply with the aforementioned requirements. However, it may also require many companies, and especially large data holders, to dedicate significant resources to assessing their algorithmic tools during the development phase and additional resources to monitoring those same tools during and after development.

Which “potential harms” require an algorithm impact assessment? The following potential harms are expressly highlighted in the text of the bill, suggesting that these are areas of focus for lawmakers:

  • Potential harms related to individuals under the age of 17;
  • Potential harms related to advertising for, access to, or restrictions on the use of housing, education, employment, healthcare, insurance, or credit opportunities;
  • Potential harms related to determining access to, or restrictions on the use of, any place of public accommodation, particularly as such harms relate to protected characteristics, including race, color, religion, national origin, sex, or disability; and
  • Potential harms related to disparate impact on the basis of individuals’ race, color, religion, national origin, sex, or disability status.

The language of the proposed bill suggests that this list of potential harms is not exhaustive. It is also worth noting that the bill is up for review at a time when there is significant regulatory attention on ad targeting and digital marketing, including by the Consumer Financial Protection Bureau, which recently issued an interpretive rule on digital marketing and expressed concern over discriminatory conduct online and “digital redlining.”2

What does it mean to “discriminate” under Section 207? One of the key questions raised by the proposed legislation, and one that will be critical to assessing compliance, is what exactly does it mean to “discriminate” under Section 207 of the ADPPA? While Section 207’s reporting requirements involve descriptions of any “disparate impact” resulting from the deployment of an algorithm in a covered entity’s business practices, it is unclear what legal standards would be used in assessing discrimination or disparate impact under the proposed legislation and what type of business justification might suffice to satisfy the proposed bill’s requirements. Depending on the algorithm, it may be very difficult—if not impossible—to completely eliminate all disparate impact against any protected classes, even when using objective and facially non-discriminatory criteria. In addition, the proposed legislation refers to “protected characteristics,” but this term is not defined, nor does the proposed legislation reference any federal or state anti-discrimination laws that explicitly enunciate the so-called “prohibited bases” that such laws are designed to protect. Moreover, the proposed bill does not address how companies are expected to perform testing in the absence of demographic data such as race or national origin and whether proxying methodologies (such as the Bayesian Improved Surname Geocoding—or “BISG”) would be required. On this point, additional clarity is welcome and may be the point of discussion among lawmakers in the days and weeks to come.

Enforcement

The ADPPA would create a Bureau of Privacy at the FTC to enforce its provisions, and any ADPPA violation would be treated as a violation of a rule defining an unfair or deceptive act or practice (“UDAP”) under section 18(a)(1)(B) of the Federal Trade Commission Act (15 U.S.C. 57a(a)(1)(B)).

With respect to Section 207, the ADPPA would authorize the FTC to promulgate regulations to establish processes by which large data holders can submit impact assessments and exclude from assessment “any algorithm that presents low or minimal risk for potential for harms to individuals.” The ADPPA would also require the FTC to publish guidance within two years of the bill’s enactment regarding compliance with Section 207 and a study within three years of the best practices for assessment and evaluation of algorithms and methods to reduce the risk of harm. These publications may help provide guidance to companies as they navigate compliance and dedicate resources to the evaluation of algorithmic tools.

Although the ADPPA as drafted includes a private right of action, it, importantly, would not apply to Section 207’s provisions related to potential discrimination. Instead, the FTC and state attorneys general would be empowered with enforcement authority.

What’s Next?

Despite significant bipartisan support, the bill has faced significant resistance from California lawmakers who argue that the bill would preempt the California Privacy Rights Act (“CPRA”), which they argue offers stronger protections to California residents. Several state attorneys general have also sent a joint letter to Congress expressing the urgent need to amend the bill to explicitly allow states to pass potentially more expansive privacy, data, and artificial intelligence-related regulations in the future as technology and online practices evolve. Two senators have also expressed concern in recent weeks regarding the perceived laxity of the bill’s enforcement provisions.

In general, there are two key issues that threaten the bill’s chances of success at the moment: (1) the preemption of state privacy laws and (2) whether the law should create a private right of action for individuals to sue companies for violations. The bill’s enactment may depend on whether politicians can reach a compromise or consensus on these issues.

Even if its enactment hangs in the balance, the ADPPA provides significant insights as to the type of oversight of AI tools that lawmakers and regulators may seek to exercise in the near future.

Companies may want to consider proactively and preemptively developing internal impact assessment forms for design teams to fill out during the development phase of algorithmic products that use covered data, paying particular attention to data integrity and data inputs; human oversight, monitoring, and control; and, potentially, disparate impact analyses. These impact assessment forms and related processes could be embedded into existing governance protocols, and training could be arranged for relevant stakeholders.

Key to these efforts will be the formation of a well-rounded team to complete evaluations of AI and algorithmic tools. Companies should consider whether their organizations would benefit from the addition of an AI committee or whether existing risk committees or other bodies can expand their remit to assess impacts of algorithmic applications. Either way, the team conducting the impact assessment should be cross-functional and diverse—design and technology experts, risk and/or compliance strategists, marketing professionals, ethicists, and lawyers are all important advisors during this process.

Finally, companies using AI tools should consider implementing a system of records retention for design evaluations/impact assessments and may want to consider engaging a third-party auditor, such as a law firm, consulting firm, or other third-party professional to assist with evaluations.