Automated Decisions, Consumer Protection & Algorithmic Scoring

EU SCHUFA Scoring Case

A credit agency automatically generated credit scores for use by third parties such as banks. The Court of Justice of the European Union held that when a score plays a decisive role in a third party's decision, the scoring activity constitutes automated individual decision-making governed by Article 22 of the GDPR.

Final Interpretive Judgment

• The court confirmed that scoring falls under the rules on automated individual decision-making when it plays a decisive role in third-party decisions.
• Such processing requires a valid legal basis and appropriate safeguards for data subjects' rights.

US COMPAS Sentencing Case

Criminal sentencing materials incorporated a commercial COMPAS recidivism risk score. The court permitted limited consideration of the score, required clear warnings regarding the tool's limitations, and prohibited using the score to determine sentence severity or whether incarceration is warranted.

State Supreme Court Decision

• The state supreme court affirmed the lower court's decision.
• The court permitted judges to consult COMPAS scores subject to explicit limitations and cautionary instructions.

EU Automated Decision-Making Explanation Case

The Court of Justice of the European Union ruled that data subjects are entitled to an explanation sufficient to understand and challenge automated decisions. Such explanations must disclose the concrete procedures and principles applied and illustrate how changes in personal data influence the resulting outcome.

Preliminary Ruling

• The explanation must convey the procedures and principles underlying the automated decision in a concise, intelligible format.
• The explanation must enable the data subject to understand how specific personal data affected the decision and to contest the outcome effectively.

UK Police Live Facial Recognition Case

South Wales Police deployed a live facial recognition system in public spaces. The Court of Appeal held that the criteria for watchlists and deployment locations lacked sufficient legal constraints, and identified deficiencies in the data protection impact assessment and compliance with the public sector equality duty.

Appellate Judgment

• The Court of Appeal held that the deployments were unlawful on three grounds.
• The governing legal framework conferred overly broad discretion regarding watchlist composition and deployment locations.

Ewert v. Canada Risk Assessment Case

The Correctional Service of Canada assessed a Métis inmate using recidivism risk tools developed and validated on non-Indigenous populations. The Supreme Court held that the correctional authority failed to take reasonable steps to confirm the validity of the tools for Indigenous persons.

Supreme Court Judgment

• The Supreme Court of Canada allowed the appeal in part.
• The Court found that the Correctional Service breached its statutory obligation to take reasonable steps to ensure that information used in decisions was accurate, up to date, and complete.

AI & Data Compliance

Map data sources, processing roles, purposes, flows and controls across AI systems.

AI Product Compliance

Embed legal review into design, development, procurement, testing, launch, operation and updates.

Model & Platform Governance

Review admission, risk tiering, content safety, monitoring, audit and accountability for models, algorithms and platforms.