AI Vendor Privacy Due

AI Vendor Privacy Due Diligence

Overview

AI services create complex controller-processor relationships that differ significantly from traditional data processing arrangements. Whether an AI vendor is a processor, joint controller, or independent controller depends on the degree of autonomy the vendor has over personal data processing — particularly regarding model training on customer data, data retention for model improvement, and the vendor's independent purposes for the data. This skill provides the framework for determining controller-processor roles in AI service relationships, conducting privacy due diligence on AI vendors, and establishing appropriate contractual protections.

Controller-Processor Determination for AI

Decision Framework

Key Determination Questions

1. Who determines the purpose of processing? — The entity deciding why personal data is processed
2. Who determines the means of processing? — The entity deciding how data is processed (but "non-essential means" may be delegated to processor)
3. Does the vendor use data for its own purposes? — Model training, benchmarking, product improvement using customer data
4. Does the vendor retain data beyond service delivery? — Data kept after inference, stored for training, retained in logs
5. Does the vendor make independent decisions about the data? — Choosing to train models, sharing with sub-processors not instructed by customer

Common AI Vendor Patterns

Pattern 1: Pure Inference API (Vendor = Processor)

• Customer sends data; receives inference result
• Vendor does not retain input data beyond processing
• Vendor does not train on customer data
• Vendor acts solely on customer's instructions
• Contractual: Art. 28 DPA with clear scope

Pattern 2: AI with Model Improvement (Vendor = Joint Controller)

• Customer sends data; receives inference result
• Vendor retains data to improve its models
• Vendor determines that model training is a purpose
• Both parties benefit from the model improvement
• Contractual: Art. 26 Joint Controller Agreement

Pattern 3: AI with Anonymised Analytics (Assessment Required)

• Customer sends data; receives inference result
• Vendor claims to anonymise data and use aggregated analytics
• If anonymisation is effective: no personal data processing for analytics
• If anonymisation is not effective (re-identification possible): vendor is controller for analytics
• Assessment: Verify anonymisation effectiveness per WP216

Pattern 4: Embedded AI with Telemetry (Vendor = Processor + may be Controller)

• AI model runs in customer environment
• Vendor collects telemetry data including model performance
• If telemetry contains personal data: vendor may be controller for telemetry processing
• Contractual: Art. 28 DPA for model support + separate arrangement for telemetry

Due Diligence Assessment Framework

Phase 1: Vendor AI Processing Inventory

For each AI vendor, document:

Phase 2: Privacy Risk Assessment

Phase 3: Contractual Assessment

AI-Specific Contractual Clauses

Model Training Restrictions

The Processor shall not use Customer Data to train, improve, fine-tune, or
otherwise develop any machine learning model, algorithm, or AI system,
whether for the Customer's benefit or for any other purpose, without prior
written consent from the Customer. Any consent granted shall specify the
scope of permitted training, the data categories involved, and the privacy
safeguards to be applied.

AI Output Accuracy

The Provider acknowledges that AI system outputs about identifiable data
subjects must comply with the accuracy principle under Art. 5(1)(d) GDPR.
The Provider shall implement measures to minimise inaccurate outputs about
data subjects and shall promptly correct inaccurate outputs upon
notification.

Model Privacy and Bias Obligations

The Provider shall conduct periodic privacy audits of AI models processing
Customer Data, including membership inference testing and training data
extraction testing, and shall make results available to the Customer upon
request. The Provider shall monitor AI systems for discriminatory outcomes
and shall implement bias mitigation measures as required.

Enforcement Relevance

• EDPB Guidelines 07/2020 on Controller-Processor: Determination depends on factual circumstances, not contractual labels. A vendor labelled "processor" that uses data for its own purposes is factually a controller.
• Garante v. OpenAI (2023): OpenAI's role as controller for ChatGPT training data confirmed — processing personal data for model training is an independent controller purpose.
• CJEU C-40/17 (Fashion ID): Joint controller status can arise when a party has influence over the purpose and means of processing, even without access to the data.
• DPC v. Meta (WhatsApp, 2023): EUR 5.5M fine — processor-controller determination must reflect actual data use, not just contractual terms.

Integration Points

• ai-dpia: Vendor AI processing must be included in DPIA scope
• ai-training-lawfulness: Vendor training on customer data requires lawful basis assessment
• ai-transparency-reqs: Data subjects must be informed about AI vendor processing
• ai-deployment-checklist: Vendor due diligence is a pre-deployment checklist item

Summary

Ensuring compliance with data protection laws when engaging AI vendors requires careful attention to roles, responsibilities, and lawful bases for processing. Providers must implement robust privacy and bias mitigation measures, maintain transparency with data subjects, and support customer oversight through audits and reporting. Customers should conduct thorough due diligence, incorporate AI vendor risks into DPIAs, and clearly define processing purposes and limits in contracts. Together, these practices help mitigate legal risks and uphold individuals’ fundamental rights in AI-driven data processing.