Data Protection Impact Assessment for AI/ML Systems
Overview
AI and ML systems present unique privacy challenges that traditional DPIA methodologies fail to adequately address. The EDPB Guidelines 04/2025 on processing personal data through AI systems establish a specialized framework that supplements the general DPIA requirements of GDPR Article 35 and WP248rev.01. AI-specific DPIAs must evaluate the entire ML pipeline — from training data collection through model deployment and inference — assessing risks that emerge from statistical learning, emergent model behaviours, and the opacity of algorithmic decision-making. This skill implements the EDPB's AI-specific DPIA methodology integrated with the EU AI Act risk classification framework.
AI-Specific DPIA Triggers
Mandatory DPIA Triggers for AI Systems
All AI processing that meets any of the following criteria requires a DPIA before deployment:
| Trigger |
Legal Basis |
Description |
| AI-based profiling with legal effects |
Art. 35(3)(a) GDPR |
ML models that produce decisions with legal or similarly significant effects on natural persons (credit scoring, hiring, insurance pricing) |
| Training on special category data |
Art. 35(3)(b) GDPR |
Models trained on health, biometric, genetic, racial, political, religious, sexual orientation, or trade union data at scale |
| AI-powered surveillance |
Art. 35(3)(c) GDPR |
Computer vision, facial recognition, behavioural analytics, or anomaly detection in public spaces |
| High-risk AI systems |
Art. 6 EU AI Act |
Systems listed in Annex III of the AI Act (biometric identification, critical infrastructure, employment, law enforcement, migration, justice) |
| Foundation models processing personal data |
EDPB Guidelines 04/2025 |
LLMs and foundation models trained on datasets containing personal data, regardless of downstream use |
| Automated inference of sensitive attributes |
EDPB Guidelines 04/2025 |
Models that infer Art. 9 special category data from non-sensitive inputs (inferring health status from purchasing patterns) |
EDPB WP248 Criteria Applied to AI
AI systems frequently trigger multiple WP248 criteria simultaneously:
- Evaluation/scoring: Inherent to classification and regression models
- Automated decision-making: Core function of deployed AI systems
- Innovative technology: Novel model architectures, training techniques
- Large-scale processing: Training datasets containing millions of records
- Matching/combining datasets: Multi-source training data aggregation
- Vulnerable data subjects: When AI is applied to children, employees, patients
When an AI system meets two or more criteria, a DPIA is presumptively required.
AI DPIA Methodology — EDPB Framework
Phase 1: AI System Description (Art. 35(7)(a) Extended)
The systematic description must cover the complete AI lifecycle:
1.1 Training Phase Documentation
- Training data sources: Origin, collection method, consent status, lawful basis for each dataset
- Data categories: All personal data categories present in training data, including data that may be inadvertently included (background individuals in images, metadata in text corpora)
- Data volume: Number of records, data subjects affected, geographic scope
- Data preprocessing: Cleaning, augmentation, labelling processes and any human review
- Feature engineering: Which personal data attributes are used as features, how derived features are computed
- Training infrastructure: Where training occurs (cloud provider, jurisdiction), data residency during training
1.2 Model Architecture Documentation
- Model type: Neural network architecture (transformer, CNN, RNN), ensemble methods, decision trees
- Model parameters: Number of parameters, model size, complexity indicators
- Explainability characteristics: Inherent interpretability level (white-box, grey-box, black-box)
- Memorization risk: Assessed propensity for the model to memorize training data (higher for large models with small datasets)
1.3 Deployment Phase Documentation
- Inference inputs: What personal data is processed at inference time
- Output types: Classifications, scores, recommendations, generated content
- Decision pipeline: How model outputs feed into decisions affecting data subjects
- Human oversight: Level and effectiveness of human review in the decision chain
- Monitoring: Drift detection, performance monitoring, feedback loops
Phase 2: AI-Specific Necessity and Proportionality
2.1 Purpose Limitation for AI
- Is the AI system necessary for the stated purpose, or could simpler processing achieve it?
- Has the controller evaluated non-AI alternatives and documented why AI is required?
- Are the training data processing purposes compatible with the original collection purposes (Art. 6(4) assessment)?
- For repurposed data: has a compatibility assessment been conducted per EDPB Guidelines 04/2025?
2.2 Data Minimisation for AI
- Has the minimum dataset required for acceptable model performance been determined through ablation studies?
- Can synthetic data, federated learning, or differential privacy reduce the personal data requirement?
- Are there personal data elements in the training data that do not contribute to model performance?
- Has the controller assessed whether anonymised or pseudonymised data could achieve adequate performance?
2.3 Training Data Lawfulness Assessment
For each training dataset, document:
| Assessment Element |
Requirement |
| Original collection purpose |
Was personal data collected for a purpose compatible with AI training? |
| Lawful basis |
Art. 6(1) basis for the training processing — legitimate interest requires balancing test |
| Consent validity |
If consent is the basis, was AI training specified as a purpose? Was consent freely given? |
| Special category conditions |
If Art. 9 data is present, which Art. 9(2) exception applies? |
| Web-scraped data |
EDPB position: web scraping for AI training generally cannot rely on legitimate interest without additional safeguards |
| Third-party datasets |
Has the controller verified the upstream lawful basis chain? |
Phase 3: AI-Specific Risk Assessment
3.1 Privacy Risk Categories for AI
| Risk Category |
Description |
Likelihood Factors |
| Training data extraction |
Adversary extracts verbatim training data from the model |
Model size, training data repetition, overfitting degree |
| Membership inference |
Adversary determines if specific data was in the training set |
Model confidence distribution, overfitting, shadow model availability |
| Model inversion |
Adversary reconstructs input features from model outputs |
Output granularity, model type, auxiliary information available |
| Attribute inference |
Model reveals sensitive attributes not provided as input |
Correlations in training data, feature interactions |
| Emergent bias amplification |
Model amplifies biases present in training data, producing discriminatory outcomes |
Training data representativeness, debiasing measures applied |
| Concept drift discrimination |
Model performance degrades unequally across demographic groups over time |
Monitoring coverage, retraining frequency |
| Re-identification through AI output |
Model outputs enable linking back to specific data subjects |
Output specificity, population uniqueness, auxiliary data |
| Automated decision errors |
Incorrect AI decisions causing material harm to data subjects |
Model accuracy, error distribution across groups |
3.2 AI Risk Scoring Matrix
Combine likelihood and severity using the EDPB-recommended matrix:
Negligible Limited Significant Maximum
Almost Certain Medium High Very High Very High
Likely Medium High High Very High
Possible Low Medium High High
Remote Low Low Medium High
3.3 AI Act Risk Level Integration
Cross-reference GDPR risk assessment with AI Act classification:
- Unacceptable risk (Art. 5 AI Act): Processing must not proceed — social scoring, real-time biometric identification in public spaces (with limited exceptions)
- High risk (Art. 6 + Annex III): Enhanced DPIA obligations, conformity assessment required
- Limited risk (Art. 50): Transparency obligations — inform users they are interacting with AI
- Minimal risk: Standard DPIA process applies
Phase 4: AI-Specific Mitigation Measures
Technical Measures
| Measure |
Risk Addressed |
Implementation |
| Differential privacy |
Training data extraction, membership inference |
Apply DP-SGD during training with calibrated epsilon (ε ≤ 8 for moderate protection, ε ≤ 1 for strong) |
| Federated learning |
Data centralisation risk |
Distribute training across data holders without centralising personal data |
| Model output perturbation |
Model inversion, attribute inference |
Add calibrated noise to model outputs, round confidence scores |
| Training data deduplication |
Memorization risk |
Remove duplicate and near-duplicate records before training |
| Membership inference testing |
Membership inference |
Run MI attacks against the model pre-deployment; retrain if leakage exceeds threshold |
| Fairness constraints |
Bias amplification |
Apply demographic parity, equalised odds, or calibration constraints during training |
| Input/output filtering |
PII leakage in generative models |
Deploy PII detection on model inputs and outputs with automated redaction |
| Model pruning and distillation |
Memorization, extraction |
Compress the model to reduce capacity for memorizing individual records |
Organisational Measures
- Establish an AI Ethics Review Board with privacy representation
- Implement model cards documenting privacy properties for each deployed model
- Conduct regular model audits (minimum annually) testing for privacy leakage
- Maintain training data provenance documentation and deletion capability
- Define retraining triggers and ensure DPIA review accompanies each retraining cycle
- Implement incident response procedures specific to AI privacy incidents
Phase 5: Human Oversight Assessment
Per AI Act Art. 14 and GDPR Art. 22, assess the human oversight mechanism:
| Oversight Element |
Assessment Question |
| Meaningful review |
Can the human reviewer effectively evaluate the AI recommendation and override it? |
| Time and resources |
Is sufficient time allocated for meaningful review, or is the human a rubber stamp? |
| Competence |
Does the reviewer have the expertise to identify AI errors? |
| Authority |
Does the reviewer have the authority and means to override the AI? |
| Feedback mechanism |
Are overrides recorded and fed back into model improvement? |
| Automation bias |
Are measures in place to mitigate the tendency to defer to the AI? |
Prior Consultation Triggers for AI
Art. 36 prior consultation with the supervisory authority is required when:
- The AI system produces high residual risk after all mitigation measures
- The AI system processes special category data at scale with novel techniques
- The supervisory authority's Art. 35(4) list specifically includes the AI use case
- The AI system is deployed for real-time biometric identification under AI Act Art. 5 exceptions
Enforcement Precedents
- Clearview AI (Multiple DPAs, 2021-2024): Fines totalling over EUR 90 million across Italy (EUR 20M), France (EUR 20M), UK (GBP 7.5M), Greece (EUR 20M) for facial recognition AI deployed without DPIA, lawful basis, or transparency
- CNIL v. Clearview AI (SAN-2022-019): Specific finding that no DPIA was conducted for biometric AI processing
- Italian DPA v. Replika (2023): Ordered cessation of AI chatbot processing due to inadequate age verification and failure to conduct DPIA for AI processing affecting minors
- Spanish DPA v. CaixaBank (PS/00421/2020): EUR 6 million fine for automated credit scoring without adequate DPIA addressing algorithmic decision-making risks
- Dutch DPA v. Tax Authority (2020): Finding that algorithmic fraud detection system (SyRI) lacked proportionality and adequate DPIA for AI-driven profiling
Integration Points
- ai-training-lawfulness: Detailed lawful basis analysis for training data feeds into Phase 2
- ai-automated-decisions: Art. 22 assessment integrates with Phase 5 human oversight
- ai-model-privacy-audit: Technical privacy testing results feed into Phase 3 risk assessment
- ai-act-high-risk-docs: AI Act conformity assessment aligns with Phase 3.3 risk classification
- ai-bias-special-category: Bias assessment results feed into risk scoring for discrimination harms
1---2name: ai-dpia3description: Conducts Data Protection Impact Assessments for AI and ML systems per EDPB Guidelines 04/2025 on AI processing. Covers training data lawfulness evaluation, model risk assessment, automated decision triggers, and AI-specific DPIA methodology. Keywords: AI DPIA, machine learning impact assessment, EDPB AI guidelines, model risk, training data.4license: Apache-2.05---67# Data Protection Impact Assessment for AI/ML Systems89## Overview1011AI and ML systems present unique privacy challenges that traditional DPIA methodologies fail to adequately address. The EDPB Guidelines 04/2025 on processing personal data through AI systems establish a specialized framework that supplements the general DPIA requirements of GDPR Article 35 and WP248rev.01. AI-specific DPIAs must evaluate the entire ML pipeline — from training data collection through model deployment and inference — assessing risks that emerge from statistical learning, emergent model behaviours, and the opacity of algorithmic decision-making. This skill implements the EDPB's AI-specific DPIA methodology integrated with the EU AI Act risk classification framework.1213## AI-Specific DPIA Triggers1415### Mandatory DPIA Triggers for AI Systems1617All AI processing that meets any of the following criteria requires a DPIA before deployment:1819| Trigger | Legal Basis | Description |20|---------|-------------|-------------|21| AI-based profiling with legal effects | Art. 35(3)(a) GDPR | ML models that produce decisions with legal or similarly significant effects on natural persons (credit scoring, hiring, insurance pricing) |22| Training on special category data | Art. 35(3)(b) GDPR | Models trained on health, biometric, genetic, racial, political, religious, sexual orientation, or trade union data at scale |23| AI-powered surveillance | Art. 35(3)(c) GDPR | Computer vision, facial recognition, behavioural analytics, or anomaly detection in public spaces |24| High-risk AI systems | Art. 6 EU AI Act | Systems listed in Annex III of the AI Act (biometric identification, critical infrastructure, employment, law enforcement, migration, justice) |25| Foundation models processing personal data | EDPB Guidelines 04/2025 | LLMs and foundation models trained on datasets containing personal data, regardless of downstream use |26| Automated inference of sensitive attributes | EDPB Guidelines 04/2025 | Models that infer Art. 9 special category data from non-sensitive inputs (inferring health status from purchasing patterns) |2728### EDPB WP248 Criteria Applied to AI2930AI systems frequently trigger multiple WP248 criteria simultaneously:3132- **Evaluation/scoring**: Inherent to classification and regression models33- **Automated decision-making**: Core function of deployed AI systems34- **Innovative technology**: Novel model architectures, training techniques35- **Large-scale processing**: Training datasets containing millions of records36- **Matching/combining datasets**: Multi-source training data aggregation37- **Vulnerable data subjects**: When AI is applied to children, employees, patients3839When an AI system meets two or more criteria, a DPIA is presumptively required.4041## AI DPIA Methodology — EDPB Framework4243### Phase 1: AI System Description (Art. 35(7)(a) Extended)4445The systematic description must cover the complete AI lifecycle:4647#### 1.1 Training Phase Documentation4849- **Training data sources**: Origin, collection method, consent status, lawful basis for each dataset50- **Data categories**: All personal data categories present in training data, including data that may be inadvertently included (background individuals in images, metadata in text corpora)51- **Data volume**: Number of records, data subjects affected, geographic scope52- **Data preprocessing**: Cleaning, augmentation, labelling processes and any human review53- **Feature engineering**: Which personal data attributes are used as features, how derived features are computed54- **Training infrastructure**: Where training occurs (cloud provider, jurisdiction), data residency during training5556#### 1.2 Model Architecture Documentation5758- **Model type**: Neural network architecture (transformer, CNN, RNN), ensemble methods, decision trees59- **Model parameters**: Number of parameters, model size, complexity indicators60- **Explainability characteristics**: Inherent interpretability level (white-box, grey-box, black-box)61- **Memorization risk**: Assessed propensity for the model to memorize training data (higher for large models with small datasets)6263#### 1.3 Deployment Phase Documentation6465- **Inference inputs**: What personal data is processed at inference time66- **Output types**: Classifications, scores, recommendations, generated content67- **Decision pipeline**: How model outputs feed into decisions affecting data subjects68- **Human oversight**: Level and effectiveness of human review in the decision chain69- **Monitoring**: Drift detection, performance monitoring, feedback loops7071### Phase 2: AI-Specific Necessity and Proportionality7273#### 2.1 Purpose Limitation for AI7475- Is the AI system necessary for the stated purpose, or could simpler processing achieve it?76- Has the controller evaluated non-AI alternatives and documented why AI is required?77- Are the training data processing purposes compatible with the original collection purposes (Art. 6(4) assessment)?78- For repurposed data: has a compatibility assessment been conducted per EDPB Guidelines 04/2025?7980#### 2.2 Data Minimisation for AI8182- Has the minimum dataset required for acceptable model performance been determined through ablation studies?83- Can synthetic data, federated learning, or differential privacy reduce the personal data requirement?84- Are there personal data elements in the training data that do not contribute to model performance?85- Has the controller assessed whether anonymised or pseudonymised data could achieve adequate performance?8687#### 2.3 Training Data Lawfulness Assessment8889For each training dataset, document:9091| Assessment Element | Requirement |92|-------------------|-------------|93| Original collection purpose | Was personal data collected for a purpose compatible with AI training? |94| Lawful basis | Art. 6(1) basis for the training processing — legitimate interest requires balancing test |95| Consent validity | If consent is the basis, was AI training specified as a purpose? Was consent freely given? |96| Special category conditions | If Art. 9 data is present, which Art. 9(2) exception applies? |97| Web-scraped data | EDPB position: web scraping for AI training generally cannot rely on legitimate interest without additional safeguards |98| Third-party datasets | Has the controller verified the upstream lawful basis chain? |99100### Phase 3: AI-Specific Risk Assessment101102#### 3.1 Privacy Risk Categories for AI103104| Risk Category | Description | Likelihood Factors |105|---------------|-------------|-------------------|106| Training data extraction | Adversary extracts verbatim training data from the model | Model size, training data repetition, overfitting degree |107| Membership inference | Adversary determines if specific data was in the training set | Model confidence distribution, overfitting, shadow model availability |108| Model inversion | Adversary reconstructs input features from model outputs | Output granularity, model type, auxiliary information available |109| Attribute inference | Model reveals sensitive attributes not provided as input | Correlations in training data, feature interactions |110| Emergent bias amplification | Model amplifies biases present in training data, producing discriminatory outcomes | Training data representativeness, debiasing measures applied |111| Concept drift discrimination | Model performance degrades unequally across demographic groups over time | Monitoring coverage, retraining frequency |112| Re-identification through AI output | Model outputs enable linking back to specific data subjects | Output specificity, population uniqueness, auxiliary data |113| Automated decision errors | Incorrect AI decisions causing material harm to data subjects | Model accuracy, error distribution across groups |114115#### 3.2 AI Risk Scoring Matrix116117Combine likelihood and severity using the EDPB-recommended matrix:118119```120 Negligible Limited Significant Maximum121Almost Certain Medium High Very High Very High122Likely Medium High High Very High123Possible Low Medium High High124Remote Low Low Medium High125```126127#### 3.3 AI Act Risk Level Integration128129Cross-reference GDPR risk assessment with AI Act classification:130131- **Unacceptable risk** (Art. 5 AI Act): Processing must not proceed — social scoring, real-time biometric identification in public spaces (with limited exceptions)132- **High risk** (Art. 6 + Annex III): Enhanced DPIA obligations, conformity assessment required133- **Limited risk** (Art. 50): Transparency obligations — inform users they are interacting with AI134- **Minimal risk**: Standard DPIA process applies135136### Phase 4: AI-Specific Mitigation Measures137138#### Technical Measures139140| Measure | Risk Addressed | Implementation |141|---------|---------------|----------------|142| Differential privacy | Training data extraction, membership inference | Apply DP-SGD during training with calibrated epsilon (ε ≤ 8 for moderate protection, ε ≤ 1 for strong) |143| Federated learning | Data centralisation risk | Distribute training across data holders without centralising personal data |144| Model output perturbation | Model inversion, attribute inference | Add calibrated noise to model outputs, round confidence scores |145| Training data deduplication | Memorization risk | Remove duplicate and near-duplicate records before training |146| Membership inference testing | Membership inference | Run MI attacks against the model pre-deployment; retrain if leakage exceeds threshold |147| Fairness constraints | Bias amplification | Apply demographic parity, equalised odds, or calibration constraints during training |148| Input/output filtering | PII leakage in generative models | Deploy PII detection on model inputs and outputs with automated redaction |149| Model pruning and distillation | Memorization, extraction | Compress the model to reduce capacity for memorizing individual records |150151#### Organisational Measures152153- Establish an AI Ethics Review Board with privacy representation154- Implement model cards documenting privacy properties for each deployed model155- Conduct regular model audits (minimum annually) testing for privacy leakage156- Maintain training data provenance documentation and deletion capability157- Define retraining triggers and ensure DPIA review accompanies each retraining cycle158- Implement incident response procedures specific to AI privacy incidents159160### Phase 5: Human Oversight Assessment161162Per AI Act Art. 14 and GDPR Art. 22, assess the human oversight mechanism:163164| Oversight Element | Assessment Question |165|-------------------|-------------------|166| Meaningful review | Can the human reviewer effectively evaluate the AI recommendation and override it? |167| Time and resources | Is sufficient time allocated for meaningful review, or is the human a rubber stamp? |168| Competence | Does the reviewer have the expertise to identify AI errors? |169| Authority | Does the reviewer have the authority and means to override the AI? |170| Feedback mechanism | Are overrides recorded and fed back into model improvement? |171| Automation bias | Are measures in place to mitigate the tendency to defer to the AI? |172173## Prior Consultation Triggers for AI174175Art. 36 prior consultation with the supervisory authority is required when:1761771. The AI system produces high residual risk after all mitigation measures1782. The AI system processes special category data at scale with novel techniques1793. The supervisory authority's Art. 35(4) list specifically includes the AI use case1804. The AI system is deployed for real-time biometric identification under AI Act Art. 5 exceptions181182## Enforcement Precedents183184- **Clearview AI (Multiple DPAs, 2021-2024)**: Fines totalling over EUR 90 million across Italy (EUR 20M), France (EUR 20M), UK (GBP 7.5M), Greece (EUR 20M) for facial recognition AI deployed without DPIA, lawful basis, or transparency185- **CNIL v. Clearview AI (SAN-2022-019)**: Specific finding that no DPIA was conducted for biometric AI processing186- **Italian DPA v. Replika (2023)**: Ordered cessation of AI chatbot processing due to inadequate age verification and failure to conduct DPIA for AI processing affecting minors187- **Spanish DPA v. CaixaBank (PS/00421/2020)**: EUR 6 million fine for automated credit scoring without adequate DPIA addressing algorithmic decision-making risks188- **Dutch DPA v. Tax Authority (2020)**: Finding that algorithmic fraud detection system (SyRI) lacked proportionality and adequate DPIA for AI-driven profiling189190## Integration Points191192- **ai-training-lawfulness**: Detailed lawful basis analysis for training data feeds into Phase 2193- **ai-automated-decisions**: Art. 22 assessment integrates with Phase 5 human oversight194- **ai-model-privacy-audit**: Technical privacy testing results feed into Phase 3 risk assessment195- **ai-act-high-risk-docs**: AI Act conformity assessment aligns with Phase 3.3 risk classification196- **ai-bias-special-category**: Bias assessment results feed into risk scoring for discrimination harms
mukul975