How do I install the Azure Data Factory skill?

Run npx skillmds@latest add microsoftdocs/azure-data-factory in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Azure Data Factory skill do?

Expert knowledge for Azure Data Factory development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when building ADF pipelines, mapping data flows, SSIS IR/SHIR, CI/CD deployments, or SAP/SaaS integrations, and other Azure Data Factory related development tasks. Not for Azure Data Explorer (use azure-data-explorer), Azure Databricks (use azure-databricks), Azure Synapse Analytics (use azure-synapse-analytics), Azure Stream Analytics (use azure-stream-analytics). It is listed under DevOps & Infra on SkillMD.

Is Azure Data Factory safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Azure Data Factory?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Azure Data Factory free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Azure Data Factory?

microsoftdocs (@microsoftdocs) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Azure Data Factory by microsoftdocs

Name: Azure Data Factory
Rating: 4.2 (5 reviews)
Author: microsoftdocs

Azure Data Factory Skill

This skill provides expert guidance for Azure Data Factory. Covers troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. It combines local quick-reference content with remote documentation fetching capabilities.

How to Use This Skill

IMPORTANT for Agent: Use the Category Index below to locate relevant sections. For categories with line ranges (e.g., L35-L120), use read_file with the specified lines. For categories with file links (e.g., [security.md](security.md)), use read_file on the linked reference file

IMPORTANT for Agent: If metadata.generated_at is more than 3 months old, suggest the user pull the latest version from the repository. If mcp_microsoftdocs tools are not available, suggest the user install it: Installation Guide

This skill requires network access to fetch documentation content:

Preferred: Use mcp_microsoftdocs:microsoft_docs_fetch with query string from=learn-agent-skill. Returns Markdown.
Fallback: Use fetch_webpage with query string from=learn-agent-skill&accept=text/markdown. Returns Markdown.

Category Index

Category	Lines	Description
Troubleshooting	L37-L91	Diagnosing and fixing ADF failures: connector/format errors, CDC, pipelines, triggers, data flows, SHIR/SSIS IR, performance, security, and known issues with concrete error patterns and workarounds.
Best Practices	L92-L114	Performance tuning, schema drift/error handling, reusable patterns, and migration best practices for Data Factory mapping data flows, copy activity, runtimes, and data lake writes.
Decision Making	L115-L133	Guidance on cost planning, connector upgrades, integration runtime and compute choices, and migration/modernization of ADF, SSIS, Synapse, and pipelines to Fabric.
Architecture & Design Patterns	L134-L142	Patterns and architectures for ADF: efficient mapping data flows, metadata‑driven copy, SSIS IR with SQL MI/failover/VNet, Cosmos DB migration, and SAP CDC design.
Limits & Quotas	L143-L150	Configuring Until/Wait activity limits and timeouts, understanding data flow reservation discounts, and reviewing ADF service quotas and capacity constraints.
Security	L151-L182	Securing Data Factory: auth, roles, encryption, Key Vault, managed identities, VNets/private endpoints, firewall/Policy configs, and secure access to SQL, Purview, and SSIS IR.
Configuration	L183-L294	Configuring ADF: integration runtimes, SSIS/SHIR, triggers, monitoring/logging, copy/data flows, formats/connectors, parameters/expressions, and pipeline/control activities.
Integrations & Coding Patterns	L295-L463	Patterns and how-tos for connecting ADF to many data sources (SAP, SaaS apps, DBs, files), using mapping data flow functions, running SSIS, Databricks/ML/Synapse jobs, and pipeline templates.
Deployment	L464-L477	CI/CD and deployment for ADF: ARM/linked templates, Azure DevOps pipelines, hotfix flows, pre/post scripts, SSIS job migration, and integration runtime install/migration.

Troubleshooting

Topic	URL
Troubleshoot Azure Data Factory change data capture	https://learn.microsoft.com/en-us/azure/data-factory/change-data-capture-troubleshoot
Troubleshoot CI/CD, Azure DevOps, and GitHub integration for ADF	https://learn.microsoft.com/en-us/azure/data-factory/ci-cd-github-troubleshoot-guide
Resolve common Azure Data Factory connector upgrade issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-deprecation-frequently-asked-questions
Diagnose and fix Azure Data Factory Amazon S3 connector errors	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-amazon-simple-storage-service
Fix Azure Blob Storage connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-blob-storage
Diagnose and fix Azure Cosmos DB connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-cosmos-db
Resolve Azure Data Explorer connector problems in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-data-explorer
Troubleshoot Azure Data Lake Storage connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-data-lake
Resolve Azure Data Factory Azure Files connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-files
Troubleshoot Azure Data Factory Azure Table Storage connector	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-table-storage
Resolve DB2 connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-db2
Fix delimited text connector errors in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-delimited-text
Resolve Dynamics 365 and Dataverse connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-dynamics-dataverse
Troubleshoot file system connector in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-file-system
Troubleshoot FTP, SFTP, and HTTP connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-ftp-sftp-http
Fix common Azure Data Factory Google Ads connector problems	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-google-ads
Diagnose and fix Azure Data Factory connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-guide
Diagnose Azure Data Factory Hive connector connection and query errors	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-hive
Troubleshoot Microsoft Fabric Lakehouse connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-microsoft-fabric-lakehouse
Fix Microsoft Fabric Warehouse connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-microsoft-fabric-warehouse
Resolve Azure Data Factory MongoDB connector errors	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-mongodb
Resolve Oracle connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-oracle
Troubleshoot ORC format connector in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-orc
Fix Parquet format connector issues in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-parquet
Fix Azure Database for PostgreSQL connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-postgresql
Diagnose REST connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-rest
Resolve Salesforce and Service Cloud connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-salesforce
Resolve SAP connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-sap
Fix ServiceNow connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-servicenow
Fix SharePoint Online list connector problems in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-sharepoint-online-list
Resolve Shopify connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-shopify
Troubleshoot Snowflake connector in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-snowflake
Troubleshoot SQL-based connectors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-synapse-sql
Fix Teradata connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-teradata
Troubleshoot Azure Data Factory XML format connector parsing and schema issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-xml
Use Fail activity to surface custom errors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-fail-activity
Troubleshoot Copy activity performance issues	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-troubleshooting
Troubleshoot external control activities in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-troubleshoot-guide
Diagnose and fix Azure Data Factory Studio issues	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-ux-troubleshoot-guide
Troubleshoot mapping data flow connector and format issues	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-connector-format
Resolve mapping data flow issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-guide
Work around known Azure Data Factory issues	https://learn.microsoft.com/en-us/azure/data-factory/known-issues-troubleshoot-guide
Troubleshoot Azure Data Factory pipeline orchestration and triggers	https://learn.microsoft.com/en-us/azure/data-factory/pipeline-trigger-troubleshoot-guide
Troubleshoot Azure Data Factory SAP CDC via SHIR logs	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-debug-shir-logs
Resolve security and access control issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/security-and-access-control-troubleshoot-guide
Use the self-hosted integration runtime diagnostic tool	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-diagnostic-tool
Troubleshoot self-hosted integration runtime issues	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-troubleshoot-guide
Troubleshoot SSIS IR connectivity with diagnose feature	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-diagnose-connectivity-faq
Diagnose and fix SSIS IR management issues	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-management-troubleshoot
Troubleshoot SSIS package execution on SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-ssis-activity-faq
Interpret pipeline failure status and error messages in ADF	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-pipeline-failure-error-handling

Best Practices

Topic	URL
Use column patterns in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-column-pattern
Reuse logic with flowlets in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-flowlet
Tune mapping data flow performance in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance
Optimize sink performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-sinks
Optimize source performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-sources
Improve transformation performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-transformations
Handle schema drift in Data Factory mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-schema-drift
Tune integration runtime performance for data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime-performance
Tune Azure-SSIS integration runtime for high performance	https://learn.microsoft.com/en-us/azure/data-factory/configure-azure-ssis-integration-runtime-performance
Optimize Azure Data Factory copy activity performance	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance
Optimize Azure Data Factory copy activity performance	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-features
Migrate on-premises HDFS data to Azure Storage	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-hdfs-azure-storage
Migrate Netezza data to Azure with Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-netezza-azure-sqldw
Migrate data from Amazon S3 to Azure Storage with ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-s3-azure-storage
Dedupe rows and detect nulls using ADF data flow snippets	https://learn.microsoft.com/en-us/azure/data-factory/how-to-data-flow-dedupe-nulls-snippets
Handle error rows and truncation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/how-to-data-flow-error-rows
Implement BCDR strategies for Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/pipelines-disaster-recovery
Apply advanced SAP CDC features and best practices	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-advanced-topics
Apply data flow best practices for writing to data lake	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-data-flow-write-to-lake

Decision Making

Topic	URL
Use automatic connector upgrades in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/automatic-connector-upgrade
Interpret pricing examples for ADF integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/better-understand-different-integration-runtime-charges
Choose the right integration runtime architecture	https://learn.microsoft.com/en-us/azure/data-factory/choose-the-right-integration-runtime-configuration
Choose compute environments for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/compute-linked-services
Use connector upgrade advisor for ADF and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/connector-upgrade-advisor
Decide how and when to upgrade ADF connectors	https://learn.microsoft.com/en-us/azure/data-factory/connector-upgrade-guidance
Buy and use ADF data flow reserved capacity for cost savings	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-reserved-capacity-overview
Plan data lake and EDW migration to Azure with ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-overview
Assess Azure Data Factory pipelines for Fabric migration	https://learn.microsoft.com/en-us/azure/data-factory/how-to-assess-your-azure-data-factory-to-fabric-data-factory-migration
Choose and provision Enterprise Edition Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-azure-ssis-ir-enterprise-edition
Upgrade Azure Data Factory pipelines to Fabric Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-upgrade-your-azure-data-factory-pipelines-to-fabric-data-factory
Modernize Synapse pipelines with Fabric Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-upgrade-your-azure-synapse-analytics-pipelines-to-fabric-data-factory
Plan and manage Azure Data Factory costs	https://learn.microsoft.com/en-us/azure/data-factory/plan-manage-costs
Assess and plan SSIS migration to Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/scenario-ssis-migration-overview
Apply SSIS migration assessment rules for ADF/Synapse	https://learn.microsoft.com/en-us/azure/data-factory/scenario-ssis-migration-rules

Architecture & Design Patterns

Topic	URL
Design efficient pipelines using mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-pipelines
Configure Azure-SSIS IR with SQL failover groups	https://learn.microsoft.com/en-us/azure/data-factory/configure-bcdr-azure-ssis-integration-runtime
Build metadata-driven large-scale copy pipelines	https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool-metadata-driven
Migrate Azure SQL schemas to Azure Cosmos DB with ADF	https://learn.microsoft.com/en-us/azure/data-factory/how-to-sqldb-to-cosmosdb
Decide when and how to join Azure-SSIS integration runtime to a virtual network	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network

Limits & Quotas

Topic	URL
Configure Until activity loops and timeouts in ADF	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-until-activity
Configure Wait activity durations in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-wait-activity
Understand reservation discounts for ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-understand-reservation-charges
Review Azure Data Factory limits and quotas from FAQ	https://learn.microsoft.com/en-us/azure/data-factory/frequently-asked-questions

Security

Topic	URL
Allow Azure Integration Runtime IP ranges through firewalls	https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses
Configure VNet injection for Azure-SSIS IR securely	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-virtual-network-configuration
Configure roles and permissions for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-roles-permissions
Configure outbound FQDN allow lists with Azure Policy for ADF	https://learn.microsoft.com/en-us/azure/data-factory/configure-outbound-allow-list-azure-policy
Use Azure credentials and managed identities in ADF	https://learn.microsoft.com/en-us/azure/data-factory/credentials
Plan secure data access strategies for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-access-strategies
Configure Azure Private Link for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-private-link
Configure managed identities for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-service-identity
Security infrastructure and data movement in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-movement-security-considerations
Enable Microsoft Entra authentication for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/enable-aad-authentication-azure-ssis-ir
Enable customer-managed keys (BYOK) for ADF encryption	https://learn.microsoft.com/en-us/azure/data-factory/enable-customer-managed-key
Encrypt credentials for self-hosted integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/encrypt-credentials-self-hosted-integration-runtime
Access secured Purview from Data Factory via private endpoints	https://learn.microsoft.com/en-us/azure/data-factory/how-to-access-secured-purview-account
Use Azure Key Vault secrets in ADF pipeline activities	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-azure-key-vault-secrets-pipeline-activities
Join Azure-SSIS IR to VNet using PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network-powershell
Join Azure-SSIS IR to VNet via portal	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network-ui
Secure Azure Data Factory with managed VNets and private endpoints	https://learn.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint
Configure modern trusted service firewall bypass in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/modern-trusted-service
Use built-in Azure Policy definitions for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/policy-reference
Implement security controls for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/secure-your-azure-data-factory
Detect and mask PII data with ADF solution template	https://learn.microsoft.com/en-us/azure/data-factory/solution-template-pii-detection-and-masking
Configure Windows authentication for SSIS packages in Azure	https://learn.microsoft.com/en-us/azure/data-factory/ssis-azure-connect-with-windows-auth
Store ADF credentials securely in Azure Key Vault	https://learn.microsoft.com/en-us/azure/data-factory/store-credentials-in-key-vault
Use private endpoints to securely copy data with ADF	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-portal-private
Configure Azure-SSIS integration runtime in virtual network	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-deploy-ssis-virtual-network
Secure self-hosted integration runtime with TLS from intranet	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-enable-remote-access-intranet-tls-ssl-certificate
Secure on-prem SQL access via Data Factory managed VNet	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-on-premise-sql-server
Configure private endpoint access to SQL Managed Instance from Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-sql-managed-instance

Configuration

Topic	URL
Define and use global parameters in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/author-global-parameters
Configure connections and global settings in ADF management hub	https://learn.microsoft.com/en-us/azure/data-factory/author-management-hub
Configure express virtual network injection for Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-express-virtual-network-injection
Manage SSIS packages with Azure-SSIS package store	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-package-store
Configure standard virtual network injection for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-standard-virtual-network-injection
Use built-in components on Azure-SSIS Integration Runtime	https://learn.microsoft.com/en-us/azure/data-factory/built-in-preinstalled-components-ssis-integration-runtime
Configure integration runtimes for Azure Data Factory and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime
Define and configure linked services in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-linked-services
Configure Azure Data Factory Oracle connector settings	https://learn.microsoft.com/en-us/azure/data-factory/connector-oracle
Configure custom ARM template parameters for ADF CI/CD	https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-resource-manager-custom-parameters
Configure Data Flow activity execution	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-data-flow-activity
Use Azure Data Factory expression language and functions	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions
Configure ForEach looping activity in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-for-each-activity
Configure Get Metadata activity for ADF datasets	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-get-metadata-activity
Set up If Condition activity in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-if-condition-activity
Use Power Query activity for data wrangling	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-power-query-activity
Configure Set Variable and pipeline return values	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-set-variable-activity
Configure Switch activity for conditional routing	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-switch-activity
Use system variables in Azure Data Factory expressions	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-system-variables
Configure Validation activity for dataset readiness in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-validation-activity
Enable data consistency verification in ADF copy	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-data-consistency
Configure fault tolerance for Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-fault-tolerance
Configure session logging for Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-log
Configure monitoring for ADF copy activities	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-monitoring
Configure Copy activity in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview
Preserve metadata and ACLs in ADF copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-preserve-metadata
Configure schema and data type mapping in ADF copy	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping
Use the Copy Data tool in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool
Create and configure Azure integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-integration-runtime
Provision Azure-SSIS integration runtime in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime
Create Azure-SSIS integration runtime via Azure portal	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-portal
Create Azure-SSIS integration runtime via PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-powershell
Deploy Azure-SSIS IR using ARM templates	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-resource-manager-template
Create shared self-hosted IR with PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/create-shared-self-hosted-integration-runtime-powershell
Configure cross-tenant Azure DevOps connections for ADF	https://learn.microsoft.com/en-us/azure/data-factory/cross-tenant-connections-to-azure-devops
Configure Aggregate transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-aggregate
Configure Alter row transformation for database updates	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-alter-row
Set and use Assert transformation in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-assert
Use Cast transformation to change column data types	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-cast
Configure Conditional split transformation in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-conditional-split
Configure Derived column transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-derived-column
Configure Exists transformation for row matching	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-exists
Configure Filter transformation to remove rows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-filter
Configure Flatten transformation for hierarchical data	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-flatten
Use Flowlet transformation to reuse data flow logic	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-flowlet
Configure Join transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-join
Configure Lookup transformation for reference data	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-lookup
Configure multiple branches in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-new-branch
Configure Parse transformation for embedded documents	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-parse
Configure Pivot transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-pivot
Configure Rank transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-rank
Author and configure mapping data flow script	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-script
Configure Select transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-select
Configure sink settings in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-sink
Configure Sort transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-sort
Configure source settings in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-source
Configure Stringify transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-stringify
Configure Surrogate Key transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-surrogate-key
Configure Union transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-union
Configure Unpivot transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-unpivot
Configure Window transformation in ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-window
Configure Avro format options in ADF and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/format-avro
Configure binary dataset format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-binary
Configure Common Data Model format in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-common-data-model
Configure delimited text datasets in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-delimited-text
Configure Delta Lake format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-delta
Configure Excel file format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-excel
Configure Iceberg table format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-iceberg
Configure JSON file format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-json
Configure ORC format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-orc
Configure Parquet format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-parquet
Configure XML format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-xml
Configure Bring Your Own Driver connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/guidance-for-byod-approach
Customize Azure-SSIS integration runtime setup with additional components	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-azure-ssis-ir-custom-setup
Configure SHIR for Azure Log Analytics collection	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-shir-for-log-analytics-collection
Configure custom Event Grid triggers for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-custom-event-trigger
Create storage event-based triggers in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger
Configure schedule triggers in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger
Configure tumbling window triggers in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger
Develop and install licensed SSIS IR components in ADF	https://learn.microsoft.com/en-us/azure/data-factory/how-to-develop-azure-ssis-ir-licensed-components
Use parameters and expression language in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-expression-language-functions
Manage Azure Data Factory studio settings and preferences	https://learn.microsoft.com/en-us/azure/data-factory/how-to-manage-settings
Control Azure Data Factory studio preview experience	https://learn.microsoft.com/en-us/azure/data-factory/how-to-manage-studio-preview-exp
Run self-hosted IR in Windows containers	https://learn.microsoft.com/en-us/azure/data-factory/how-to-run-self-hosted-integration-runtime-in-windows-container
Schedule start and stop of Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/how-to-schedule-azure-ssis-integration-runtime
Use trigger metadata parameters in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-trigger-parameterization
Reconfigure Azure-SSIS integration runtime settings	https://learn.microsoft.com/en-us/azure/data-factory/manage-azure-ssis-integration-runtime
Configure diagnostic settings and Log Analytics for ADF	https://learn.microsoft.com/en-us/azure/data-factory/monitor-configure-diagnostics
Use monitoring reference data for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/monitor-data-factory-reference
Configure monitoring for Azure Data Factory integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/monitor-integration-runtime
Monitor integration runtimes in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/monitor-integration-runtime
Set up ADF diagnostic logs via Azure Monitor REST API	https://learn.microsoft.com/en-us/azure/data-factory/monitor-logs-rest
Configure monitoring for managed VNet integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/monitor-managed-virtual-network-integration-runtime
Monitor SHIR Azure VMs with diagnostics	https://learn.microsoft.com/en-us/azure/data-factory/monitor-shir-in-azure
Monitor SSIS operations in ADF using Azure Monitor	https://learn.microsoft.com/en-us/azure/data-factory/monitor-ssis
Apply naming rules for Azure Data Factory artifacts	https://learn.microsoft.com/en-us/azure/data-factory/naming-rules
Parameterize linked services in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/parameterize-linked-services
Parameterize mapping data flows from ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/parameters-data-flow
Operate and configure SAP CDC processes in ADF	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-management
Set up SAP CDC linked service and dataset	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-prepare-linked-service-source-dataset
Configure prerequisites for Azure Data Factory SAP CDC	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-prerequisites-configuration
Configure self-hosted IR for SAP CDC connector	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-shir-preparation
Configure SHIR autoupdate and expiry notifications	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-auto-update
Configure self-hosted integration runtime as proxy for Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-proxy-ssis
Configure source control for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/source-control
Supported file formats and compression in ADF copy	https://learn.microsoft.com/en-us/azure/data-factory/supported-file-formats-and-compression-codecs
Configure tumbling window trigger dependencies in ADF	https://learn.microsoft.com/en-us/azure/data-factory/tumbling-window-trigger-dependency
Return values from child pipelines in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-pipeline-return-value

Integrations & Coding Patterns

Topic	URL
Configure SAP CDC connector to load data into Fabric OneLake	https://learn.microsoft.com/en-us/azure/data-factory/change-data-capture-from-sap-to-onelake-with-azure-data-factory
Use expression builder for ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-expression-builder
Connect Azure Data Factory to Microsoft Purview	https://learn.microsoft.com/en-us/azure/data-factory/connect-data-factory-to-azure-purview
Integrate Amazon Marketplace Web Service with Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-marketplace-web-service
Integrate Amazon RDS for Oracle with Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-rds-for-oracle
Copy data from Amazon RDS for SQL Server with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-rds-for-sql-server
Integrate Azure Data Factory with Amazon Redshift	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-redshift
Configure ADF Amazon S3-compatible storage connector	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-s3-compatible-storage
Copy and transform data in Amazon S3 with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-simple-storage-service
Transform AppFigures data with ADF Data Flows	https://learn.microsoft.com/en-us/azure/data-factory/connector-appfigures
Transform Asana data with ADF Data Flows	https://learn.microsoft.com/en-us/azure/data-factory/connector-asana
Copy and transform data in Azure Blob Storage with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage
Transform data in Cosmos DB analytical store via ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-analytical-store
Configure ADF copy activity with Azure Cosmos DB for NoSQL	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-db
Copy data to and from Azure Cosmos DB for MongoDB	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-db-mongodb-api
Copy and transform data in Azure Data Explorer with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-data-explorer
Copy and transform data in Azure Data Lake Storage Gen2	https://learn.microsoft.com/en-us/azure/data-factory/connector-az

…(truncated)

Azure Data Factory

File contents

Azure Data Factory Skill

How to Use This Skill

Category Index

Troubleshooting

Best Practices

Decision Making

Architecture & Design Patterns

Limits & Quotas

Security

Configuration

Integrations & Coding Patterns

Frequently asked questions

Azure Data Factory

File contents

Azure Data Factory Skill

How to Use This Skill

Category Index

Troubleshooting

Best Practices

Decision Making

Architecture & Design Patterns

Limits & Quotas

Security

Configuration

Integrations & Coding Patterns

Related skills

Frequently asked questions

How do I install the Azure Data Factory skill?

What does the Azure Data Factory skill do?

Is Azure Data Factory safe to use?

Which AI agents work with Azure Data Factory?

Is Azure Data Factory free to use?

Who published Azure Data Factory?