How do I install the Azure Data Factory skill?

Run npx skillmds@latest add comeonoliver/azure-data-factory in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Azure Data Factory skill do?

Azure Data Factory Skill It is listed under DevOps & Infra on SkillMD.

Is Azure Data Factory safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Azure Data Factory?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Azure Data Factory free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Azure Data Factory?

ComeOnOliver (@comeonoliver) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Azure Data Factory by comeonoliver — Agent Skill

Name: Azure Data Factory
Author: ComeOnOliver

Azure Data Factory Skill

This skill provides expert guidance for Azure Data Factory. Covers troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. It combines local quick-reference content with remote documentation fetching capabilities.

How to Use This Skill

IMPORTANT for Agent: Use the Category Index below to locate relevant sections. For categories with line ranges (e.g., L35-L120), use read_file with the specified lines. For categories with file links (e.g., [security.md](security.md)), use read_file on the linked reference file

IMPORTANT for Agent: If metadata.generated_at is more than 3 months old, suggest the user pull the latest version from the repository. If mcp_microsoftdocs tools are not available, suggest the user install it: Installation Guide

This skill requires network access to fetch documentation content:

Preferred: Use mcp_microsoftdocs:microsoft_docs_fetch with query string from=learn-agent-skill. Returns Markdown.
Fallback: Use fetch_webpage with query string from=learn-agent-skill&accept=text/markdown. Returns Markdown.

Category Index

Category	Lines	Description
Troubleshooting	L37-L90	Diagnosing and fixing ADF issues: connector and format errors, pipeline/orchestration and data flow failures, CI/CD and Studio problems, SHIR/SSIS IR troubleshooting, and performance/debug guidance.
Best Practices	L91-L116	Performance, reliability, and migration best practices for ADF: tuning data flows/IRs, optimizing copy/source/sink, handling schema drift/errors, DataOps/BCDR, SAP CDC, and common migration patterns.
Decision Making	L117-L137	Guidance on ADF cost planning, pricing models, runtimes and compute choices, connector selection and upgrades, and assessing/migrating pipelines, SSIS, and data warehouses to ADF/Fabric
Architecture & Design Patterns	L138-L148	Patterns and architectures for ADF pipelines: efficient mapping data flows, metadata‑driven copy at scale, SSIS IR with SQL/MI/VNet, Cosmos DB migration, and SAP CDC design.
Limits & Quotas	L149-L154	Info on ADF connector lifecycle stages and timelines, plus how reservation discounts work for Mapping Data Flows and how they affect cost and capacity planning
Security	L155-L182	Securing Data Factory with identity, encryption, Key Vault, network isolation (Private Link, managed VNets, private endpoints), firewall rules, policies, and secure runtimes/SQL access.
Configuration	L183-L318	Configuring ADF and integration runtimes (Azure, self-hosted, SSIS, Airflow), datasets, activities, data flows, triggers, formats, CDC, monitoring, logging, networking, and parameters.
Integrations & Coding Patterns	L319-L502	Using ADF with external systems: connector how-tos, copy/transform patterns, SSIS/Databricks/Synapse/ML integration, REST/webhooks, SDK usage, and pipeline templates for common ETL scenarios.
Deployment	L503-L519	CI/CD and deployment for ADF and Workflow Orchestration Manager: ARM/linked templates, Azure DevOps pipelines, hotfix flows, pre/post scripts, and integration runtime install/containerization

Troubleshooting

Topic	URL
Resolve Azure Data Factory change data capture issues	https://learn.microsoft.com/en-us/azure/data-factory/change-data-capture-troubleshoot
Troubleshoot CI/CD, Azure DevOps, and GitHub integration for ADF	https://learn.microsoft.com/en-us/azure/data-factory/ci-cd-github-troubleshoot-guide
Resolve common Azure Data Factory connector upgrade issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-deprecation-frequently-asked-questions
Fix Azure Data Factory Amazon S3 connector errors	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-amazon-simple-storage-service
Resolve Azure Blob Storage connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-blob-storage
Troubleshoot Azure Cosmos DB connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-cosmos-db
Fix Azure Data Explorer connector problems in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-data-explorer
Resolve Azure Data Lake Storage connector errors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-data-lake
Fix Azure Files connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-files
Resolve Azure Table Storage connector errors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-table-storage
Troubleshoot DB2 connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-db2
Fix delimited text format connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-delimited-text
Resolve Dynamics 365 and Dataverse connector errors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-dynamics-dataverse
Troubleshoot file system connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-file-system
Fix FTP, SFTP, and HTTP connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-ftp-sftp-http
Troubleshoot Google Ads connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-google-ads
Troubleshoot Azure Data Factory connector failures	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-guide
Resolve Hive connector errors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-hive
Troubleshoot Microsoft Fabric Lakehouse connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-microsoft-fabric-lakehouse
Fix Microsoft Fabric Warehouse connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-microsoft-fabric-warehouse
Troubleshoot MongoDB connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-mongodb
Resolve Oracle connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-oracle
Fix ORC format connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-orc
Troubleshoot Parquet format connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-parquet
Troubleshoot Azure Database for PostgreSQL connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-postgresql
Resolve REST connector errors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-rest
Troubleshoot Salesforce and Service Cloud connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-salesforce
Fix SAP Table, BW Open Hub, and ODP connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-sap
Troubleshoot ServiceNow connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-servicenow
Resolve SharePoint Online list connector errors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-sharepoint-online-list
Troubleshoot Snowflake connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-snowflake
Troubleshoot SQL-based connectors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-synapse-sql
Fix Teradata connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-teradata
Troubleshoot XML format connector problems in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-xml
Troubleshoot Copy activity performance issues	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-troubleshooting
Troubleshoot external control activities in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-troubleshoot-guide
Diagnose and fix Azure Data Factory Studio issues	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-ux-troubleshoot-guide
Troubleshoot mapping data flow connector and format issues	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-connector-format
Resolve mapping data flow issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-guide
Iteratively debug Azure Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/iterative-development-debugging
Work around known Azure Data Factory issues	https://learn.microsoft.com/en-us/azure/data-factory/known-issues-troubleshoot-guide
Troubleshoot Azure Data Factory pipeline orchestration and triggers	https://learn.microsoft.com/en-us/azure/data-factory/pipeline-trigger-troubleshoot-guide
Send SHIR logs to debug SAP CDC connector issues	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-debug-shir-logs
Resolve security and access control issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/security-and-access-control-troubleshoot-guide
Use the self-hosted integration runtime diagnostic tool	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-diagnostic-tool
Diagnose and fix self-hosted IR issues in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-troubleshoot-guide
Use SSIS IR connectivity diagnostics to fix failures	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-diagnose-connectivity-faq
Resolve SSIS Integration Runtime management problems	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-management-troubleshoot
Troubleshoot SSIS package execution on SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-ssis-activity-faq
Interpret pipeline failure status and error messages in ADF	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-pipeline-failure-error-handling

Best Practices

Topic	URL
Apply DataOps practices to Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/apply-dataops
Use automatic connector upgrades in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/automatic-connector-upgrade
Use column patterns in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-column-pattern
Reuse logic with flowlets in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-flowlet
Tune mapping data flow performance in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance
Optimize sink performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-sinks
Optimize source performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-sources
Improve transformation performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-transformations
Handle schema drift in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-schema-drift
Tune integration runtime performance for data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime-performance
Use nested activities effectively in Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/concepts-nested-activities
Tune Azure-SSIS integration runtime for high performance	https://learn.microsoft.com/en-us/azure/data-factory/configure-azure-ssis-integration-runtime-performance
Optimize Copy activity performance and scalability	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance
Migrate on-premises HDFS data to Azure Storage	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-hdfs-azure-storage
Migrate Netezza data to Azure with Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-netezza-azure-sqldw
Migrate data from Amazon S3 to Azure Storage with ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-s3-azure-storage
Use data flow snippets to dedupe rows and find nulls	https://learn.microsoft.com/en-us/azure/data-factory/how-to-data-flow-dedupe-nulls-snippets
Handle SQL truncation and error rows in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/how-to-data-flow-error-rows
Implement BCDR strategies for Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/pipelines-disaster-recovery
Apply advanced patterns and best practices for SAP CDC	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-advanced-topics
Operate and manage SAP CDC pipelines in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-management
Apply best practices for writing data lake files with ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-data-flow-write-to-lake

Decision Making

Topic	URL
Estimate costs with Workflow Orchestration Manager pricing	https://learn.microsoft.com/en-us/azure/data-factory/airflow-pricing
Interpret pricing examples for ADF integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/better-understand-different-integration-runtime-charges
Choose the right integration runtime architecture	https://learn.microsoft.com/en-us/azure/data-factory/choose-the-right-integration-runtime-configuration
Choose compute environments for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/compute-linked-services
Decide when to use Workflow Orchestration Manager	https://learn.microsoft.com/en-us/azure/data-factory/concepts-workflow-orchestration-manager
Use connector upgrade advisor for ADF and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/connector-upgrade-advisor
Plan and execute Azure Data Factory connector upgrades	https://learn.microsoft.com/en-us/azure/data-factory/connector-upgrade-guidance
Buy and use ADF data flow reserved capacity	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-reserved-capacity-overview
Plan data lake and warehouse migration with ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-overview
Answer common Azure Data Factory usage questions	https://learn.microsoft.com/en-us/azure/data-factory/frequently-asked-questions
Assess Azure Data Factory pipelines for Fabric migration	https://learn.microsoft.com/en-us/azure/data-factory/how-to-assess-your-azure-data-factory-to-fabric-data-factory-migration
Choose and provision Enterprise Edition for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-azure-ssis-ir-enterprise-edition
Assess and upgrade Azure Data Factory pipelines to Fabric	https://learn.microsoft.com/en-us/azure/data-factory/how-to-upgrade-your-azure-data-factory-pipelines-to-fabric-data-factory
Choose the right Azure Data Factory SAP connector	https://learn.microsoft.com/en-us/azure/data-factory/industry-sap-connectors
Plan and manage Azure Data Factory costs	https://learn.microsoft.com/en-us/azure/data-factory/plan-manage-costs
Plan migration of on-premises SSIS workloads to ADF	https://learn.microsoft.com/en-us/azure/data-factory/scenario-ssis-migration-overview
Apply migration assessment rules for SSIS to ADF	https://learn.microsoft.com/en-us/azure/data-factory/scenario-ssis-migration-rules

Architecture & Design Patterns

Topic	URL
Design efficient pipelines using mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-pipelines
Configure Azure-SSIS IR with SQL failover groups	https://learn.microsoft.com/en-us/azure/data-factory/configure-bcdr-azure-ssis-integration-runtime
Build metadata-driven large-scale copy pipelines	https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool-metadata-driven
Migrate Azure SQL schemas to Azure Cosmos DB with ADF	https://learn.microsoft.com/en-us/azure/data-factory/how-to-sqldb-to-cosmosdb
Use Azure SQL Managed Instance with Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-sql-managed-instance-with-ir
Decide when and how to join Azure-SSIS integration runtime to a virtual network	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network
Understand architecture of SAP CDC in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-introduction-architecture

Limits & Quotas

Topic	URL
Understand connector release stages and timelines in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-release-stages-and-timelines
Understand reservation discounts for ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-understand-reservation-charges

Security

Topic	URL
Allow Azure Integration Runtime IP ranges in firewalls	https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses
Configure roles and permissions for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-roles-permissions
Configure outbound FQDN allow lists with Azure Policy for ADF	https://learn.microsoft.com/en-us/azure/data-factory/configure-outbound-allow-list-azure-policy
Use Azure credentials and secrets in ADF	https://learn.microsoft.com/en-us/azure/data-factory/credentials
Plan secure data access strategies for ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-access-strategies
Configure Azure Private Link for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-private-link
Configure managed identities for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-service-identity
Security infrastructure and data movement in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-movement-security-considerations
Enable Microsoft Entra authentication for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/enable-aad-authentication-azure-ssis-ir
Use Azure Key Vault as secrets backend for Airflow	https://learn.microsoft.com/en-us/azure/data-factory/enable-azure-key-vault
Enable customer-managed keys (BYOK) for ADF encryption	https://learn.microsoft.com/en-us/azure/data-factory/enable-customer-managed-key
Encrypt on-premises credentials for self-hosted IR	https://learn.microsoft.com/en-us/azure/data-factory/encrypt-credentials-self-hosted-integration-runtime
Access secured Purview from Data Factory via private endpoints	https://learn.microsoft.com/en-us/azure/data-factory/how-to-access-secured-purview-account
Use Azure Key Vault secrets in ADF pipeline activities	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-azure-key-vault-secrets-pipeline-activities
Use managed virtual networks and private endpoints in ADF	https://learn.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint
Apply Azure Policy built-ins for Data Factory governance	https://learn.microsoft.com/en-us/azure/data-factory/policy-reference
Secure Azure Data Factory with network and identity controls	https://learn.microsoft.com/en-us/azure/data-factory/secure-your-azure-data-factory
Detect and mask PII data with ADF template	https://learn.microsoft.com/en-us/azure/data-factory/solution-template-pii-detection-and-masking
Configure Windows authentication for SSIS packages in Azure	https://learn.microsoft.com/en-us/azure/data-factory/ssis-azure-connect-with-windows-auth
Store ADF credentials securely in Azure Key Vault	https://learn.microsoft.com/en-us/azure/data-factory/store-credentials-in-key-vault
Use private endpoints to securely copy data with ADF	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-portal-private
Secure self-hosted integration runtime with TLS from intranet	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-enable-remote-access-intranet-tls-ssl-certificate
Secure on-prem SQL access via Data Factory managed VNet	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-on-premise-sql-server
Configure private endpoint access to SQL Managed Instance from Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-sql-managed-instance

Configuration

Topic	URL
Reference supported Apache Airflow configurations in ADF	https://learn.microsoft.com/en-us/azure/data-factory/airflow-configurations
Retrieve IP address of Workflow Orchestration Manager cluster	https://learn.microsoft.com/en-us/azure/data-factory/airflow-get-ip-airflow-cluster
Install private Python packages in Airflow runtime	https://learn.microsoft.com/en-us/azure/data-factory/airflow-install-private-package
Define and use global parameters in ADF	https://learn.microsoft.com/en-us/azure/data-factory/author-global-parameters
Use the Azure Data Factory management hub	https://learn.microsoft.com/en-us/azure/data-factory/author-management-hub
Configure express virtual network injection for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-express-virtual-network-injection
Manage SSIS packages with Azure-SSIS package store	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-package-store
Configure standard virtual network injection for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-standard-virtual-network-injection
Configure virtual network for Azure-SSIS integration runtime injection	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-virtual-network-configuration
Reference built-in and preinstalled components on Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/built-in-preinstalled-components-ssis-integration-runtime
Configure change data capture resources in ADF	https://learn.microsoft.com/en-us/azure/data-factory/concepts-change-data-capture-resource
Define and configure datasets in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-datasets-linked-services
Configure integration runtimes in Data Factory and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime
Configure linked services in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-linked-services
Configure Append Variable activity in pipelines	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-append-variable-activity
Configure Data Flow activity execution	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-data-flow-activity
Configure Execute Pipeline activity chaining	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-pipeline-activity
Use Azure Data Factory expression language and functions	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions
Use Fail activity to raise custom errors	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-fail-activity
Configure Filter activity for pipeline arrays	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-filter-activity
Configure ForEach looping in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-for-each-activity
Use Get Metadata activity on datasets	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-get-metadata-activity
Configure If Condition branching in pipelines	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-if-condition-activity
Configure Lookup activity for dynamic datasets	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity
Use Power Query activity for data wrangling	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-power-query-activity
Use Set Variable activity and return values	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-set-variable-activity
Configure Switch activity for case routing	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-switch-activity
Use system variables in Azure Data Factory expressions	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-system-variables
Configure Until looping with timeouts	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-until-activity
Configure Validation activity for datasets	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-validation-activity
Configure Wait activity for pipeline delays	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-wait-activity
Enable data consistency verification in Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-data-consistency
Configure fault tolerance for Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-fault-tolerance
Configure session logging for Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-log
Monitor Copy activity runs and metrics	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-monitoring
Configure Copy activity in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview
Use Copy activity performance optimization features	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-features
Preserve metadata and ACLs in Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-preserve-metadata
Configure schema and data type mapping in Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping
Use Copy Data tool for guided ingestion	https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool
Configure an Apache Airflow environment in ADF	https://learn.microsoft.com/en-us/azure/data-factory/create-airflow-environment
Create and configure Azure integration runtime for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-integration-runtime
Provision Azure-SSIS integration runtime in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime
Create Azure-SSIS integration runtime via Azure portal	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-portal
Create Azure-SSIS integration runtime with Azure PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-powershell
Create and configure self-hosted integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/create-self-hosted-integration-runtime
Create shared self-hosted integration runtime with PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/create-shared-self-hosted-integration-runtime-powershell
Configure cross-tenant Azure DevOps connections for ADF	https://learn.microsoft.com/en-us/azure/data-factory/cross-tenant-connections-to-azure-devops
Configure Aggregate transformation in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-aggregate
Configure Alter row transformation for upserts	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-alter-row
Set data quality assertions in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-assert
Configure Cast transformation for type changes	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-cast
Configure Conditional split transformation rules	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-conditional-split
Configure Derived column expressions in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-derived-column
Configure Exists transformation for row matching	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-exists
Configure Filter transformation for row selection	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-filter
Configure Flatten transformation for hierarchical data	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-flatten
Use Flowlet transformation for reusable logic	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-flowlet
Configure Join transformation for combining streams	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-join
Configure Lookup transformation for reference data	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-lookup
Configure multiple branches in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-new-branch
Configure Parse transformation for embedded documents	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-parse
Configure Pivot transformation for row-to-column	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-pivot
Configure Rank transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-rank
Author and manage mapping data flow script in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-script
Configure Select transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-select
Configure sink transformation options in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-sink
Configure Sort transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-sort
Configure source transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-source
Use Stringify transformation for complex types in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-stringify
Configure Surrogate Key transformation in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-surrogate-key
Configure Union transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-union
Configure Unpivot transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-unpivot
Configure Window transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-window
Deactivate activities in Azure Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/deactivate-activity
Configure Delete activity for file cleanup	https://learn.microsoft.com/en-us/azure/data-factory/delete-activity
Configure diagnostics logs and metrics for Airflow IR	https://learn.microsoft.com/en-us/azure/data-factory/diagnostic-logs-and-metrics-for-workflow-orchestration-manager
Configure Avro format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-avro
Configure Binary format handling in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-binary
Configure Common Data Model format in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-common-data-model
Configure delimited text datasets in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-delimited-text
Configure Delta Lake format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-delta
Configure Excel file format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-excel
Configure Iceberg table format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-iceberg
Configure JSON file format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-json
Configure ORC format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-orc
Configure Parquet format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-parquet
Configure XML format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-xml
Configure Bring Your Own Driver connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/guidance-for-byod-approach
Configure CDC resource from ADLS Gen2 to Azure SQL	https://learn.microsoft.com/en-us/azure/data-factory/how-to-change-data-capture-resource
Configure CDC with schema evolution to Delta sinks	https://learn.microsoft.com/en-us/azure/data-factory/how-to-change-data-capture-resource-with-schema-evolution
Customize Azure-SSIS integration runtime setup with additional components	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-azure-ssis-ir-custom-setup
Configure self-hosted integration runtime for Log Analytics	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-shir-for-log-analytics-collection
Configure custom Event Grid triggers for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-custom-event-trigger
Create storage event-based triggers in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger
Configure schedule triggers in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger
Configure tumbling window triggers in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger
Develop and install licensed components for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/how-to-develop-azure-ssis-ir-licensed-components
Use parameters and expressions in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/how-to-expression-language-functions
Process fixed-width text files with ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/how-to-fixed-width
Manage Azure Data Factory studio settings and preferences	https://learn.microsoft.com/en-us/azure/data-factory/how-to-manage-settings
Manage preview features in Azure Data Factory studio	https://learn.microsoft.com/en-us/azure/data-factory/how-to-manage-studio-preview-exp
Schedule start and stop of Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/how-to-schedule-azure-ssis-integration-runtime
Use trigger metadata parameters in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-trigger-parameterization
Join Azure-SSIS integration runtime to a virtual network via PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network-powershell
Join Azure-SSIS integration runtime to a virtual network via portal	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network-ui
Configure Kubernetes secret for private container images	https://learn.microsoft.com/en-us/azure/data-factory/kubernetes-secret-pull-image-from-private-container-registry
Reconfigure Azure-SSIS integration runtime settings in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/manage-azure-ssis-integration-runtime
Configure diagnostic settings and Log Analytics for ADF	https://learn.microsoft.com/en-us/azure/data-factory/monitor-configure-diagnostics
Monitor Azure Data Factory with Azure Monitor	https://learn.microsoft.com/en-us/azure/data-factory/monitor-data-factory
Use monitoring reference data for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/monitor-data-factory-reference
Configure monitoring for Azure Data Factory integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/monitor-integration-runtime
Monitor integration runtimes in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/monitor-integration-runtime
Set up ADF diagnostic logs via Azure Monitor REST API	https://learn.microsoft.com/en-us/azure/data-factory/monitor-logs-rest
Monitor managed virtual network integration runtimes in ADF	https://learn.microsoft.com/en-us/azure/data-factory/monitor-managed-virtual-network-integration-runtime
Monitor Azure VMs hosting self-hosted integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/monitor-shir-in-azure
Monitor SSIS operations in ADF using Azure Monitor	https://learn.microsoft.com/en-us/azure/data-factory/monitor-ssis
Visually monitor pipelines and activities in ADF	https://learn.microsoft.com/en-us/azure/data-factory/monitor-visually
Apply naming rules for Azure Data Factory artifacts	https://learn.microsoft.com/en-us/azure/data-factory/naming-rules
Parameterize linked services in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/parameterize-linked-services
Parameterize mapping data flows in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/parameters-data-flow
Change user passwords in Workflow Orchestration Manager	https://learn.microsoft.com/en-us/azure/data-factory/password-change-airflow
Create SAP CDC linked services and datasets	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-prepare-linked-service-source-dataset
Configure prerequisites for the SAP CDC connector	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-prerequisites-configuration
Prepare self-hosted IR for SAP CDC workloads	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-shir-preparation
Configure autoupdate and version management for self-hosted IR	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-auto-update
Configure self-hosted integration runtime as proxy for Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-run

…(truncated)

Azure Data Factory

File contents

Azure Data Factory Skill

How to Use This Skill

Category Index

Troubleshooting

Best Practices

Decision Making

Architecture & Design Patterns

Limits & Quotas

Security

Configuration

Frequently asked questions

Azure Data Factory

File contents

Azure Data Factory Skill

How to Use This Skill

Category Index

Troubleshooting

Best Practices

Decision Making

Architecture & Design Patterns

Limits & Quotas

Security

Configuration

Related skills

Frequently asked questions

How do I install the Azure Data Factory skill?

What does the Azure Data Factory skill do?

Is Azure Data Factory safe to use?

Which AI agents work with Azure Data Factory?

Is Azure Data Factory free to use?

Who published Azure Data Factory?