How do I install the Azure Data Factory skill?

Run npx skillmds@latest add luokai0/azure-data-factory in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Azure Data Factory skill do?

Expert knowledge for Azure Data Factory development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when designing ADF pipelines, Mapping Data Flows, SHIR/SSIS IR, secure VNet/Private Link setups, or CI/CD deployments, and other Azure Data Factory related development tasks. Not for Azure Synapse Analytics (use azure-synapse-analytics), Azure Databricks (use azure-databricks), Azure Stream Analytics (use azure-stream-analytics), Azure Data Explorer (use azure-data-explorer). It is listed under DevOps & Infra on SkillMD.

Is Azure Data Factory safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Azure Data Factory?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Azure Data Factory free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Azure Data Factory?

luokai0 (@luokai0) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Azure Data Factory by luokai0 — Agent Skill

Name: Azure Data Factory
Rating: 4.2 (5 reviews)
Author: luokai0

Azure Data Factory Skill

This skill provides expert guidance for Azure Data Factory. Covers troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. It combines local quick-reference content with remote documentation fetching capabilities.

How to Use This Skill

IMPORTANT for Agent: Use the Category Index below to locate relevant sections. For categories with line ranges (e.g., L35-L120), use read_file with the specified lines. For categories with file links (e.g., [security.md](security.md)), use read_file on the linked reference file

IMPORTANT for Agent: If metadata.generated_at is more than 3 months old, suggest the user pull the latest version from the repository. If mcp_microsoftdocs tools are not available, suggest the user install it: Installation Guide

This skill requires network access to fetch documentation content:

Preferred: Use mcp_microsoftdocs:microsoft_docs_fetch with query string from=learn-agent-skill. Returns Markdown.
Fallback: Use fetch_webpage with query string from=learn-agent-skill&accept=text/markdown. Returns Markdown.

Category Index

Category	Lines	Description
Troubleshooting	L37-L90	Diagnosing and fixing ADF issues: connector/format errors, copy & data flow performance, pipelines/triggers, SHIR/SSIS IR problems, security/access, and interpreting failure logs.
Best Practices	L91-L113	Performance, DataOps, and reliability best practices for ADF: tuning data flows/copy, optimizing sources/sinks/IR, handling schema drift/errors, and migrating data (HDFS, Netezza, S3) to Azure.
Decision Making	L114-L134	Guidance on ADF architecture, runtimes, connector upgrades, cost optimization, and migration/modernization of pipelines, SSIS, Synapse, and data warehouses to Azure/Fabric
Architecture & Design Patterns	L135-L145	Patterns and architectures for ADF: efficient mapping data flows, metadata‑driven copy, SSIS IR with SQL MI/failover/VNet, Cosmos DB migration, and SAP CDC design.
Limits & Quotas	L146-L150	Info on ADF connector lifecycle stages and timelines, plus how reservation discounts work for Mapping Data Flows and how they affect cost and capacity planning
Security	L151-L181	Securing Data Factory with identity, encryption, Key Vault, and Azure Policy, plus network controls like VNets, Private Link, firewalls, private endpoints, and secure runtimes (Azure-SSIS, self-hosted).
Configuration	L182-L295	Configuring ADF artifacts and runtime: pipelines, activities, data flows, triggers, copy behavior, formats, IRs (Azure/self-hosted/SSIS), monitoring, logging, parameters, and DevOps setup.
Integrations & Coding Patterns	L296-L482	Connector how-tos, patterns, and activities for copying/transforming data between ADF and many services (databases, SaaS apps, files, Fabric, Databricks, SSIS, ML, Synapse) using data flows and SDKs.
Deployment	L483-L496	CI/CD and deployment for ADF: ARM/linked templates, Azure DevOps pipelines, hotfix flows, pre/post scripts, IR automation, SSIS job migration, and networked IR migration.

Troubleshooting

Topic	URL
Troubleshoot Azure Data Factory change data capture	https://learn.microsoft.com/en-us/azure/data-factory/change-data-capture-troubleshoot
Troubleshoot CI/CD, Azure DevOps, and GitHub integration for ADF	https://learn.microsoft.com/en-us/azure/data-factory/ci-cd-github-troubleshoot-guide
Resolve common Azure Data Factory connector upgrade issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-deprecation-frequently-asked-questions
Resolve Azure Data Factory Amazon S3 connector errors	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-amazon-simple-storage-service
Fix Azure Blob Storage connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-blob-storage
Troubleshoot Azure Cosmos DB connectors in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-cosmos-db
Resolve Azure Data Explorer connector problems in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-data-explorer
Troubleshoot Azure Data Lake Storage connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-data-lake
Resolve Azure Files connector errors in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-files
Troubleshoot Azure Table Storage connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-table-storage
Fix DB2 connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-db2
Troubleshoot delimited text format connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-delimited-text
Resolve Dynamics 365 and Dataverse connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-dynamics-dataverse
Troubleshoot file system connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-file-system
Troubleshoot FTP, SFTP, and HTTP connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-ftp-sftp-http
Fix Google Ads connector errors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-google-ads
Troubleshoot Azure Data Factory connector failures	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-guide
Troubleshoot Hive connector issues in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-hive
Troubleshoot Microsoft Fabric Lakehouse connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-microsoft-fabric-lakehouse
Fix Microsoft Fabric Warehouse connector issues in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-microsoft-fabric-warehouse
Resolve MongoDB connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-mongodb
Resolve Oracle connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-oracle
Troubleshoot ORC format connector in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-orc
Fix Parquet format connector issues in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-parquet
Fix Azure Database for PostgreSQL connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-postgresql
Troubleshoot REST connector errors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-rest
Resolve Salesforce and Service Cloud connector issues	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-salesforce
Troubleshoot SAP Table and BW Open Hub connectors	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-sap
Fix ServiceNow connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-servicenow
Troubleshoot SharePoint Online list connector in ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-sharepoint-online-list
Resolve Shopify connector issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-shopify
Troubleshoot Snowflake connector errors in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-snowflake
Troubleshoot SQL-based connectors in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-synapse-sql
Fix Teradata connector failures in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-teradata
Fix XML format connector problems in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-xml
Troubleshoot Copy activity performance issues	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-troubleshooting
Troubleshoot external control activities in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-troubleshoot-guide
Diagnose and fix Azure Data Factory Studio issues	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-ux-troubleshoot-guide
Troubleshoot mapping data flow connector and format issues	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-connector-format
Resolve mapping data flow issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-guide
Work around known Azure Data Factory issues	https://learn.microsoft.com/en-us/azure/data-factory/known-issues-troubleshoot-guide
Troubleshoot Azure Data Factory pipeline orchestration and triggers	https://learn.microsoft.com/en-us/azure/data-factory/pipeline-trigger-troubleshoot-guide
Debug SAP CDC connector issues via SHIR logs	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-debug-shir-logs
Resolve security and access control issues in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/security-and-access-control-troubleshoot-guide
Use the self-hosted integration runtime diagnostic tool	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-diagnostic-tool
Diagnose and fix self-hosted IR issues in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-troubleshoot-guide
Troubleshoot SSIS IR connectivity with diagnose feature	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-diagnose-connectivity-faq
Diagnose and fix SSIS IR management issues	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-management-troubleshoot
Troubleshoot SSIS package execution on SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/ssis-integration-runtime-ssis-activity-faq
Interpret pipeline failure status and error messages in ADF	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-pipeline-failure-error-handling

Best Practices

Topic	URL
Apply DataOps practices to Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/apply-dataops
Use column patterns in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-column-pattern
Reuse logic with flowlets in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-flowlet
Tune mapping data flow performance in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance
Optimize sink performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-sinks
Optimize source performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-sources
Improve transformation performance in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-transformations
Handle schema drift in Data Factory mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-schema-drift
Tune integration runtime performance for data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime-performance
Tune Azure-SSIS integration runtime for high performance	https://learn.microsoft.com/en-us/azure/data-factory/configure-azure-ssis-integration-runtime-performance
Optimize Azure Data Factory copy activity performance	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance
Migrate on-premises HDFS data to Azure Storage	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-hdfs-azure-storage
Migrate Netezza data to Azure with Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-netezza-azure-sqldw
Migrate data from Amazon S3 to Azure Storage with ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-s3-azure-storage
Dedupe rows and detect nulls using ADF data flow snippets	https://learn.microsoft.com/en-us/azure/data-factory/how-to-data-flow-dedupe-nulls-snippets
Handle error rows and truncation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/how-to-data-flow-error-rows
Implement BCDR strategies for Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/pipelines-disaster-recovery
Apply advanced SAP CDC features and best practices	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-advanced-topics
Apply data flow best practices for writing to data lake	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-data-flow-write-to-lake

Decision Making

Topic	URL
Use automatic connector upgrades in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/automatic-connector-upgrade
Interpret pricing examples for ADF integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/better-understand-different-integration-runtime-charges
Choose the right integration runtime architecture	https://learn.microsoft.com/en-us/azure/data-factory/choose-the-right-integration-runtime-configuration
Choose compute environments for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/compute-linked-services
Choose Azure Data Factory connector release stages	https://learn.microsoft.com/en-us/azure/data-factory/connector-release-stages-and-timelines
Use connector upgrade advisor for ADF and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/connector-upgrade-advisor
Decide how and when to upgrade ADF connectors	https://learn.microsoft.com/en-us/azure/data-factory/connector-upgrade-guidance
Buy and use ADF data flow reserved capacity for cost savings	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-reserved-capacity-overview
Plan data lake and EDW migration to Azure with ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-overview
Answer common Azure Data Factory usage questions	https://learn.microsoft.com/en-us/azure/data-factory/frequently-asked-questions
Assess Azure Data Factory pipelines for Fabric migration	https://learn.microsoft.com/en-us/azure/data-factory/how-to-assess-your-azure-data-factory-to-fabric-data-factory-migration
Choose and provision Enterprise Edition Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-azure-ssis-ir-enterprise-edition
Assess and upgrade Azure Data Factory pipelines to Fabric	https://learn.microsoft.com/en-us/azure/data-factory/how-to-upgrade-your-azure-data-factory-pipelines-to-fabric-data-factory
Modernize Azure Synapse pipelines to Fabric Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-upgrade-your-azure-synapse-analytics-pipelines-to-fabric-data-factory
Plan and manage Azure Data Factory costs	https://learn.microsoft.com/en-us/azure/data-factory/plan-manage-costs
Assess and plan SSIS migration to Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/scenario-ssis-migration-overview
Apply SSIS migration assessment rules for ADF/Synapse	https://learn.microsoft.com/en-us/azure/data-factory/scenario-ssis-migration-rules

Architecture & Design Patterns

Topic	URL
Design efficient pipelines using mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-performance-pipelines
Configure Azure-SSIS IR with SQL failover groups	https://learn.microsoft.com/en-us/azure/data-factory/configure-bcdr-azure-ssis-integration-runtime
Build metadata-driven large-scale copy pipelines	https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool-metadata-driven
Migrate Azure SQL schemas to Azure Cosmos DB with ADF	https://learn.microsoft.com/en-us/azure/data-factory/how-to-sqldb-to-cosmosdb
Use Azure SQL Managed Instance with Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-sql-managed-instance-with-ir
Decide when and how to join Azure-SSIS integration runtime to a virtual network	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network
Understand architecture of SAP CDC in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-introduction-architecture

Limits & Quotas

Topic	URL
Understand reservation discounts for ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-understand-reservation-charges

Security

Topic	URL
Allow Azure Integration Runtime IP ranges through firewalls	https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses
Configure VNet injection for Azure-SSIS IR securely	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-virtual-network-configuration
Configure roles and permissions for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-roles-permissions
Configure outbound FQDN allow lists with Azure Policy for ADF	https://learn.microsoft.com/en-us/azure/data-factory/configure-outbound-allow-list-azure-policy
Use Azure credentials and managed identities in ADF	https://learn.microsoft.com/en-us/azure/data-factory/credentials
Plan secure data access strategies for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-access-strategies
Configure Azure Private Link for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-private-link
Configure managed identities for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/data-factory-service-identity
Security infrastructure and data movement in ADF	https://learn.microsoft.com/en-us/azure/data-factory/data-movement-security-considerations
Enable Microsoft Entra authentication for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/enable-aad-authentication-azure-ssis-ir
Enable customer-managed keys (BYOK) for ADF encryption	https://learn.microsoft.com/en-us/azure/data-factory/enable-customer-managed-key
Encrypt credentials for self-hosted integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/encrypt-credentials-self-hosted-integration-runtime
Access secured Purview from Data Factory via private endpoints	https://learn.microsoft.com/en-us/azure/data-factory/how-to-access-secured-purview-account
Use Azure Key Vault secrets in ADF pipeline activities	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-azure-key-vault-secrets-pipeline-activities
Join Azure-SSIS IR to VNet using PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network-powershell
Join Azure-SSIS IR to VNet via portal	https://learn.microsoft.com/en-us/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network-ui
Use managed virtual networks and private endpoints in ADF	https://learn.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint
Use built-in Azure Policy definitions for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/policy-reference
Secure Azure Data Factory with network and identity controls	https://learn.microsoft.com/en-us/azure/data-factory/secure-your-azure-data-factory
Detect and mask PII data with ADF solution template	https://learn.microsoft.com/en-us/azure/data-factory/solution-template-pii-detection-and-masking
Configure Windows authentication for SSIS packages in Azure	https://learn.microsoft.com/en-us/azure/data-factory/ssis-azure-connect-with-windows-auth
Store ADF credentials securely in Azure Key Vault	https://learn.microsoft.com/en-us/azure/data-factory/store-credentials-in-key-vault
Use private endpoints to securely copy data with ADF	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-portal-private
Configure Azure-SSIS integration runtime in virtual network	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-deploy-ssis-virtual-network
Secure self-hosted integration runtime with TLS from intranet	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-enable-remote-access-intranet-tls-ssl-certificate
Secure on-prem SQL access via Data Factory managed VNet	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-on-premise-sql-server
Configure private endpoint access to SQL Managed Instance from Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-sql-managed-instance

Configuration

Topic	URL
Define and use global parameters in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/author-global-parameters
Configure connections and global settings in ADF management hub	https://learn.microsoft.com/en-us/azure/data-factory/author-management-hub
Configure express virtual network injection for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-express-virtual-network-injection
Manage SSIS packages with Azure-SSIS package store	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-package-store
Configure standard virtual network injection for Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/azure-ssis-integration-runtime-standard-virtual-network-injection
Reference built-in and preinstalled components on Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/built-in-preinstalled-components-ssis-integration-runtime
Define and configure datasets in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-datasets-linked-services
Configure integration runtimes in Data Factory and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime
Define and configure linked services in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/concepts-linked-services
Override default ARM template parameters for ADF CI/CD	https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-resource-manager-custom-parameters
Configure Append Variable activity in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-append-variable-activity
Configure Data Flow activity execution	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-data-flow-activity
Configure Execute Pipeline activity in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-pipeline-activity
Use Azure Data Factory expression language and functions	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions
Use Fail activity to raise custom errors in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-fail-activity
Configure Filter activity for arrays in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-filter-activity
Configure ForEach activity loops in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-for-each-activity
Use Get Metadata activity in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-get-metadata-activity
Configure If Condition activity in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-if-condition-activity
Configure Lookup activity for external data in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity
Use Power Query activity for data wrangling	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-power-query-activity
Use Set Variable activity and pipeline return values	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-set-variable-activity
Configure Switch activity for conditional routing in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-switch-activity
Use system variables in Azure Data Factory expressions	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-system-variables
Configure Until activity loops in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-until-activity
Configure Validation activity for dataset readiness in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-validation-activity
Use Wait activity to pause Data Factory pipelines	https://learn.microsoft.com/en-us/azure/data-factory/control-flow-wait-activity
Enable data consistency verification in ADF copy	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-data-consistency
Configure fault tolerance for Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-fault-tolerance
Configure session logging for Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-log
Configure monitoring for ADF copy activities	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-monitoring
Configure Copy activity in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview
Use Copy activity performance optimization features	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-features
Preserve metadata and ACLs in ADF copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-preserve-metadata
Configure schema and data type mapping in Copy activity	https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping
Use the Copy Data tool in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool
Create and configure Azure integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-integration-runtime
Provision Azure-SSIS integration runtime in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime
Create Azure-SSIS integration runtime via Azure portal	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-portal
Create Azure-SSIS integration runtime via PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-powershell
Deploy Azure-SSIS IR using ARM templates	https://learn.microsoft.com/en-us/azure/data-factory/create-azure-ssis-integration-runtime-resource-manager-template
Create and configure self-hosted integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/create-self-hosted-integration-runtime
Create shared self-hosted IR with PowerShell	https://learn.microsoft.com/en-us/azure/data-factory/create-shared-self-hosted-integration-runtime-powershell
Configure cross-tenant Azure DevOps connections for ADF	https://learn.microsoft.com/en-us/azure/data-factory/cross-tenant-connections-to-azure-devops
Configure Aggregate transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-aggregate
Use Alter Row transformation for database updates	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-alter-row
Configure Assert transformation for data quality checks	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-assert
Use Cast transformation to change column data types	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-cast
Configure Conditional Split transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-conditional-split
Configure Derived Column transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-derived-column
Configure Exists transformation for row matching	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-exists
Configure Filter transformation for row selection	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-filter
Configure Flatten transformation for hierarchical data	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-flatten
Use Flowlet transformation for reusable logic in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-flowlet
Configure Join transformation in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-join
Configure Lookup transformation for reference data in data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-lookup
Configure multiple branches in mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-new-branch
Author and configure mapping data flow script	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-script
Configure Select transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-select
Configure source transformation in ADF data flows	https://learn.microsoft.com/en-us/azure/data-factory/data-flow-source
Configure Delete activity for file cleanup	https://learn.microsoft.com/en-us/azure/data-factory/delete-activity
Configure Avro format options in ADF and Synapse	https://learn.microsoft.com/en-us/azure/data-factory/format-avro
Configure binary dataset format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-binary
Configure Common Data Model format in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-common-data-model
Configure delimited text datasets in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-delimited-text
Configure Delta Lake format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-delta
Configure Excel file format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-excel
Configure Iceberg table format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-iceberg
Configure JSON file format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-json
Configure ORC format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-orc
Configure Parquet format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-parquet
Configure XML format in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/format-xml
Configure Bring Your Own Driver connectors in ADF	https://learn.microsoft.com/en-us/azure/data-factory/guidance-for-byod-approach
Customize Azure-SSIS integration runtime setup with additional components	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-azure-ssis-ir-custom-setup
Configure SHIR for Azure Log Analytics collection	https://learn.microsoft.com/en-us/azure/data-factory/how-to-configure-shir-for-log-analytics-collection
Configure custom Event Grid triggers for Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-custom-event-trigger
Create storage event-based triggers in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger
Configure schedule triggers in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger
Configure tumbling window triggers in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger
Develop and install licensed SSIS IR components in ADF	https://learn.microsoft.com/en-us/azure/data-factory/how-to-develop-azure-ssis-ir-licensed-components
Use parameters and expression language in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/how-to-expression-language-functions
Manage Azure Data Factory studio settings and preferences	https://learn.microsoft.com/en-us/azure/data-factory/how-to-manage-settings
Control Azure Data Factory studio preview experience	https://learn.microsoft.com/en-us/azure/data-factory/how-to-manage-studio-preview-exp
Run self-hosted IR in Windows containers	https://learn.microsoft.com/en-us/azure/data-factory/how-to-run-self-hosted-integration-runtime-in-windows-container
Schedule start and stop of Azure-SSIS integration runtime	https://learn.microsoft.com/en-us/azure/data-factory/how-to-schedule-azure-ssis-integration-runtime
Use trigger metadata parameters in ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-trigger-parameterization
Reconfigure Azure-SSIS integration runtime settings	https://learn.microsoft.com/en-us/azure/data-factory/manage-azure-ssis-integration-runtime
Configure diagnostic settings and Log Analytics for ADF	https://learn.microsoft.com/en-us/azure/data-factory/monitor-configure-diagnostics
Monitor Azure Data Factory with Azure Monitor	https://learn.microsoft.com/en-us/azure/data-factory/monitor-data-factory
Use monitoring reference data for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/monitor-data-factory-reference
Configure monitoring for Azure Data Factory integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/monitor-integration-runtime
Monitor integration runtimes in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/monitor-integration-runtime
Set up ADF diagnostic logs via Azure Monitor REST API	https://learn.microsoft.com/en-us/azure/data-factory/monitor-logs-rest
Configure monitoring for managed VNet integration runtimes	https://learn.microsoft.com/en-us/azure/data-factory/monitor-managed-virtual-network-integration-runtime
Monitor SHIR Azure VMs with diagnostics	https://learn.microsoft.com/en-us/azure/data-factory/monitor-shir-in-azure
Monitor SSIS operations in ADF using Azure Monitor	https://learn.microsoft.com/en-us/azure/data-factory/monitor-ssis
Visually monitor pipelines and activities in ADF	https://learn.microsoft.com/en-us/azure/data-factory/monitor-visually
Apply naming rules for Azure Data Factory artifacts	https://learn.microsoft.com/en-us/azure/data-factory/naming-rules
Parameterize linked services in Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/parameterize-linked-services
Parameterize mapping data flows from ADF pipelines	https://learn.microsoft.com/en-us/azure/data-factory/parameters-data-flow
Operate and configure SAP CDC processes in ADF	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-management
Set up SAP CDC linked service and dataset	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-prepare-linked-service-source-dataset
Configure prerequisites for the SAP CDC connector	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-prerequisites-configuration
Configure self-hosted IR for SAP CDC connector	https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-shir-preparation
Configure SHIR autoupdate and expiry notifications	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-auto-update
Configure self-hosted integration runtime as proxy for Azure-SSIS IR	https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-proxy-ssis
Configure source control for Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/source-control
Supported file formats and compression in ADF copy	https://learn.microsoft.com/en-us/azure/data-factory/supported-file-formats-and-compression-codecs
Configure tumbling window trigger dependencies in ADF	https://learn.microsoft.com/en-us/azure/data-factory/tumbling-window-trigger-dependency
Return values from child pipelines in Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/tutorial-pipeline-return-value

Integrations & Coding Patterns

Topic	URL
Configure SAP CDC connector to load data into Fabric OneLake	https://learn.microsoft.com/en-us/azure/data-factory/change-data-capture-from-sap-to-onelake-with-azure-data-factory
Use expression builder for ADF mapping data flows	https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-expression-builder
Connect Azure Data Factory to Microsoft Purview	https://learn.microsoft.com/en-us/azure/data-factory/connect-data-factory-to-azure-purview
Configure ADF connector for Amazon Marketplace Web Service	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-marketplace-web-service
Integrate Amazon RDS for Oracle with Azure Data Factory	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-rds-for-oracle
Copy data from Amazon RDS for SQL Server with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-rds-for-sql-server
Copy data from Amazon Redshift using ADF/Synapse	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-redshift
Copy data from S3-compatible storage using ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-s3-compatible-storage
Copy and transform data in Amazon S3 with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-simple-storage-service
Transform AppFigures data with ADF Data Flows	https://learn.microsoft.com/en-us/azure/data-factory/connector-appfigures
Transform Asana data with ADF Data Flows	https://learn.microsoft.com/en-us/azure/data-factory/connector-asana
Copy and transform data in Azure Blob Storage with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage
Transform Azure Cosmos DB analytical store data with ADF	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-analytical-store
Copy and transform data in Azure Cosmos DB for NoSQL	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-db
Copy data to and from Azure Cosmos DB for MongoDB	https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-db-mongodb-api
Copy and transform dat

…(truncated)

Azure Data Factory

File contents

Azure Data Factory Skill

How to Use This Skill

Category Index

Troubleshooting

Best Practices

Decision Making

Architecture & Design Patterns

Limits & Quotas

Security

Configuration

Integrations & Coding Patterns

Frequently asked questions

Azure Data Factory

File contents

Azure Data Factory Skill

How to Use This Skill

Category Index

Troubleshooting

Best Practices

Decision Making

Architecture & Design Patterns

Limits & Quotas

Security

Configuration

Integrations & Coding Patterns

Related skills

Frequently asked questions

How do I install the Azure Data Factory skill?

What does the Azure Data Factory skill do?

Is Azure Data Factory safe to use?

Which AI agents work with Azure Data Factory?

Is Azure Data Factory free to use?

Who published Azure Data Factory?