Update: Chenye Charlie Zhu has posted an excellent article about this new functionality over at the Microsoft ADF blog! It’s titled Comment Out Part of Pipeline. Part of one of my consulting gigs is helping a client lift and shift SQL Server Integration Services (SSIS) to Azure Data Factory (ADF). While developing some test pipelines …
Continue reading ADF Activity State Property Disables Pipeline Activities
Author:Andy Leonard
Christian, husband, dad, grandpa, Data Philosopher, Data Engineer, Azure Data Factory, SSIS guy, and farmer. I was cloud before cloud was cool. :{>
One Complex Example of ADF Pipeline Return Value
This post is part of a short series on Azure Data Factory pipeline return value functionality, which is in preview at the time of this writing. The Problem I am Trying to Solve I want to develop an Azure Data Factory (ADF) design pattern for calling focused, unit-of-work, function-y ADF pipelines that perform focused tasks. …
Continue reading One Complex Example of ADF Pipeline Return Value
One Simple Example of ADF Pipeline Return Value
This post is part of a short series on Azure Data Factory pipeline return value functionality, which is in preview at the time of this writing. The Problem I am Trying to Solve I want to develop an Azure Data Factory (ADF) design pattern for calling focused, unit-of-work, function-y ADF pipelines that perform focused tasks. …
Continue reading One Simple Example of ADF Pipeline Return Value
ADF Pipeline Return Value Series
While delivering Azure Data factory (ADF) consulting for a client, I had the opportunity to develop some utility pipelines that used Pipeline return values functionality (in preview at the time of this writing). For me, this exercise was a two-step process, which I share in this series: One Simple Example of ADF Pipeline Return Value …
Continue reading ADF Pipeline Return Value Series
SSIS in Visual Studio 2022
The Visual Studio 2022 Integration Services Extension is Generally Available (GA). I tested a couple things that, while not bugs per se, “bugged” me. The “bugged-me’s” I observed appear fixed. Enjoy! :{>
Changing ADF Execute SSIS Package Activity to Manual Entries
I found this error confusing, too. Why? One reason was I made an assumption that the SSIS Catalog reference (a reference is an SSIS Catalog configuration setting that connects an SSIS project to an SSIS Catalog Environment) would override any values being sent from the ADF Execute SSIS Package activity to the SSIS Project / Package execution call.
Learn ADF – From Me! – 7 Sep 2023
It has been quite a while since I’ve delivered live training online. That changes 7 Sep 2023 with a live online delivery of Master the Fundamentals of Azure Data Factory!
One Way to Provision an Azure Key Vault
One way to keep confidential information confidential is to store confidential values in Azure Key Vault. This post describes one way to provision an Azure Key Vault. In addition to other values, I use key vault to store login usernames – as well as passwords – in key vault. Why? I don’t like storing half …
Continue reading One Way to Provision an Azure Key Vault
XMheck (formerly “XMhell”)
09 Aug 2023 Update: Yitzhak Khabinsky and I met via Microsoft Teams to continue the conversation that started in the comments section of this post and was continued on LinkedIn. There will likely be more to come… Why Use XML to Transfer Data? I asked Brave Search, “Why use XML?” Brave responded with a Summarizer …
Continue reading XMheck (formerly “XMhell”)
Searching for Your Next Opportunity?
A friend compiled these thoughts about searching for your Next Opportunity: Reading “How to Survive if You Were Part of the Tech Layoffs” Wired Magazine: How to Survive if You Were Part of the Tech Layoffs | WIRED Online LinkedIn has a ton of posts and resources. This search will pull the latest from LinkedIn. Random …
Continue reading Searching for Your Next Opportunity?