Nesting ForEach Activities in Azure Data Factory

Near the end of June 2020, I delivered a free Enterprise Data & Analytics Summer o’ ADF webinar titled Using the Azure Data Factory ForEach Activity. My friend Meagan Longoria (Blog | @MMarie | LinkedIn) was in attendance. If you don’t know Meagan, you should – especially if you work with analytics. She’s scary-smart about Azure Data Factory and Power BI. And, she’s an awesome presenter.

Meagan asked great questions about the ADF ForEach activity. One question was about nesting ForEach activities – or adding a ForEach activity inside another ForEach activity. As you can see from this screenshot – taken from within a ForEach activity’s “inner activities” – the ForEach activity isn’t even available for selection in the Activities > Iteration & conditionals category:

Meagan and I speculated about calling a child pipeline using the Execute Pipeline activity (also shown in the screenshot above). We wondered how many child pipelines containing more ForEach activities we could “stack” in this manner (like turtles…).

I am not in the mood to test just how many (or to see if I can trick an ADF pipeline into nesting a ForEach activity inside another by copying and pasting the JSON), but I was in the mood to test calling a child pipeline with a ForEach activity in it. It worked, so we know we can nest at least one pipeline in a ForEach activity that includes another – technically nested – ForEach activity.

We are good for two turtles.

:{>

Need Help Getting Started with SSIS and ADF?

Enterprise Data & Analytics specializes in helping enterprises modernize their data engineering by lifting and shifting SSIS from on-premises to the cloud. Our experienced engineers grok enterprises of all sizes. We’ve done the hard work for large and challenging data engineering enterprises. We’ve earned our blood-, sweat-, and tear-stained t-shirts. Reach out. We can help.