In an earlier post titled Fabric Data Factory Design Pattern – Basic Parent-Child, I demonstrated one way to build a basic parent-child design pattern in Fabric Data Factory by calling one pipeline (child) from another pipeline (parent). In a later earlier post titled Fabric Data Factory Design Pattern – Parent-Child with Parameters, I modified the parent and child pipelines to demonstrate passing a parameter value from a parent pipeline when calling a child pipeline that contains a parameter.

In this post, I modify a parent pipeline to explore parameterizing which child pipeline will be called by the parent pipeline. In this post, we will:

• Copy the child pipeline id
• Clone a parent pipeline
• Add and configure a pipeline variable for the child pipeline id
• Test the dynamic pipeline id

Copy the Child Pipeline ID

Before proceeding, we need to obtain the pipelineId for the pipeline that will serve as the child pipeline in this demonstration of a dynamic parent-child design pattern demo: the “child-1” pipeline.

If you’ve been following our demonstrations in this series, please connect to Fabric Data Factory and find the “child-1” pipeline in the wsPipelineTest workspace. If you haven’t been following along, we can still be friends. You can find where we created the “wsPipelineTest” workspace and the “child-1” pipeline in the earlier post titled Fabric Data Factory Design Pattern – Basic Parent-Child.

1. Click on the workspace name in the left menu (my workspace is named “wsPipelineTest”)
2. Click on the “child-1” pipeline link:

When the “child-1” pipeline displays,

1. Click the “View” tab
2. Click “Edit JSON code”:

When the “Edit JSON code” dialog displays,

1. Highlight the JSON “objectId” property (listed beneath the pipeline “name” JSON property) value
2. Right-click the “objectId” property value and then click “Copy”:

The “objectId” property value should now be on your clipboard. Feel free to paste this value into a text editor (like Notepad) for safe keeping until you need it (later in this post), and then close the “Edit JSON code” dialog.

Clone a Parent Pipeline

Revisit the wsPipelineTest workspace.

1. Click the ellipsis beside the “invoke-child-pipeline” pipeline
2. Click “Save as”:

When the “Save as” dialog displays,

1. Rename the cloned pipeline “invoke-child-pipeline-dynamic”:
2. Click the “Save” button:

Add and Configure a Pipeline Variable for the Child Pipeline Id

When the “invoke-child-pipeline-dynamic” pipeline displays,

1. Click the “Variables” tab
2. Click the “+ New” button/link
3. Set the new variable’s “Name” property to “pipelineId”
4. Make sure the new variable’s “Type” property is set to “String” (String is the default Type)
5. Set the new variable’s “Default value” property to the “child-1” pipeline’s pipelineId property value you copied (and hopefully stored) earlier:

To configure the Invoke Pipeline (Preview) activity to use the new pipelineId variable,

1. Click on the Invoke Pipeline (Preview) activity
2. Click the “Settings” tab
3. Click the “Pipeline” property dropdown
4. Click “Add dynamic content”:

When the “Add dynamic content” blade displays,

1. Click the “Variable” tab
2. Click “pipelineId”
3. Click the “OK” button:

When you click the OK button to close the “Add dynamic content” blade:

1. The “Pipeline” property is replaced with the “Pipeline ID” property
2. The “Pipeline ID” property value now displays the expression @variables('pipelineId'):

Test the Dynamic Pipeline Id

First, save the pipeline, and then click the “Run” button:

Let’s test-execute the parent pipeline named “invoke-child-pipeline-dynamic”. If all goes as hoped, the parent pipeline executes and succeeds.

Click the Output icon:

Clicking the Output icon opens the Output blade which includes a link to the child pipeline execution’s pipelineRunId:

Click the pipelineRunId to visit the Monitoring hub for the execution of the “child-1” pipeline, which also succeeded:

Conclusion

To learn more from me about the new Fabric Data Factory Invoke Pipeline (Preview) activity, register for Enterprise Data & Analytics Fabric Data Factory training and subscribe to my newsletter:

Need Help?

Enterprise Data & Analytics delivers data engineering consulting and data engineering code reviews!. Let our experienced teams lead your enterprise data integration implementation. Contact us today!