The Microsoft Fabric Team released a new Invoke Pipeline Activity in September 2024. This post describes one way to implement a parent-child design pattern using the new activity. In this post, we will:

• Create a new workspace
• Build a child pipeline
• Build a parent pipeline
• Test

Create a New Workspace

Connect to Fabric Data Factory.

1. On the left menu, click Workspaces
2. When the Workspaces blade displays, click the “+ New workspace” button:

When the “Create a workspace” blade displays, enter the Name “wsPipelineTest” and then click the “Apply” button:

Build a Child Pipeline

When the workspace opens,

1. Click the “+ New” dropdown”
2. Click “Data pipeline”:

Name the new pipeline “child-1” and then click the “Create” button:

When the pipeline displays,

1. Click “Start with a blank canvas”
2. (Optionally) search for “wait”
3. Select a Wait activity from the activity list”:

Accept the default settings for the Wait activity and save the new pipeline:

Build a Parent Pipeline

On the left menu,

1. Click on wsPiplineTest
2. Click the “+ New” dropdown and then click “Data pipeline” (not shown in the image below, but identical to the procedure for creating a new child data pipeline above)
3. Name the new pipeline “invoke-child-pipeline”
4. Click the “Create” button:

As when building the child pipeline, when the pipeline displays,

1. Click “Start with a blank canvas”
2. (Optionally) search for “invoke”
3. Select an Invoke Pipeline (Preview) activity from the activity list”:

When the “invoke-child-pipeline” displays, select the “Invoke Pipeline (Preview)” activity and then edit the Name property to read “Invoke child pipeline”:

1. Click the “Settings” tab
2. Click the “Connections” dropdown
3. Click “More” from the Connections dropdown list:

When the “Get data” dialog displays, click “Fabric Data Pipelines”:

When the “Connection credentials” dialog displays,

1. Click the “Connection” dropdown
2. Click “Create new connection”
3. Click the “Connect” button:

1. Edit the “Connection name” property
2. Select the “Authentication kind” property (the default option “Organizational account” is selected below)
3. Click the “Sign in” button:

After signing in (not shown), the new Connection is configured. Click the “Connect” button to proceed:

The pipeline is displayed once more.

1. The new Connection is displayed in the “Connection” property
2. The Workspace name is configure for the “Workspace” property
3. Click the “Pipeline” dropdown
4. Select the “child-1” pipeline:

Test

1. Click the “Save” button
2. Click the “Run” button:

If all goes as planned, the “invoke-child-pipeline” pipeline executes and succeeds.

Click the Output icon to open the Output properties:

The Output properties display a link to the child pipeline Run Id. Click the link in the value of the “pipelineRunId” property to open the child-1 pipeline execution results in the Monitoring hub:

The Monitoring hub displays pipeline run details for the execution of the child pipeline:

Conclusion

To learn more from me about the new Invoke Pipeline (Preview) activity, register for Enterprise Data & Analytics Fabric training and subscribe to my newsletter:

Need Help?

Enterprise Data & Analytics delivers data engineering consulting and data engineering code reviews!. Let our experienced teams lead your enterprise data integration implementation. Contact us today!