DILM Suite + Azure Data Factory Integration Runtime

DILM Suite was designed to support enterprise data engineering / data integration with SSIS. But the solutions, utilities, and tools work well with Azure Data Factory Integration Runtime (ADF IR, or “SSIS in the cloud”).

As I mentioned in my post titled The Cloud Costs Money, leaving ADF Integration Runtime running can get pricey – and fast.

Most DILM Suite Functions Do Not Require ADF Integration Runtime to be Running

That’s right. Since DILM Suite tools and utilities connect directly to the database, they do not require Azure Data Factory Integration Runtime to be in a running state for most operations.

Here, for example, I’ve connected Framework Browser to an instance of SSIS Framework Community Edition – deployed to the ADF Integration Runtime as demonstrated in the latest Summer-O’-ADF webinar, Designing a Custom ADF SSIS Execution Framework – doesn’t have to be in a Running state to allow interaction with Framework Browser and Catalog Browser (click to enlarge):

That’s handy information right there. And it can save you money.

:{>

Updates to Three DILM Suite Applications

It’s been a busy couple weeks here at Enterprise Data & Analytics in Farmville. I just published updates to not one, not two, but three – yes, three! – applications that are part of the Data Integration Lifecycle Management Suite, or DILM Suite.

SSIS Framework Community Edition

To prepare for Thursday’s (12 Jul 2018) webinar titled Designing a Custom ADF SSIS Execution Framework, I updated the documentation for SSIS Framework Community Edition. Why did I update the documentation? Because I want folks to know they can use SSIS Framework Community Edition in the cloud! That’s right, SSIS Framework Community Edition – still free, still open source – works with the SSIS Catalog behind Azure Data Factory Integration Runtime.

SSIS Framework Browser

Also in preparation for Thursday’s webinar, I updated SSIS Framework Browser! Version 0.4.2.0 (beta) remains free and now also works with SSIS Framework Community Edition implemented in Azure Data Factory Integration Runtime. Use Framework Browser to view metadata for SSIS Applications and Application Packages stored in the Framework.

SSIS Catalog Browser

Finally, I updated the login experience for those using SSIS Catalog Browser to connect to Azure Data Factory Integration Runtime. As with all Catalog Browser updates, I’ve applied the same updates to the user experience in SSIS Catalog Compare. Which reminds me…

SSIS Catalog Compare

On Thursday 19 Jul 2018 at noon EDT, I’m delivering another free webinar titled Use SSIS Catalog Compare to Lift and Shift SSIS to ADF! In this webinar, I will be demonstrating some of the features in version 3, which is approaching Preview.

I hope you will join me for these upcoming webinars and that you’ll take the free DILM Suite products for a test drive!

The Summer-O’-ADF Series Continues!

:{>

Free Webinar – The Azure Data Factory Controller Design Pattern – 28 Jun

Andy’s Summer-o’-ADF continues with another free (yes, FREE) webinar!

 

Abstract

Join me as I begin to explore a brand new topic: Azure Data Factory Design Patterns. This first webinar in the series is titled The Azure Data Factory Controller Design Pattern. In this demo-tastic session I show how to apply the pattern described in this post – titled SSIS Design Pattern: Controller Pattern – to ADF.

Registration is required to attend The Azure Data Factory Controller Design Pattern.

Sign up today!

In conclusion, I hope to see you there.

:{>

The Recording for Introduction to Lifting and Shifting SSIS to the Cloud is Now Available

 

The recording of my free webinar – Introduction to Lifting and Shifting SSIS to the Cloud – is now available (registration required).

Abstract

  • How does an enterprise lift and shift SSIS projects from SSIS Catalogs on-premises to the Azure Data Factory Integration Runtime?
  • What are the best practices?
  • Do SSIS lifecycle management tools exist to support SSIS lift and shift?

Join Andy Leonard – SSIS author, trainer, and consultant – to learn the answers to these questions. In this webinar, Andy discusses and demonstrates migrating SSIS to the Azure Data Factory Integration Runtime.

Enjoy!

You may want to hire Enterprise Data & Analytics to help your enterprise SSIS to Azure Data Factory because we grok ADF Integration Runtime.

Contact us today.

:{>

ADF Execute SSIS Package Activity

The good people who work on Azure Data Factory recently added an Execute SSIS Package activity. It’s pretty cool. Let’s tinker with it some, shall we?

First, you will need to create an Azure Data Factory SSIS Integration Runtime. If you don’t know how, that’s ok – I’ve written a post titled Lift and Shift SSIS Part 0: Creating the ADF Integration Runtime that describes one way to set up ADFIR.

0. Connect to Azure Data Factory

You can connect to the Azure Data Factory (ADF) dashboard here. After you connect…

1. Navigate to the Author Page

The “author” is indicated by the pencil icon on the left side of the ADF dashboard:

On the Author page you may add ADF resources by clicking the “+” symbol beside the search textbox. The dropdown menu contains three items at the time of this writing: Pipeline, Dataset, and Copy Data. Select Pipeline:

2. Drag an Execute SSIS Package Activity Onto the Surface

Click  the new Execute SSIS Package activity to edit it, clicking the tabs shown below the Azure Data Factory Integration Runtime (ADFIR) surface. Configure the Execute SSIS Package activity name on the General tab:

3. Configure the SSIS Package to Execute

Configure the Execute SSIS Package activity’s SSIS Package on the General tab. Set the Catalog Package Path property to <folder>/<project>/<package> as shown below (click to enlarge):

I can hear you thinking, “But Andy, what if I don’t know the Package Path?”

Fear not. The next release of Catalog Browser – a free utility from the DILM Suite (Data Integration Lifecycle Management Suite) – displays the Catalog Package Path when you click on an SSIS Package:

Please note: Catalog Browser displays backslashes in the Catalog Package Path and ADFIR’s Execute SSIS Package activity expects forward slashes.

The next version of Catalog Browser is scheduled for release on or before 1 Jul 2018.

That’s it. A quick Debug execution will verify the ADF pipeline executes:

Use SQL Server Management Studio (SSMS) to connect to the server hosting your Azure Data Factory Integration Runtime. View the All Executions report and confirm the execution happened and succeeded:

For more information about using SSMS to connect to ADFIR, please see the post titled Lift and Shift SSIS Part 1: Deploy Integration Runtime SSIS.

Learn More:
Free webinar: Introduction to Lifting and Shifting SSIS to the Cloud
Andy Leonard
21 Jun 2018 at noon EDT
Register today!

:{>