I’m learning more about Apache Storm which is used for stream processing for near-real-time analytics in Azure HDInsight. I grabbed the image above from an AWS presentation on slideshare (link). I got all giggly inside as I learned about Storm architecture. Why? Because I built this much of this same functionality as part the DILM …
Continue reading Apache Storm Stream Processing in Azure HDInsight
Category:DILM
The Next Book Project: Frameworks
Kent Bradshaw, Shannon Lowder, and I explore stored procedure-based data integration patterns, an SSIS execution framework, and two Biml frameworks in a book we are writing, tentatively titled Frameworks.
Why I Built DILM Suite, by Andy Leonard
The following is Chapter 6: Catalog Browser from my latest book titled Data Integration Life Cycle Management with SSIS: A Short Introduction by Example: I was honored to be a Microsoft SQL Server MVP for five years (2007-2012). One cool thing about a being a Microsoft MVP was access to the internal developer teams. Everyone could …
Continue reading Why I Built DILM Suite, by Andy Leonard
ETL Instrumentation: Logging SSIS Variable Values
During the December 2017 delivery of Expert SSIS, I was asked if there is an SSIS Catalog Logging Mode that will display the value of variables. I responded that I wasn’t aware of a logging level that accomplishes this, but then – as sometimes happens – I could not let it go. It is an excellent …
Continue reading ETL Instrumentation: Logging SSIS Variable Values
Coming Soon to SSIS Catalog Compare: Values Everywhere
I’ve been testing a new feature in SSIS Catalog Compare‘s catalog browser. I call it “values everywhere.” What do I mean by values everywhere? In the browser shown above, please note the reference mapping of the ConnectionString property for the package connection manager in SimplePackage.dtsx. Each Reference is listed as a child node of the reference …
Continue reading Coming Soon to SSIS Catalog Compare: Values Everywhere
The Next Expert SSIS Delivery is 2.5 Days – 5-7 Mar 2018!
Yep, we’re doing it again! In cooperation with Brent Ozar Unlimited, the next delivery of Expert SSIS Training is 5-7 Mar 2018. I had a great time delivering this course with Brent in September and December 2017 and I’m excited to have the opportunity to share this material with you again! We’ve extended the course …
Continue reading The Next Expert SSIS Delivery is 2.5 Days – 5-7 Mar 2018!
Website Ch-ch-changes…
Inspired by Frank La Vigne’s (blog | @Tableteer) snappy update to the Data Driven home page, I decided to spruce up a couple / three of my websites including this one, andyleonard.blog. I added the sidebar shown on the right of the site. It contains a more-readily-available search box and links to other websites I maintain – …
Continue reading Website Ch-ch-changes…
On Data Frameworks…
You may not realize this, but Apache Spark is a framework. Spark is cluster-computing engine that manages parallel executions extremely well. Spark enables other technologies including Java, Scala, Python, R, and graph processing. Spark stitches together previously-disparate functionality into a cohesive, syntactically-similar set of commands. Spark’s architecture is library-driven and includes the following libraries: Spark SQL …
Continue reading On Data Frameworks…
Presenting A Day of Intelligent Data Integration in NYC 18 May!
I’m honored to present A Day of Intelligent Data Integration – a SQL Saturday NYC precon – 18 May! Description What is Intelligent Data Integration? SSIS packages developed using tried and true design patterns, built to participate in a DevOps enterprise practicing DILM, produced using Biml and executed using an SSIS Framework. Attend a day …
Continue reading Presenting A Day of Intelligent Data Integration in NYC 18 May!
Data Integration Life Cycle Management with SSIS is Available!
My latest book – Data Integration Life Cycle Management with SSIS – is now available! In this focused (185-page) book, I walk through ways to manage SSIS in the enterprise using what ships in the box. I point out what works well and what’s missing. I share workarounds for the stuff that’s missing, and then I …
Continue reading Data Integration Life Cycle Management with SSIS is Available!