My Day – Friday, 20 Jul 2018 – #SQLCareer

5:06 AM

– awaken after hitting the snooze on my phone alarm once.

5:06 – 9:30 AM

– Weigh in, pray, read the Bible, coffee, check the burn barrel.
This morning I weigh 201 pounds.
There’s a bunch to pray about. I start with praise followed by gratitude, thanking God for another day.
This morning I read Psalm 150 and the first half of Romans 14, which later prompts a couple social media posts about how I should stop despising and judging.
Coffee: ’nuff said.
The fire in the burn barrel did not do well last night. The bigger pieces I left to smolder overnight are still there. It’ll take a couple tries to get it burning this morning, I imagine. I start Try #1 with some dead twigs.
I comment on Brent’s post about #SQLCareer and fill up the Buffer buffer for the day.
I open Outlook and see an email from Kent Bradshaw.
I meet with Kent via SkypeForBusinessLync to discuss the goings-on at Enterprise Data & Analytics. Consulting is slow now, which is normal for this time of year. In about the month we’ll start getting pings for work. That’s fine, we both have other things going on (vacations, a trip to the Data Platform Summit 2018 in Bangalore, India, etc.).
Christy and I have breakfast (keto-friendly). She started baking yesterday for the Farmer’s Market tomorrow. After breakfast, I resist the small tray of brownies on the counter… for now. I had a couple yesterday and they are, as ever, awesome.
Stevie Ray finishes camp today and he needs to be picked up at 5:30. It’s about an hour away so I will likely knock off mid-afternoon today.
Riley is finishing up his homeschool testing this weekend. Since he’s doing math I’m on for questions. This will likely consume much of the morning.
Write. I add to my Notepad document to record my day for the next few days.
Startup vDemo – my demonstration virtual machine – in preparation for recording the SSIS Academy Data Flows 2 lab (video only, no audio). Let’s see if we can capture the video before I need to proctor Riley’s exam…

9:30 AM – 12:00 PM

Capture the video for the SSIS Academy Data Flows 2 lab which validates the lab document. Save the lab doc as PDF.
Proctor Riley’s exam.
Christy is cooking like a mad-woman for the Farmer’s Market and doesn’t want to stop to eat lunch (she somehow managed to make supper in the crockpot already!), so I grab some leftovers and eat in the office.

12:00 – 3:30 PM

Worked on SSIS Academy Data Flows 2 course. Added some files to the Data Flows 1 collateral.
I did about an hour’s worth of “Pine Therapy” – keeping the burn barrel going and enjoying some time outside in this lower humidity limbing up a pine I felled a couple days back and piling brush.
I decided to call it quits a little early as we prepare to pick up Stevie Ray from camp. It’s about an hour’s drive into the sticks. You have to put “the sticks” in context and realize that I’m writing that from five miles outside of Farmville Virginia…

The Recording for Use SSIS Catalog Compare to Lift and Shift SSIS to ADF is Available

The recording for the Summer-O’-ADF webinar titled Use SSIS Catalog Compare to Lift and Shift SSIS to ADF is now available (registration required)!

Abstract

SSIS Catalog Compare facilitates DevOps for enterprise SSIS, saves time, improves code quality, and lifts and shifts SSIS projects to Azure Data Factory’s Integration Runtime. More than just an “Package / Project Mover,” SSIS Catalog Compare scripts and/or deploys entire SSIS Catalog configurations including folders, projects, environments, environment variables, references, and reference mappings.

Watch SSIS author, consultant, trainer, and blogger Andy Leonard demonstrate using SSIS Catalog Compare to lift and shift enterprise SSIS to ADF!

My Day – Thursday, 19 Jul 2018 – #SQLCareer

#SQLCareer post for Thursday, 19 Jul 2018

4:26 – 5:30 AM

– Wake up (beating the alarm), weigh in, pray, read the Bible, coffee, check the burn barrel.
Christy and I have been on a keto diet since 5 Mar. I started out at 238 pounds. This morning I weigh 199 pounds. I’ve been bouncing in the 196-203 range since early May. Part of the reason for the plateau is cheating on the diet (diet fatigue). Part of the reason is I’ve probably lost what I can with diet alone and need to add more exercise. My exercise lately has been work around the farm.
There’s a bunch to pray about. I start with praise followed by gratitude, thanking God for another day.
I’ve been reading through the Book of Psalms recently and this morning I read 149 (of 150). I’m also studying the books of Romans and Colossians.
Coffee: ‘Nuff said.
I’m slowly burning limbs and brush from areas I’m clearing around the farm. There are hot coals from the three larger pieces I put in the barrel last night. I add three more larger pieces.

5:30 – 6:00 AM

– Write. I create a Notepad document to record my day for the next few days. After reading Brent Ozar’s first entry last night – in which he reminded me of Steve Jones’ SQLCareer post earlier this month – I decided to share a few days.
Startup vDemo – my demonstration virtual machine – in preparation for today’s Summer O’ ADF webinar titled Use SSIS Catalog Compare to Lift and Shift SSIS to ADF.
While vDemo is starting, I log into Azure Data Factory and start my Integration Runtime. The webinar is over 6 hours from now but I don’t want to wait until the last minute – it takes 20-30 minutes for ADFIR to start. I note the free Azure account I set up for the Summer O’ ADF has $181 left and is probably down to about 10 days remaining in the free month with $200 of credit. I note the accounting doesn’t (yet) reflect the 5 hours or so I had Integration Runtime running yesterday. My Azure spend has latency and I am attempting to monitor the latency. At present I see resource usage reflected in billing 36-60 hours after the resource usage.
I fire up vSQL2014 to begin testing SSIS Catalog Compare v3.2.1.1 (Preview). I’ve been working on this latest release of Catalog Compare for a year. Or more. Reasons for the delay, in order:
0. I bit off a lot. I made major changes to the graphical (treeview) representation of the SSIS Catalog. If you’ve been playing along at home, you can see these changes in the free DILM Suite utility, Catalog Browser. Catalog Browser and Catalog Compare share a lot of code.
1. Enterprise Data & Analytics landed two large data engineering projects at about the same time one year ago. One client doubled their request at the beginning of 2018 which resulted in doubling our people. Both projects are done but the both clients indicate they will have more work for us in the future. I still deliver consulting work – designing and building SSIS and database solutions – while managing the company. Nick Harris manages the business development cycle; a skill at which he is most gifted and for which I have neither proclivity or the inclination to proclive (…to quote the sage wisdom of Tom Smykowski [Office Space], “Engineers are not good at dealing with customers.”). Consulting and management took time away from software development.
2. I am a C# n00b. A year ago I was working my way up to becoming a C# n00b. (This maybe should have been first…)
While walking through the script for today’s webinar, I found a bug in the scripting of SSIS parameter literals. Booleans are… difficult. The string value of a Boolean is “true” or “false.” The corresponding T-SQL values are 1 or 0. Since SSIS Catalog parameter values are stored as sql_variants, getting the value right is vital. True/false not only doesn’t work, it causes the T-SQL script to fail. Hence, SSIS Catalog Compare v3.2.1.0 (Preview) was updated yesterday to SSIS Catalog Compare v3.2.1.1 (Preview). I’ll put it through some paces this morning by running through the webinar script again. If all goes well, I will drop it into the Dropbox folder and let the Early Adopters know there’s a new version available.
Software development is hard. Product development is harder. Product management is harder still. I am getting there…

6:00 – 6:30 AM

– Check the Buffer(.com) buffer. Load it up with posts for today from andyleonard.blog and a few other sites.
Check the results of yesterday’s newsletter campaign.
Look over stats from the business websites:
Enterprise Data & Analytics
DILM Suite
Biml Academy – more sales this month compared to last month, most likely due to new content.
SSIS Academy – launched this week and has decent traffic for a newly-launched site.
Poke around on social media for a few minutes. I start with LinkedIn and use it more than Twitter and Facebook these days. I really like the Daily Rundown notification and read at least the headlines each day. The article that catches my attention today is about drug companies retracting price hikes or announcing they will not increase drug prices in response to a tweet by the president. I’m drawn to this on a couple levels:
1. Many of Enterprise Data & Analytics’ customers are medical-related.
2. I am a Type-2 diabetic and one of my medications is expensive (~$500/month). (Related: the keto diet I mentioned earlier has helped. I now take 1/3rd of the medications I used to take and my blood sugar is lower.)
I also read about Amazon’s Prime Day and how it improved revenue for other online retailers, specifically Target.

6:30 – 7:45 AM

– I get a low battery warning on the laptop. I don’t always bring the laptop downstairs after work but I did yesterday hoping I’d get a chance to look at the SSIS Catalog Compare bug I’d identified earlier in the day. I did have time and I fixed it (I think) – more testing today… I shut down the laptop for now.
I make a keto-friendly breakfast for Christy and me and fry up some bacon for Emma and Riley (Stevie Ray is at camp this week). I put Christy’s omelette (spinach and cheddar) in the preheated-and-then-shut-off oven along with the bacon for the kids. I have an omelette and some fried bologna. The keto diet is weird if you’re used to non-fat diets. Fat is part of the keto diet, as are complex carbs. And it’s time for my second liter of coffee.
I catch up on some reading while eating. I am currently reading The Holiness of God by R. C. Sproul during mornings and The Culture Series by Iain M. Banks before sleeping. Reading is important and I enjoy both reading and writing.
I check the fire a couple times to make sure the wood I added is catching up. It’s not, so I add some more, smaller pieces this time. The wood I put in before sunrise was damp with the light dew. Sitting on the coals has dried it out and the twigs catch in a few minutes.
I load and start the dishwasher. I forgot about the clothes in the washer last night – it was a busy and late night for me with praise band rehearsal – so I re-wash that load.
I empty the trash and make a run to the landfill, checking the fire on the way out and when I return. It’s burning nicely.

8:00-8:20 AM

– I get to the office and settled a little before 8:00 AM. The laptop battery is happy (well, happier, as it charges).
I contemplate live-blogging the day and decide I will not be able to provide many updates, but it’s worth starting, and so I do. After some proofreading, it’s ready to publish.

Update 1: 8:20-8:45 AM

– I proofread (again!) this post and correct typos and misspellings and verb tenses and… you get the picture. I promise I proofread it in Preview. But for some reason, it’s not the same as reading it after it’s posted.

I pop open Outlook and check my email. Nick has closed another gig for what we call “SSIS Review.” I need to add a page to the Enterprise Data & Analytics website about this service. We’ve been doing a lot of this lately. We typically set the contract at not-to-exceed 40 hours (minimum) and only invoice for hours as they are consumed (unless the customer requests otherwise… some customers have budgets that they must consume within a fiscal year. If the customer requests it, we invoice up front.).

I also see a Customer Status Update from Penny. Penny has been working with Enterprise Data & Analytics for a couple years now. She’s my daughter, Mom to three of my favorite five grandchildren, and quickly becoming a senior SSIS developer. I’m proud of her.

Time to hit Update and post about this post on social media.

More later…

Update 2: 8:45-9:45 AM

I restarted vDemo and starting walking through the script for today’s webinar.

I put the wash in the dryer and started a new load. I propped open the dishwasher so the load that just finished will dry faster.

I checked the fire again and, durnit! Still not going like I want. I added some more small stuff and let it set for a while. It’s started burning well as you can see here.

I rescued a small bird from a couple of our cats that must’ve just left the nest.

I posted a LinkedIn video about the upcoming webinar:

Update 3: 9:45-10:45 AM

Webinar script rehearsal is complete and was successful.

Update 4: 10:45-11:45 AM

Took a break away from the laptop. Why? I like to do something else before presenting – something… different. Plus it’s a gorgeous day in Farmville Virginia with surprisingly low humidity for this time of year. Emma and I took advantage of the remaining coolness of the morning and piled some brush near the burn barrel.

It’s now time to start the webinar.

Update 5: 11:45 AM-1:15 PM

The webinar is done and, by all indications, was a success. It’s lunch time!

Update 6: 1:15-2:30 PM

Lunch was keto-friendly, of course. A salad and a meat-stuffed squash.

Christy and I dropped off her van across town. It needs new tires. The shop tells us they can get it done today which is great because we have a couple other errands to run this afternoon – we’re meeting friends from church and I am getting a haircut (or several hairs cut…).

Update 7: 2:30-4:30 PM

I posted The Recording for Use SSIS Catalog Compare to Lift and Shift SSIS to ADF is Available.

I got a haircut.

I restocked the burn barrel.

I edited the PowerPoint and lab document for the upcoming SSIS Academy Data Flows 2 course. Online courses take probably more time to build than you think – especially if you do it right. I spent a lot of time documenting the Data Flow 1 course. I’m spending just as much time going through the documentation for the Data Flow 2 course, editing and clarifying. Why? I don’t want to have to try to remember (or figure out) the mechanics of publishing a course each time. I want to spend my brain cycles on developing awesome courses!

At this point, I’ve been at it for over 12 hours. Do you know what entrepreneurs call a 12-hour shift? A half day.

Update 8: 4:30-6:00 PM

I updated pages at Biml Academy, SSIS Academy, and Enterprise Data & Analytics.

The shop finished with Christy’s van so I took her across town to pay for the work and pick it up.

I folded the last of the clothes.

I captured the video and began editing the audio script for SSIS Academy Data Flow 2 lecture. I hope to record the lecture audio tomorrow but it may have to wait until Saturday – tomorrow we pick Stevie Ray up from camp.

It’s time to take another look at the burn barrel, grab supper with the sexiest redhead on the planet, watch some Food Network, read some Culture Series, and then get a good night’s rest.

I am thankful. Today was a pretty good day.

:{>

One Solution to Presentation Categories

I’ve shared suggestions for improving presentation categories:

My chief concern is the scalar nature of the selections (the “Naked Scalar”). As I pointed out in both posts, I’ve been delivering presentations to the SQL Server Community for more than a decade and I’ve never – not even once – delivered a presentation that was strictly Beginner, Intermediate, or Advanced; or Level 100, 200, 300, 400, or 500.

It’s a mix. Every. Single. Time.

PASS Tracks

That said, I really like the definitions for PASS Tracks. Click that link and read through them. PASS did a good job distinguishing different types of helpful presentations and defining tracks.

As a data person, I get the logic behind the scalar values. Scalars are easier to store and way easier to search.

My post about presentation levels was inspired more by feedback from attendees who shared they thought my session level was incorrect – that it should have been higher or lower (I’ve actually received both complaints – that the session level was advertised as too high and too low for the same delivery of the same presentation…).

As an engineer, I’d like more accuracy:

But…

I’m not sure sharing more accurately would improve the conference attendee experience. Sharing more than a scalar – at least like the chart above – would increase the costs of printed material. It would require more engagement on the part of the attendee and more detail – and management – on the part of the presenter. Why management? My presentations evolve over time (I like to think they get better but I am biased…)

Would it be worth the investment? I think probably not.

So, while I continue to loathe the Naked Scalar wherever I encounter it, perhaps a scalar is the best solution after all.

I will continue to place these charts in my presentations and share them at the beginning of my session, in case someone thinks the presentation will cover more of one topic they wish to learn about. In this way, people have enough information – early enough in the presentation – to vote with their feet and attend a different presentation.

:{>

Announcing SSIS Academy!

I’m pleased to announce the availability of SSIS Academy!

I have a confession: Late last year when I wrote a post titled The Second-Best Time, I was writing to yours truly.

I was inspired by some friends and presenters I met at the PASS Summit 2017. I jotted down some notes – just like I advised in that blog post – and was reading through my notes when I returned to Farmville.

One of those ideas? Set up online training.

And so I began. I started by learning everything I could about WordPress-integrated learning management systems. I read reviews, downloaded some trials, kicked the tires, took ’em for a spin around the farm… and found one I liked.

I used it to set up Biml Academy. I put a ton of hours into Biml Academy. If you browse over to the site you may look at it and think, “It doesn’t look like you put a ton of hours into it, Andy!” You may have a point, but if you wear a loose-enough fitting hat, most people won’t notice. I had a lot to learn about learning management systems – and WordPress. I’ve been delivering training professionally since 1986. Granted, I am still learning about delivering training, but I needed to learn more about delivering recorded training online.

This is different than my Summer-O’-ADF series of webinars and Expert SSIS – the next delivery is 10-12 Sep 2018 – that I deliver in cooperation with Brent Ozar Unlimited. Way different.

Delivering Online Training

Biml Academy wasn’t my only idea; it’s just the beginning.

I’m pleased to announce the availability of SSIS Academy!

Data Flows 1 Course

This course introduces the SSIS Data Flow Task and familiarizes the student with building introductory data integration solutions using SSIS. Data Flows 1 includes the following course units, in order:

Lecture: Data Flows 1
Demo: Building Your First SSIS Package
Lab Exercise: Build a Real Data Flow, Part 1
Final Quiz

There’s even a certificate if you complete the course (click to enlarge):

Check out Data Flows 1 and let me know what you think.

:{>

Learn More:
Free Webinar – Use SSIS Catalog Compare to Lift and Shift SSIS to ADF – Thursday, 19 Jul 2018 at noon EDT
Biml Academy – Using Biml to Build a Target Database

Biml Academy – Using Biml to Build a Target Database

Using Biml to Build a Target Database is the latest unit available at Biml Academy.

You can see a description of Using Biml to Build a Target Database at the link above, but I ask you to consider Basic Biml Training – which includes:

There are additional reasons you should consider Basic Biml Training.

  1. Basic Biml Training is less expensive than the sum of the individual courses.
  2. If you sign up for Basic Biml Training now, your membership does not expire. (This will change. Soon.)
  3. This means if you sign up for Basic Biml Training now,  you will get access to the current content and all future Biml Basic Training content – at no additional charge.
  4. With the release of each new module, the cost of Basic Biml Training increases.

DILM Suite + Azure Data Factory Integration Runtime

DILM Suite was designed to support enterprise data engineering / data integration with SSIS. But the solutions, utilities, and tools work well with Azure Data Factory Integration Runtime (ADF IR, or “SSIS in the cloud”).

As I mentioned in my post titled The Cloud Costs Money, leaving ADF Integration Runtime running can get pricey – and fast.

Most DILM Suite Functions Do Not Require ADF Integration Runtime to be Running

That’s right. Since DILM Suite tools and utilities connect directly to the database, they do not require Azure Data Factory Integration Runtime to be in a running state for most operations.

Here, for example, I’ve connected Framework Browser to an instance of SSIS Framework Community Edition – deployed to the ADF Integration Runtime as demonstrated in the latest Summer-O’-ADF webinar, Designing a Custom ADF SSIS Execution Framework – doesn’t have to be in a Running state to allow interaction with Framework Browser and Catalog Browser (click to enlarge):

That’s handy information right there. And it can save you money.

:{>

SSIS Catalog Compare vNext Preview

I am happy to announce that the next version of SSIS Catalog Compare is officially in Preview!

What exactly does “Preview” mean? I’m glad you asked. It means I am sharing copies of SSIS Catalog Compare with users of the current version to get their feedback, find bugs, and help identify improvements.

Here’s a screenshot:

Looking at this screenshot (click to enlarge), you may notice some things:

  1. Font color indications – in addition to background color changes (which provide a shading for people who have difficulty seeing colors), the font color of “different” items is also different.
  2. The SSIS Catalog loaded into Catalog 2 is an Azure Data Factory Integration Runtime. SSIS Catalog Compare now speaks cloud.

I will be demonstrating using this preview of SSIS Catalog Compare in an upcoming (free) webinar titled Use SSIS Catalog Compare to Lift and Shift SSIS to ADF. The webinar is 19 Jul 2018 at noon EDT. I hope to see you there!

:{>

Updates to Three DILM Suite Applications

It’s been a busy couple weeks here at Enterprise Data & Analytics in Farmville. I just published updates to not one, not two, but three – yes, three! – applications that are part of the Data Integration Lifecycle Management Suite, or DILM Suite.

SSIS Framework Community Edition

To prepare for Thursday’s (12 Jul 2018) webinar titled Designing a Custom ADF SSIS Execution Framework, I updated the documentation for SSIS Framework Community Edition. Why did I update the documentation? Because I want folks to know they can use SSIS Framework Community Edition in the cloud! That’s right, SSIS Framework Community Edition – still free, still open source – works with the SSIS Catalog behind Azure Data Factory Integration Runtime.

SSIS Framework Browser

Also in preparation for Thursday’s webinar, I updated SSIS Framework Browser! Version 0.4.2.0 (beta) remains free and now also works with SSIS Framework Community Edition implemented in Azure Data Factory Integration Runtime. Use Framework Browser to view metadata for SSIS Applications and Application Packages stored in the Framework.

SSIS Catalog Browser

Finally, I updated the login experience for those using SSIS Catalog Browser to connect to Azure Data Factory Integration Runtime. As with all Catalog Browser updates, I’ve applied the same updates to the user experience in SSIS Catalog Compare. Which reminds me…

SSIS Catalog Compare

On Thursday 19 Jul 2018 at noon EDT, I’m delivering another free webinar titled Use SSIS Catalog Compare to Lift and Shift SSIS to ADF! In this webinar, I will be demonstrating some of the features in version 3, which is approaching Preview.

I hope you will join me for these upcoming webinars and that you’ll take the free DILM Suite products for a test drive!

The Summer-O’-ADF Series Continues!

:{>