Lift and Shift is Not a Strategy…

Lift and shift is a bridge.

If your enterprise is considering migrating enterprise data and data integration workloads from on-premises to the cloud, lifting and shifting is not the end of the story. Lifting and shifting is, rather, the mechanism – the conversion project – that positions your enterprise to leverage the economies of scale afforded by cloud technology.

Andy’s Lift and Shift FAQ

“Should We Lift and Shift?”

I hear this question often and my response is, “When it makes sense to do so, yes, you should lift and shift.” This begs the next question, which is…

“How Do We Know It’s Time to Lift and Shift?”

My engineer-y response to this question is, “You will know it’s time to lift and shift to the cloud when you want to leverage functionality available in the cloud that is not (or not yet) available in on-premises versions of the enterprise platform(s) in use in your enterprise.”

“What Are Some Advantages of Migrating to the Cloud?”

The biggest advantage of lifting and shifting enterprise data to the cloud is the ability to efficiently scale operations. By efficiently, I mean quickly and easily – especially when compared to the time and expense (don’t forget opportunity cost when calculating expense) to scale up systems on-premises.

The ability to scale up and scale down on demand is a huge advantage for some business models which experience “spike-y” demand for operations at different times of the year, quarter, or month. But even if that’s not the case, all data scales. It’s very handy to be able to connect to the Azure Portal and move a slider (as opposed to purchasing and provisioning more hardware…).

There’s a brand new (in my opinion) “knob” exposed by cloud-enabled efficient scaling. As I wrote in my post titled Time and Money in the Cloud:

Let’s say you pay $100 to incrementally load your data warehouse and the load takes 24 hours to execute at the scale you’ve selected in the cloud. Prior to thinking in DTUs, engineers and business people would think, “That’s just the way it is. If I want more or faster, I need to pay for more or faster.” But DTU math doesn’t quite work that way. Depending on your workload and DTU pricing at the time (FULL DISCLOSURE: DTU PRICING CHANGES REGULARLY!), you may be able to spend that same $100 on more compute capabilities and reduce the amount of time required to load the same data into the same data warehouse to minutes instead of hours…

The fact that the cost/performance curve can be altered in seconds instead of months meta-changes everything.

“Are There Disadvantages of Migrating to the Cloud?”

It depends. (You knew that was coming, didn’t you?)

Enterprise Data & Analytics helps enterprises migrate data, data integration, lifecycle management, and DevOps to the cloud. In some cases (~30%), the enterprises spend a little more money in the near-term. There are two reasons for this:

  1. When upgrading, it’s always a good idea to operate new systems in tandem with existing systems. In a lift and shift scenario, this means additional expenses for cloud operations while maintaining the expense of on-premises operations. As cloud operations are validated, on-premises operations are shut off; thereby reducing operating expenses. In truth, though, this dynamic (and expense) exists whether one is lifting and shifting to the cloud or simply upgrading system on-premises.
  2. “Standing on the bridge” (more in a bit) can cost more than remaining either on-premises or lifting and shifting the entire enterprise workload to the cloud.
  3. Regulatory requirements – including privacy and regulations about which data is allowed to leave nation-states – will constrain many industries, especially government agencies and NGOs (non-governmental organizations) who interact heavily with government agencies.

Standing On The Bridge

One option we at Enterprise Data & Analytics consider when assisting enterprises in lift and shift engagements is something we call “standing on the bridge.” 

Standing on the bridge is present in each lift and shift project. It’s one strategy for implementing hybrid data management, which almost every enterprise in the cloud today has implemented. Hybrid means part of the enterprise data remains on-premises and part of the enterprise data is lifted and shifted to the cloud. 

Hybrid is implemented for a variety of reasons which include:

  • Mitigating regulatory concerns; and
  • As part of the normal progression of lifting and shifting enterprise data and data workloads to the cloud.

Standing on the bridge for too long is a situation to avoid. 

“How Do We Avoid Standing on the Bridge For Too Long?”

Planning. Planning is how an enterprise avoids standing on the bridge too long. Your enterprise wants advice from experienced professionals to shepherd the lift and shift operation. 

Enterprise Data & Analytics can help.

Helpful Tools

Enterprise Data & Analytics has been delivering, and writing and speaking about Data Integration Lifecycle Management for years. 

We’ve built helpful tools and utilities that are available at the DILM Suite. Most of the DILM Suite tools are free and some are even open source. 

The Cloud Costs Money

It’s true: the cloud costs money.

Yesterday, when I logged into my Azure Portal to set up an Azure Data Factory Integration Runtime for my presentation, the balance for my Azure free subscription (which allows me $100USD per month) was sitting around $82USD. This morning, after leaving a demo instance of ADFIR running overnight, my balance has dropped:

I absolutely love the cloud! This is why I’ve been delivering webinars about the cloud and blogging and presenting about Azure Data Factory.

I would just like to clarify a few things:

The Cloud Is Not Free

This much is true. And while the cloud can save enterprises – especially smaller enterprises – lots of cash on “burst-y” workloads via scale-on-demand – the cloud can actually cost more money when unneeded services are accidentally left running.

The Cloud Will Not Fix Bad Code

In a previous consulting life, a customer contacted us and asked for an evaluation of their architecture. They were attempting to scale the business and encountering… obstacles. A team looked over their software and database designs and recommended a rewrite of their custom code. We supplied a proposal (including an expensive estimate) to deliver the re-architecture, redesign, and rewriting of their code.

They felt the price was too high.

We understood, so we countered with a proposal to coach their team to deliver the same result. This would cost less, the outcome would be the same, and their team would grok the solution (because their team would build the solution). The catch? It would take longer.

They felt this solution would take too long.

We understood, so we offered to help in any way they would allow. The company sought help from other firms. A competing consulting firm offered to migrate their code and databases to a cloud provider. The company liked this solution and embarked on a migration.

The migration succeeded.
It took longer than expected.
The company hemorrhaged cash and clientele throughout the process, and is now a fraction of its previous size in people, clients, cash flow, and profit.

The lesson? Migrating to the cloud wasn’t a bad move for them.
Ignoring the advice of experienced architects was a bad move for them.

Did this client need to move to the cloud? Perhaps. More than anything, this client needed to pay off a tremendous amount of technical debt. By the time we were called, they were well into technical inflation and slipping into a technical depression.

Yes, the cost for a fix was high.
No, the cost for a fix wasn’t as high a price as the costs they have paid for not fixing their code. And…

… the cloud did not fix their bad code.

Heartbreaking

This was heartbreaking to witness. We could have helped them. Why were we unable to help them? We told them the truth; our competition told them what they wanted to hear.

As Brent Ozar pointed out in his post titled What It’s Like to Work for a Bad Contracting Company:

See, bad contracting companies have a huge sales force that goes out and asks customers, ‘Hey, what problems are you having? Sure, we can solve those. Just sign here.’ They make impossible promises about their – well, YOUR – capabilities and timelines.

Brent is right. This is why I ask, “Are you getting the best technical help available? or are you being sold technology by the best sales people?” There is sometimes a difference between consultants who are good at sales (or who hire good sales people) and consultants who are good at technology.

I’m going to take that one step further:

The very best technologist – the one most able to solve your problem for the lowest cost – may be someone who personally irritates you and who charges an hourly rate higher – perhaps multiples higher – than the competition.

Allow me to be (more) blunt:

  1. Did the company in the (true) account above believe they were going to save money?Yes, they did.
  2. Did the company in the (true) account above save money?No, they did not.

In my post titled A True Story of Value vs. Hourly Rate I share another true story of how our awesome, experienced team at Enterprise Data & Analytics was able to deliver value to a client. Even though we charged more per hour than the competition we delivered more value. How do we know?

The client did the math.

I’m writing this post to beg you to also do the math.
I’m writing this post to ask you to buy consulting services (if you need them) and not just be sold consulting services (whether you need them or not).
I’m writing this post to let you know that the cloud is awesome. But the cloud is not free.

“How Much Does Azure Cost?”

I hear this question a lot and it is difficult to answer. You can certainly browse over to the Azure Pricing Page to learn more. The difficulty is estimating how many __U’s you are going to use (see what I did there?).

Brent asked a really good question on Twitter and got this awesome answer:

The website is azureinstances.info and it. is. awesome.

How to Fix Being Charged for an Unused ADF Integration Runtime?

Stop ADF IR:

Delete unused cloud resources:

Peace.

Learn More:
Free Webinar – The Azure Data Factory Controller Design Pattern – 28 Jun at noon EDT

Honored to Present an SSIS and ADF Precon at Data Platform Summit 2018!

Expert SSIS – Live, Online, 2.5 Days – 10-12 Sep 2018 – delivered in cooperation with Brent Ozar Unlimited

Honored to Deliver Intelligent Data Integration – a PASS Summit 2018 Precon