Time and Money in the Cloud

What Could Your Enterprise Do With Extra Time?

Benjamin Franklin is credited with saying, “Time is money.”

The 2011 movie, In Time, depicts an entire economy based on time. (A friend pointed out that time is not fungible, but I digress…)

So, what could your enterprise do with more time?

When you think about it, this question lies at the heart of many data- and software-related enterprise activities, such as:

  • Disaster Recovery
  • High Availability
  • DevOps
  • Software development methodologies
  • Project management methodologies
  • Lifecycle management

I believe this is an important question – perhaps the important question of the cloud-era.
I believe time is replacing money in a way we’ve never before experienced.
I believe the cloud is driving this new economy.

Technology Economy Conversations

Not long ago I wrote of my experience when I left an instance of Azure Data Factory SSIS Integration Runtime running overnight and it cost me about $30USD. The post was titled The Cloud Costs Money and reflects some of the thinking of this post.

Not long after that, I was honored to chat with Stuart Ainsworth (@codegumbo) at Atlanta Azure DataFest:

Chatting with Stu

In this Data Driven DataPoint, captured while attending the inaugural Atlanta Azure Data Fest, I was honored to speak with Rie Irish, Julie Smith, Tim Radney, Geoff Hiten, and Stuart Ainsworth.

The event itself was an astounding event on two levels:

  1. The velocity of technological innovation is increasing (“well duh, Captain Obvious”) so, if you haven’t attended such an event recently – and by “recently” I mean the past eighteen months – you should attend to see how folks are combining cloud, Internet-of-Things (IoT), analytics, machine learning, artificial intelligence, on-premises, and hybrid technologies to deliver – frankly – amazing solutions.
  2. Community. Networking with people will change your career. It will change your career in a way that will change your life. Ask anyone who is engaged in a Microsoft data community. My synopsis of Atlanta Azure DataFest is here and my theme is “it is not too late to jump in”:

The next Azure DataFest (@AzureDataFest) is in Reston Virginia 11-12 Oct 2018!


Learn more and register here!

On Time and Money…

Stu and I spoke about the dynamic of time and money and how both relate to DTUs – the unit of measure for Azure data-related computing. So what’s a DTU?

According to Andy Mallon (@AMtwo) – paraphrasing Microsoft’s documentation in this post titled What the heck is a DTU? :

A [Database Transaction Unit] is a blended measure of CPU, memory, and data I/O and transaction log I/O in a ratio determined by an OLTP benchmark workload designed to be typical of real-world OLTP workloads. Doubling the DTUs by increasing the performance level of a database equates to doubling the set of resource available to that database.

That’s a great definition. But what are the implications?

Stu and I discussed the following data integration scenario:  Your enterprise hardware is currently fixed – which fixes the capacity of your data-related workload. You can change your enterprise’s workload capacity at any time; you can increase capacity by buying more or better hardware.

Imagine your enterprise migrates your data and data-related workloads to the cloud. (I know a company that can help! :))  After migration, your enterprise can scale hardware up to meet demand, and then scale it back down again when demand drops. The economics of pay-for-only-what-you-need-when-you-need-it is compelling, to be sure, and it drives almost all decisions to migrate to the cloud.

But there’s more.

Time to market matters to many enterprises.
Time to market matters more than ever to some enterprises.
The impact of time to market is easy to underestimate.

Thinking in DTUs

Consider the math: A DTU is a DTU. How the DTU cycles are distributed across time and processors doesn’t really matter.

Let’s say you pay $100 to incrementally load your data warehouse and the load takes 24 hours to execute at the scale you’ve selected in the cloud. Prior to thinking in DTUs, engineers and business people would think, “That’s just the way it is. If I want more or faster, I need to pay for more or faster.” But DTU math doesn’t quite work that way. Depending on your workload and DTU pricing at the time (FULL DISCLOSURE: DTU PRICING CHANGES REGULARLY!), you may be able to spend that same $100 on more compute capabilities and reduce the amount of time required to load the same data into the same data warehouse to minutes instead of hours.

That’s DTU Math.

The shift to DTU thinking is subtle but vital.

We are used to thinking the only way to make things faster is to spend more money. That’s simply no longer accurate. The shape of the line between cost and performance is may still trend linear but you can dramatically – and very, very quickly – alter the slope of that line, especially with regards to time.

The fact that the cost/performance curve can be altered in seconds instead of months meta-changes everything.

The statements above are examples of DTU thinking and DTU math. So, please ask yourself: “What could my enterprise do with more time?”

Why is that so important?
Because Ben was right: Time is money.

Experience Matters

This is a picture of Kent Bradshaw, Microsoft Certified Data Scientist. Kent and I work together at Enterprise Data & Analytics delivering data engineering, business intelligence, and data-based solutions that help businesses turn their data into actionable information. We do not do this alone, we are surrounded by an excellent team that holds diverse skills, perspectives, and work and life experience.

In this post I want to focus on experience because I believe experience is the most valuable component a consultant brings to a customer.

When Experience Hurts

Experience can hurt if we allow it. We can become locked into that’s-the-way-it’s-done syndrome. There are new frameworks and methodologies introduced every day (literally). It’s too much for any one person to keep up with the latest and greatest and most shiny new stuff.

This is why a team delivers value to customers.

A diverse team – like the team at Enterprise Data & Analytics – is incredibly valuable. We’re not just a bunch of old guys, we are also Millennials. Our younger team members are constantly learning about new technologies and methodologies and bringing them the attention of our older team members. The energy our younger team members bring to our projects is inspiring and amazing, and…

One of the most rewarding parts of my job as Chief Data Engineer is watching our younger folks thrive, grow, and succeed.

I love how our team works together! I love it when we all pile on WebEx to solve a problem for a customer or design part of a data warehouse solution. It’s thrilling! I’m proud beyond words of our team.

Although we haven’t advertised it much, the team at Enterprise Data & Analytics has grown – nearly doubling in 2018 (so far).

Diversity in backgrounds, experiences, and levels of experience helps us overcome the downsides of experience.

When Experience Helps

Kent and I (and others on the Enterprise Data & Analytics team) have delivered many data-related solutions. I like to joke and say, “when we started working with data we had to carve our own chips out of wood!” That’s not entirely accurate, but I learned Motorola 6800 machine code in 1975 (I was 11) and Kent started when punch cards were the norm.

Having lots of experience gives Kent and me perspective. We understand the maturity cycles of technological, corporate-cultural, and even projects. Why? We’ve experienced a number of said cycles. When the inevitable “bumps in the road” occur, we don’t panic or lose perspective. We’ve likely experienced something very similar in the past, made mistakes attempting to recover, and learned the best way forward from those mistakes.

Do we still make mistakes? Yep. But fewer, and we have a deeper, richer well from which to draw experience-based, time-tested solutions. Perhaps most important to our customers, we have the confidence that only comes from recovering from a mistake (or several).

Plus, our more experienced team members have the privilege and honor of sharing our experience with our younger team members. Together, we learn from each other. I love that!

Our more experienced team members bring leadership and management experience to bear on each and every Enterprise Data & Analytics project. My experience leading a team of 40 ETL developers at Unisys gives me a deeper appreciation for the total cost of ownership of any solution (which can dwarf the cost of development – especially short-sighted development).

Snapping It Together

With our diverse backgrounds, experiences, and levels of experience, Enterprise Data & Analytics is able to deliver the best solutions, period.

How may our team serve you? We offer:

  • Data Warehouse Rescue – is your data warehouse project stalled? We can help.
  • SQL Server and MySQL database performance tuning, development, and support – both on-premises and in the cloud (AWS and Azure).
  • SSIS (SQL Server Integration Services) coaching, development, performance tuning, and training.
  • Biml (Business Intelligence Markup Language) coaching, development, and training.
    • Coaching is my favorite! We navigate and you fly.
  • Data science to help you visualize your data, and data engineering to prepare your data for data science and business intelligence.
  • Automation solutions including SSIS Catalog Compare, Catalog Browser, SSIS Framework Community Edition, the BimlExpress Metadata Framework, and more!
  • Our team writes the books others read to learn SSIS.

Contact us today for more information.

:{>

On Working Remotely

I read a re-published article titled Working From Home Vs. Working At The Office: Who Gets More Done? by
Nate Hindman. The article was published on the Small Business blog of an insurer, The Hartford.

Of interest to me – a proponent of working remotely – was the comparison section, accompanying data, and the conclusion (which I share here):

Lister notes that when determining the efficacy of remote vs. in-office employees the culture of a team is as important as the work itself.

“If a team doesn’t know each other that well, it is far better to work face to face,” she says. In-person interaction “gives people the time to build social connections that make it easier to collaborate remotely and create that level of trust to be effective when they’re not in the same room.”

The salient points?

  1. Culture matters
  2. Rapport counts

Culture Matters

You can have the best people. They can be intelligent, talented individuals who know how to play nice with others. But if your culture is oppressive, their endeavors will fail.

Since I know and work with talented technologists I see this time and time again. Tip to employers:

If you want to keep top talent, you cannot pay them enough to put up with your crap. You have to stop the crap.

If you, as an employer, want to practice handing out participation trophies, you need to engage in a neighborhood sports league. People who are self-motivated enough to seek the knowledge they need – and wise enough to then apply that knowledge in creative, innovative ways – are demotivated by the recognition of “everyone as equal.”

But there’s a flip side: These same individuals are also not motivated by applause at the team meeting, Employee of the Month parking spots, or Starbucks gift cards.

I can hear you thinking, “So how do I motivate my top performers, Andy?”

I’m glad you asked,

Training.

That’s it. Training. Send them to training. They’ve self-identified as people who like to learn. Do you have a top shelf developer interested in cloud technology? Send her to Reinvent or Build. Save your Starbucks gift card money and allow someone else to park closer to the building. Send your top performers to more training.

I have been told thrice in the past six weeks about great people who left their last job or are about to leave their current job because their employer killed the training budget. Is training expensive? Yes. But not as expansive as finding a new employee – let alone a new top performer.

Employers, please do the math.

Rapport Counts

At Enterprise Data & Analytics, we often consult remotely, but we insist on starting projects face-to-face. Why? See the quote from the article above. Email is extremely subject to interpretation. People who don’t know me may misinterpret what I write in an email – unless they’ve had an opportunity to get to know me. there’s no substitute for building rapport. There’s no other way to build trust. Both trust and rapport are best built face-to-face.

A little-known fact: poor rapport with the customer team is the number one project killer. I and my team cannot be successful if we are unsuccessful at working with your team. Fortunately, I and my team are fun to work with! We are not prima donna’s, we learned because other people shared with us and one of the things we learned from the people who shared with us is: We should share with others. We love learning! I think that’s why we love sharing and helping others learn.

Customers are often shocked that I consult. I have a great team working with me but yep, I still show up and deliver. I think folks get the impression that someone who’s authored / co-authored a dozen books just isn’t accessible. Nothing could be further from the truth!

I help people every day!

In addition to public training with Brent Ozar Unlimited and SQLSkills, I deliver customized private training to companies on-demand.

I and my team deliver data integration solutions for customers, and we bring value when we do. (The post linked in the last sentence is worth the read. The short version is: Expensive developers are not as expensive as they appear.)

How may we serve you? Let us know!

Conclusion

  1. Be a dependable adult. This isn’t about age. It’s about maturity. Penny Trupe is a millennial and she’s a rocking consultant at Enterprise Data & Analytics.
  2. Share knowledge if you have it, seek out a mentor if you need to learn. (Pro tip: we all have something to share and something to learn.)
  3. Work hard and work smart. For more information, see Grant Cardone’s books The 10X Rule and Be Obsessed or Be Average. A tip o’ the hat to my Data Driven Podcast co-host and brother from another mother, Frank La Vigne (blog | @Tableteer), for turning me on to Grant.

:{>

I Want a Pony

Years ago, I emailed the Microsoft SSIS Developer Team with a request. It’s a long story. The short version is: there’s a bug in the way the SSIS runtime applies package configurations and this bug violates a principle of computing (the command line always wins). I should blog about it…

In the email, I included a request for a pony. Matt Masson (Twitter) replied with this image attached to his email and a note stating, “You cannot have a pony.”

Funny guy, that Matt.

We All Want a Pony

At one time or another I believe we all want a pony – especially when it comes to software projects. What do I mean? Well, it requires some background:

The Expectations Gap

Andy’s First Rule of Projects is:

In every project, be it internal or hired out to consultants or sent offshore; gaps exist in the expectations of the customer, the end-users, and the developers.

Andy’s First Rule of Projects (A1RP) is addressed by one and only one remedy: communication.

There is no way, in my humble opinion, to eliminate the A1RP gap. The best we can hope and strive for is to minimize the A1RP gap by communicating more and more effectively.

Communicating more is pretty easy to figure out.

Communicating more effectively, though? That’s hard and best described with an anti-pattern. I can hear you thinking, “What’s one anti-pattern of effective communications, Andy?” I’m glad you asked!

In It To Win It

We all like to win. As I type this, the 2018 Winter Olympics is happening in PyeongChang, South Korea. Athletes from all around the world have gathered after training for years. You better believe they want to win! In sports competitions, winning is the goal.

Sports contests are short term activities. As I wrote in Coopetition:

The idea is to do “whatever it takes to win” the contest. Rules exist to be stretched and circumvented. Vote early and often. Coerce the panel. Stack the deck. Winning is the most important thing.

Don’t believe me? Ask the Russian Olympic team.

In business this thinking manifests in negotiators who do not feel they’ve “won” unless the other side “loses.” I’ve been working in this field now for decades. I’ve been a manager in a large company. I’ve been a contractor. I’ve been a consultant. I’ve hired and fired, and been hired and fired. I’ve been on projects that were spectacular successes and projects that failed. I am here to testify:

I’ve never seen this kind of “winner” “win.”

Not even by their own standards. It. Just. Never. Happens. Without exception, I see these “winners” move on. They blame anyone and everyone for the failure of the project. They wag their tongues, spin the facts, and toss hard-working folks under the bus.

Not only do they not win – no one wins.

Thousands – millions, sometimes – is written off as a sunk cost. It’s a tragedy. It’s wasteful. And it’s a durn shame.

People and companies compete in business. What’s the best way to compete? Effective software projects require longer-term thinking and planning, and should strive to meet longer-term goals. Instead of fighting to win, we should…

Fight Well

Instead of doing whatever it takes to win, act according to principles.

“What’s the difference between fighting to win and fighting well, Andy?” I’m glad you asked. It’s pretty simple: Instead of doing whatever it takes to win, act according to principles. Many act according to preferences instead of principles today. We see it frequently in politics. A politician will give a speech against some behavior or political activity today and tomorrow the same politician will advocate for the same behavior or activity. We call that hypocrisy for good reason; it is hypocritical.

Fighting well means maintaining your integrity.

It is the lie of situational ethics – most often expressed as “for the greater good.” Situational ethics aren’t fighting well, they’re fighting to win. Ethics – sans “situational” – are fighting well. Here’s a good test of whether you’re fighting well or to win:

In a given situation, if your opponent lacks some information that would strengthen their position or argument – information which you posses, which may or may not weaken your position or argument – do you share that information or withhold it?

If you withhold such information I will not call you a liar (although I believe omitting the truth is lying). I will also not call you a truth-teller or refer to you as “a person of integrity.”

Fighting well means I sometimes lose.
Fighting well means I never lose my integrity.

Fighting well means maintaining your integrity. Some business examples may help:

  • If I hire a subcontractor and agree to pay them for 20 hours of labor on a deliverable I know will consume 200 hours or their time, I am not being a person of integrity.
  • If I overbid a contract, I am not being a person of integrity.
  • If my demonstrated goal is anything other than delivering maximum value to my client, I am not being a person of integrity.

Fighting well means I sometimes lose.
Fighting well means I never lose my integrity.

Conclusion

Ponies are expensive and require support and maintenance and care and feeding.

As someone procuring business intelligence consulting or training services, I recommend you buy products and services rather than be sold products and services. One way to tell the difference? A vendor fighting well, protecting everyone’s best interests, whose goal is wins all around, will sometimes demolish your expectations by telling you, “No pony for you.”

Dashing your expectations, based upon their experience and expertise, is the consultant delivering free consulting to you, which makes said consulting of infinite value.

The Second-Best Time

I recently returned from the PASS Summit 2017 where I had an awesome time hanging out with members of the SQL Server Community, learning, and presenting. Attending a conference inspires a bunch of ideas. If you’re like me,  you might kick yourself and think, “Why didn’t I think of that sooner?!”

If I had a time machine, I’d go back and share some thoughts with Younger Andy (lots of thoughts!). Alas, I don’t have a time machine. But I’ve learned this advice from others and I find it to be true:

The best time to do something may have been years ago. The second-best time is right now.

My Advice

I follow a two-step process:

  1. Take notes at conferences. Email yourself or write stuff down. Whatever works for you.
  2. Complete those items when you return home.

That’s it. When you realize something needs to be done, record it for posterity. Then do it. Get it done. Ship. Results, not excuses.

A question to consider: What are you doing, right now?

:{>

(Some of) Why We Do What We Do, an Opinion

I encounter different personalities in the Technology Communities to which I belong. I’m not merely a consumer, I contribute a different personality as well.

I’ve been playing and working in technology since 1975. One thing I’ve learned is: people believe the things they believe for a reason. Or reasons. I can hear some of you thinking, “What does that even mean, Andy”? I’m glad you asked.

Have you ever read something written by a fellow citizen of a technology community and thought (or even commented), “That’s just WRONG!” On technological and engineering matters, I grant that may be an accurate and objective response. But on other matters? Matters of preference? People believe what they believe for a reason. And to them, the reason is valid.

Behavioral Economics

I am a student of behavioral economics. From my studies to date one fact stands out: All behavior is driven by economics.

You may read that statement and cringe. You may ask how I – as a follower of Someone who famously said “It is easier for a camel to go through the eye of a needle than for a rich person to enter the kingdom of God.” – can write, much less believe, such a thing.  I didn’t write “money,” I wrote “economic.” There’s a difference (though I believe Jesus’ words apply equally to both money and economics). Please allow me to explain.

Many people read non-rational – or anything other-than-rational – as negative. I humbly submit this is an emotional response.

Economics includes more than just money. In technical communities we use economic terms to describe software architecture; terms such as “technical debt.” Technical debt costs money, it’s true; but technical debt is not a monetary description. It is an economic description the results of earlier, less-than-optimal decisions coming home to roost.

I write “all behavior is driven by economics” not as a criticism. I wrote it because people routinely and regularly make decisions based on the output of their amygdala – the “fight or flight,” survival-instinct part of the brain that communicates in fast, powerful, non-rational (not the same as irrational) emotion.

Many people read non-rational – or anything except-rational – as negative. I humbly submit that this is an emotional response.

Similar to my thinking about failure, I believe we could all benefit from reassigning our emotions about, well, emotions.

One Demonstration

My friend and brother Grant Fritchey recently tweeted an example of an emotional response. I am not criticizing Grant when I say his tweet is an emotional response. If you believe characterizing Grant’s tweet as “an emotional response” is a criticism, please start at the top of this post and re-read because you’ve missed an important point.

Sometimes there’s no substitute for expertise.

I think Grant is describing a conversation in which someone blew his mind. I’ve had similar conversations, some of them with Grant, who regularly schools me on SQL Server and Azure database technology and leadership. I consider Grant an expert on SQL Server and Azure databases and an example of servant leadership.

Another description of Grant’s conversation might be, “Grant learned a lot of new stuff talking to this person.” These days, it’s difficult impossible to know everything about everything – even in a technical field. We specialize. That’s a good thing because the option is to generalize, and experts are rarely generalists. Sometimes there’s no substitute for expertise.

Emotions are Real

Some read this and think, “Grant is being self-deprecating.” That’s just a little this side of accusing someone of false humility (there’s nothing false about Grant’s humility), which is a little this side of either accusing them of lying or being self-unaware (Grant is neither).

It may help to examine a different context: paranoia.

How does paranoia feel to the paranoid person?

How do we respond to paranoid people? We mostly criticize them, right? It’s culturally acceptable to poke fun at paranoid people, or to label behavior as paranoid – or even label a person as paranoid – especially if a person is behaving in a manner that suggests they believe people are out to get them. Consider this: How does paranoia feel to the paranoid person?

Do we imagine this person woke up this morning and decided, “I’m going to be paranoid today.”? Dig deeper with me: What is paranoia? It’s fear. Do we decide what we fear? I don’t think so. Behavioral economics supports me on this – our amygdalas are hard-wired with fight-or-flight in response to… fear. Can we learn to control our response to fear? Some, yes. By some, I mean some people can manage some responses to some fears. If Behavioral Economics is accurate, though, we all behave non-rationally – was are even irrational at times – when we make decisions.

To the paranoid, it’s Just Fear.

How does a paranoid person feel, then? They genuinely fear the things about which they are paranoid. To a paranoid person there’s no difference between real fear and the non-real fear they merely perceive or interpret as something to fear. To the paranoid, it’s Just Fear.

Is it socially and culturally acceptable to criticize someone for fear? “It depends.” That’s a topic for another blog post on another blog. To quote the philosopher Forrest Gump, “That’s all I have to say about that.”

The Science of Fear

In his 2009 book, The Science of Fear, author Daniel Gardner presents science – physiology and data science – to postulate humans are lousy at rational thinking. It’s not just me, others feel the same about… feelings. The Science of Fear is an interesting book. It’s not the easiest book to read but it is, in my opinion, worth reading.

My Point

I have two points, actually:

  1. Before cranking up the criticism, please try to imagine what the other person is (or people are) feeling.
  2. Please remember: people believe what they believe for a reason. Often, that reason is fear.

:{>

Remain Standing

Why am I writing about this on a technical blog (instead of my personal blog where I discuss faith and culture)? In this post I discuss personal and professional perseverance. I make no apology for my faith or for sharing it in every setting – personal and professional. People are singletons; you cannot decouple components of a person. I am a Christian and I am not ashamed of the gospel of Christ.

Have you ever heard a successful person share their story? Every success story I’ve heard starts with someone experiencing challenges in life and ends with their success. One key component? Perseverance.

I want to encourage you today: Remain standing.

What is perseverance? Perseverance is keeping on keeping on. It’s moving towards your goal, whatever that goal may be. Sometimes circumstances get so hard, life piles on so much, that perseverance is no longer moving – it’s simply standing.

I want to encourage you today: Remain standing.

Things may be hard right now. They will change.
Things changed and brought you to this low place, didn’t they?
Things will change again.

Just. Don’t. Stop.

Stand. Just stand.

:{>