Originally published at Engineer Of Data as a newsletter. Subscribe to my newsletter here.

50 Years

Last month, I Celebrated 50 Years in Software. That link will take you to a recording of a show in which my youngest daughter, Emma, interviewed me to discuss this milestone. I think Emma is an excellent interviewer, but I biased.

Embracing AI

In his book, Reclaiming Reality, Andrew Torba makes a case for embracing AI and leveraging it to facilitate human flourishing. His context is a Christian worldview. You will not agree with everything he writes. If you learn more about Torba, you will not agree with everything he’s said, done, and believes. In this regard, Torba is like everyone else on the planet (including me).

Except you.

In one important way, though, Andrew Torba is different:
He’s acted.
I recommend this book because Andrew shares his bias for action.

How 50 Years’ Experience Informs my Recommendation

Colossus is a movie (The Forbin Project) based upon a series of books written by D. F. Jones about a powerful computer that goes rogue. The theme of good-technology-gone-bad was popular when I was younger, and remains popular today (Terminator, anyone?)

How realistic is the theme?
Has it aged well?

Exhibit A: The Internet

How’d that work out?

I could provide a number of examples that demonstrate outcomes anticipated by very few. I’ll cite one: advertising. Advertising everywhere (writes the guy who injects ads into every newsletter… guilty as charged…).

Prolific advertising did not make the paragraph above. Advertising-driven media wasn’t invented by the internet, to be fair. Advertising-driven addiction was, however. Don’t take mental health and well-being advice from a data engineer; watch the inventors of the technology describe what they built, how it works, and – most importantly – whether they let their children participate in it in The Social Dilemma.

Fifty years of experience informs me that Torba is right about at least one thing:

If we are not intentional about learning and applying AI for human flourishing, others will learn and apply AI for their own personal flourishing. – Andy, circa 2025

“Learn to vibe-code”?

Did I learn to code? Yep.
Did it work out for me? Yep.
Do I believe learning to code will work out well for everyone? Not a chance.
Do I think the same holds for “vibe-coding?” I am not sure.

Vibe-coding is the popular description given to the practice of providing instructions to a large language model (LLM) – also know as prompting – with the goal of having the LLM generate software code that accomplishes the desires of person providing the prompt. I can hear some of you thinking…

“Does vibe-coding work, Andy?”

And there’s my experience.

In March 2025, I decided to take a whack at vibe-coding. I share the following disclaimers:

• I am an experienced software developer
• I am an engineer
• I am a data engineer

This is my story.

<ad>

Final week, data teams!

The Advanced Azure Data Factory Training Day kicks off next week and there are only a few seats left.

For just $249 (normally $999), you get:

✅ Advanced ADF training (intermediate to advanced level)
✅ Workbook to follow along and practice
✅ Private 30-min post-class Q&A with the expert instructor
✅ 1-year access to the session replay

🚀 Whether you’re trying to build reusable pipelines or eliminate technical debt, this session is your fast track to productivity.

👉 Grab your seat now before registration closes!

</ad>

My Vibe-Coding Story

I intended to record the presentation but Microsoft Teams and I experienced a miscommunication for the first half of 2025 (so far) and, well, let’s just say my attempts at achieving course recordings have been less than successful. Nevertheless, the prompt worked and Claude was successful in building a simple Microsoft Fabric Data Factory pipeline.

I was all giggly inside.
I was concerned.

I asked the obvious question: “IS AI GOING TO TAKE MY JOB!?!?!”

I noodled on the experience for a few days and then asked Claude to write some C# that would perform some ReST API calls. I have some experience retrieving data from ReST APIs in Azure Data Factory and Fabric Data Factory – even a little experience hitting ReST APIs from SSIS (and SOAP in VB back in the day… I am old). I had less experience doing so from C# and wanted to see what Claude might have to say on the matter.

Claude’s first pass contained errors.
I copied the errors and pasted them as the next prompt.
Claude’s response fixed some of the errors, not all, and introduced a new error.
Copy. Paste.

This continued for some number of iterations until the code Claude produced almost successfully interacted with an API method I chose to test with. For some reason, Claude became less-helpful as the number of iterations increased – at times ignoring suggested fixes (that I had tested in the code and that I, therefore, knew said fixes worked).

I conducted additional tests asking Claude to generate other code solutions. The pattern I noticed was: Claude was good for about two dozen prompt-responses / versions of the generated code. After that, Claude appeared to get a little, well, lost.

Important Hindsight Interlude

1. By my seat-of-the-pants estimate, Claude was generating good code about 80% of the time.
2. In my first 45 minutes working with Claude – when I possessed my least experience prompting – Claude generated code it would have taken me several hours to learn and grasp well-enough to write without help. Given my available for time for learning to such a level of understanding, I estimate (again, seat-of-the-pants), Claude saved me about a week.
   In 45 minutes.
   In my first 45 minutes.
3. I was impressed.
   And even more concerned.

The Rest of the Story (Part 1)

I note Sonnet 4 didn’t begin losing context clues until somewhere around version 60, an improvement to my experience of Sonnet 3’s 24-ish version limit. And then I learned a couple tricks that helped increase that version limit:

1. Rather than have Claude generate and, more importantly – regenerate – complete code version listings, I prompted it to only supply code changes and include instructions on how and where to apply said changes.
2. After I make and test the suggested changes, I upload the file(s) to Claude and ask it to analyze the file(s). So far, I’ve not reached a “version limit.” I’m north of 300 versions on one project. And counting.
   1. “Andy? You aren’t sharing details about these other projects…”
   2. Yes.
3. My prompting skills improved with practice. This surely led to faster CAD (Claude-Assisted Development). I’ve learned to ask better questions (prompts), too. Sometimes the generated code includes out-of-date extensions. Some of these extensions come with security warnings. I’ve learned to request the “safest and most robust libraries and extensions available.” I’ve also learned to prompt for alternatives…. and then ask for Claude to rank the alternatives and supply justification for the rankings… and then provide a recommendation.
4. The best way to learn is to do. (It’s that bias for action thing. Again.)

Conclusion

I’m about 12 weeks into this journey. I’m no Doug Finke (whom you should also follow on LinkedIn to learn more about AI and LLMs and PowerShell). I’m still working my way up to becoming a n00b.

Need Help?

Consulting & Services

Enterprise Data & Analytics delivers data engineering consulting with ADF and Fabric, Data Architecture Strategy Reviews, and Data Engineering Code Reviews!. Let our experienced teams lead your enterprise data integration implementation. EDNA data engineers possess experience in Azure Data Factory, Fabric Data Factory, Snowflake, and SSIS. Contact us today!

Training

We deliver Fabric Data Factory training live and online. Subscribe to Premium Level to access all my recorded training – which includes recordings of previous Fabric Data Factory trainings – for a full year.

We deliver data.

:{>