I recently wrote a series of articles about Data Engineering Patterns. The four articles focus on the differences between software engineering and data engineering and the need to apply software engineering principles to data engineering. But I also cover why that’s not as simple as it sounds. the series, in order, is as follows: We Borrowed …
Continue reading The Data Engineering Patterns Series
Tag:Governance
The Words for the Work
What a month of naming posts was really all about [originally posted at Engineer of Data Substack] I have spent a chunk of a career in the part of data that has no name. Not the dashboards. Not the models everyone lines up to demo. The layer underneath – the part that moves the data, …
Continue reading The Words for the Work
You Can’t Govern What You Can’t Name
Governance is downstream of a name [originally posted at Engineer of Data Substack] Last week, I wrote: Observable Run – every execution leaves a durable, queryable record of what ran, with what, and how it ended. I called it a pattern. It is also a door. Behind it is the question every data team eventually gets …
Continue reading You Can’t Govern What You Can’t Name
We Borrowed the Tools, Not the Patterns
Data engineering in a software engineering world [originally posted at Engineer of Data Substack] Your orchestration runs. It Just Works every night. You built something real and that is not nothing. The ache isn’t that it’s broken – it’s that it lives in three people’s heads and a folder of conventions nobody wrote down. You …
Continue reading We Borrowed the Tools, Not the Patterns
Announcing: SSIS Lifecycle Advisory
I’ve launched a new fixed-scope advisory offer for organizations running SSIS in production. Many teams do not have an SSIS problem.They have a lifecycle management problem. Deployments feel risky.Change visibility is limited.Governance is inconsistent.Modernization questions keep piling up. The SSIS Lifecycle Advisory is designed to help teams get clear on: deployment and release risk governance …
Continue reading Announcing: SSIS Lifecycle Advisory