A friend pinged me recently to ask about rolling back Azure Data Factory (ADF) pipeline versions. My response was a question: Are you using source control with ADF? That did not help the current situation.
I thought of the way I often build ADF pipelines and shared my methodology, which is relatively simple (it has to be simple for me to understand it!):
I start by building a pipeline and getting something – anything – to work. Next, I click the ellipsis beside my pipeline in the Factory Resources blade and then click “Clone”:
I add some more functionality and get it working, and then I rinse and repeat – cloning this new version of the pipeline.
If I hit a snag, I decide whether to clone the current (snag-y) pipeline and beat work on it some more from where I’m stuck, or simply back up a step and clone a fresh copy of the pipeline when it last worked.
When I’m done, I delete pipelines I do not need.