Sync Blob Storage Containers with Azure Automation
I constructed this solution for a customer of mine needing to replicate files an application they had in multiple regions. While geo-replicated storage, a component feature of Azure keeping data synced across regions, would provide some level of redundancy it did not protect against accidental user errors in the customer’s application where data was removed nor does geo-replicated storage allow you to select the specific region data is replicated to.
What we needed was fine-grained control over which containers (folders) needed to be replicated to another region and to have complete control over which region we copied the contents to. A simple PowerShell script would work well to copy small amounts of data; but when the number and sizes of blobs grows this becomes a challenge to detect updates/additions/deletions and make reflective changes in the destination. Another requirement was this be automatic and hands-off — also not requiring a workstation or server to run the script from. The workflow will recursively walk through the folder structure of the specified container and copy everything!
The solution ended up being an Azure Automation Workflow that has some benefits of being ‘put to sleep’ if it exceeds runtime execution limits provided by the Automation facilities. When it resumes, because it persisted state on what iteration it was on each run, it will pick up where it left off.
About the Author
Ryan Berry
Hello! I am a technology professional with over 20 years of successfully selling, designing, and implementing IT solutions of all sizes. I have a sincere passion for technology both inside and outside of work involving myself with community events such as .NET user groups, FIRST robotics and in helping college bound high schoolers prepare for success through mock technology interviews. It’s easy to be successful in a role that you love and coming to work for me means I get to play with new and innovative technologies to bring bold new ideas to customers solving their business challenges. In my current role, I take the complex world of the cloud and Microsoft Azure, a portfolio of capabilities consisting of over 100 products, and help customers in properly selecting capabilities, tooling, and technical frameworks to product results. My vast technical background spanning technologies such as embedded development, Windows debugging and C# to open-source technologies such as PHP, Python, MySQL and Linux aids allows me to quickly dive into a problem and identify solutions producing quick results using the power of the cloud!
Listen/Watch on Stitcher
