A provider of real estate advisory services contacted Imaginet for support on a system we had built for them a few years ago (mostly built using Azure SQL). They have multiple databases that have monthly costs associated with them. Our client was looking for a cheaper solution as the monthly costs were quite significant. They also had an existing capacity agreement with Databricks, so our goal was to conduct a Databricks migration with our clients databases.
What We Did
We started from scratch and built all the coordinates and objects for our client’s architecture. We used metadata to create their tables and developed a custom autoloader to transfer files to their destination tables, utilizing Databricks-driven technology in the backend.
To standardize the data deployment strategy, we built Databricks bundles. As part of our delivery, we reconciled with their existing SQL sources. This provides our client with the ability to compare the results from the data in Databricks to their previously existing sources. Furthermore, we compiled all their data into a single database, making it more easily accessible.
Why Our Solution Was Helpful
Our solution caused our client to take advantage of their partnership with Databricks, which allowed them to save money. Having their data together in one location is more convenient for our client and makes it easier to organize and access data. Additionally, we built jobs for them, removing the need for a data factory – another benefit of the data being in one location.
Why This Project Was Unique
This was Imaginet’s first time working with Autoloader in a Databricks environment. We were able to quickly learn and adapt and provided the client with a fully customized solution that could address their business needs.
Our client was also new to Autoloader, and they were impressed with the convenience and support offered by the solution. Overall, our customization of Autoloader will allow for additional, easy transformations. Thus, eliminating the need for multiple transformations, which is what they were experiencing with their previous Azure SQL environment.
Our team has been working hard these last few months, and we are excited to share our success with you. Don’t forget to subscribe to our newsletter to stay updated on our most recent Microsoft 365 projects. Fill out the form at the bottom of the page if you think a Databricks migration is right for your organization.
discover more
As a non-technical person, I wanted to understand more about Microsoft Fabric and why people love this tool. I sat down with one of our data experts, Anastasiia, to get a better understanding of how Microsoft Fabric helps her in…
Earlier this year, Microsoft announced it was rolling out a significant change to how it handles unlicensed OneDrive user accounts. Microsoft’s new OneDrive archiving policy is something every IT admin and business leader should be aware of. Enforcement began on…
Many organizations have them: old applications written in time immemorial, poorly documented, and with nonexistent tests. Yet, they’re still responsible for holding up key parts of the production environment, even if no one understands exactly how. Venturing into the code…
Let’s Build Something Amazing Together
From concept to handoff, we’d love to learn more about what you are working on.
Send us a message below or drop us a line at 1-800-989-6022.