Project Summary

Databricks Migration Saves Money and Improves Data Accessibility

Databricks Migration Saves Money and Improves Data Accessibility

A provider of real estate advisory services contacted Imaginet for support on a system we had built for them a few years ago (mostly built using Azure SQL). They have multiple databases that have monthly costs associated with them. Our client was looking for a cheaper solution as the monthly costs were quite significant. They also had an existing capacity agreement with Databricks, so our goal was to conduct a Databricks migration with our clients databases.  

What We Did 

We started from scratch and built all the coordinates and objects for our client’s architecture. We used metadata to create their tables and developed a custom autoloader to transfer files to their destination tables, utilizing Databricks-driven technology in the backend.  

To standardize the data deployment strategy, we built Databricks bundles. As part of our delivery, we reconciled with their existing SQL sources. This provides our client with the ability to compare the results from the data in Databricks to their previously existing sources. Furthermore, we compiled all their data into a single database, making it more easily accessible.  

Why Our Solution Was Helpful  

Our solution caused our client to take advantage of their partnership with Databricks, which allowed them to save money. Having their data together in one location is more convenient for our client and makes it easier to organize and access data. Additionally, we built jobs for them, removing the need for a data factory – another benefit of the data being in one location. 

Why This Project Was Unique  

This was Imaginet’s first time working with Autoloader in a Databricks environment. We were able to quickly learn and adapt and provided the client with a fully customized solution that could address their business needs.  

Our client was also new to Autoloader, and they were impressed with the convenience and support offered by the solution. Overall, our customization of Autoloader will allow for additional, easy transformations. Thus, eliminating the need for multiple transformations, which is what they were experiencing with their previous Azure SQL environment.  

Our team has been working hard these last few months, and we are excited to share our success with you. Don’t forget to subscribe to our newsletter to stay updated on our most recent Microsoft 365 projects. Fill out the form at the bottom of the page if you think a Databricks migration is right for your organization.    

Technologies Used

Databricks Migration

discover more

New Application Development Projects Enhance Resource Management 

Imaginet Resources CorpApr 11, 20243 min read

A telecommunications organization contacted Imaginet for assistance on 2 new application development projects. The first was a portfolio tracker, and the second was an office furniture ordering application.    Portfolio Tracker    The portfolio tracker we built for our client tracks and…

QR Code Phishing Attacks: Are You Protected?

Carlos TansecoApr 9, 20244 min read

QR Code Phishing Attacks: Are You Protected?  April 09, 2024 QR code phishing attacks or “Quishing” are on the rise, and it’s crucial to raise awareness about this evolving threat. Scanning these codes has become second nature to many of…

Virtual Workspaces Are Here! How Will You Use Them? 

Chris PowellApr 4, 20248 min read

Virtual Workspaces Are Here! How Will You Use Them?   April 4, 2024 My YouTube feed has been full of reviews of the Apple Vision Pro since it became available in the US on February 2. In one of the videos,…

Let’s Build Something Amazing Together

From concept to handoff, we’d love to learn more about what you are working on.
Send us a message below or drop us a line at 1-800-989-6022.