Orchestrating Change: The Unity Catalog Migration Experience – at Data Science Summit 2025

The talk “Orchestrating Change: The Unity Catalog Migration Experience” captured this journey—from a fragmented data environment to a unified, secure, and scalable data management model. You can find the recording of the session below—it’s well worth watching to learn the details of this transformation firsthand.

At the outset, Robert Marek introduced the audience to the context of the APEX platform—a central data environment used by teams across the organization. APEX is where data is processed, modeled, and leveraged at scale across the entire bank.

However, as the platform grew, new challenges emerged:

• the need for better data access control,
• lack of complete, automated data lineage,
• increasing regulatory requirements (including BCBS 239),
• a growing number of teams and users.

Unity Catalog in Databricks became the natural direction for development—not as “just another feature,” but as the foundation for a new approach to data governance. The key moment: the migration decision was strategic, not merely technical.

What did this change look like from the perspective of teams working with data daily? Let’s go back to the pre-migration days.

Before:

• Data was scattered across different workspaces,
• Access required requests and waiting,
• There were no consistent naming conventions or structure,
• Finding the right data source was often a challenge. 

After migrating to Unity Catalog:

• Data access is managed directly in SQL,
• Teams can share data without involving DevOps,
• Data lineage is visible end-to-end,
• Row-level security and dynamic masking of sensitive data have been implemented. 

Result: Less manual work, greater transparency, and a real improvement in the daily life of data engineers and analysts.

Migrating to Unity Catalog didn’t mean “switching off the old world” with a single click. The team had to address dozens of questions about the impact of changes on existing microservices, user management, and automations.

The solution was an iterative approach:

• testing in development environments,
• gradually simplifying the process,
• automating migration steps,
• repeatedly cycling through test → insights → improvements. 

Thanks to this approach, the service window for a single workspace was reduced to around two hours—while maintaining full control over the process.

Conclusion: A successful enterprise-scale migration is the result of a well-designed process, not a one-time deployment.

The final, yet crucial perspective was presented by Jacek Cieślak, highlighting the significant role of communication and coordination.

With over 150 workspaces and around 1,500 users:

• every rollout was pre-consulted,
• users received clear messages and reminders,
• the team was prepared for post-migration requests,
• user support and education continued even after deployment. 

Result: The migration was completed in three months, with no critical production incidents.

Key lesson: Even the best architecture won’t work without human collaboration.

The talk showed that Unity Catalog is more than just a metadata management tool. It represents a shift in the way we think about data—as a shared asset, something that requires transparency and control, and as the foundation for business and regulatory decision-making.

If you’d like to explore the topic further, watch the recording of our presentation, where we share technical details, production examples, and key takeaways from a migration carried out in an enterprise environment. We also invite you to review the presentation in English that we discussed during our session.