According to Gartner, 83% of all data migration projects either fail or exceed their budgets and timelines. The complexity of the as-is data warehouse architecture poses one of the major risks of migrating to the cloud. Migration planning requires a discovery of the as-is DW architecture, resolving tech debt, and figuring out lineage and interdependencies across workflows.
Manual discovery requires a lot of time from the experts, which can create a resource bottleneck. Due to the complexity and share volume of data objects, the discovery is likely to have a lot of unknowns and the resulting documentation has shortcuts that lack an audit trail, which increases migration risks.
Very often companies waste time and money on migrating everything. Data asset and flow overlap in a legacy environment can be significant (even 30-50%). To mitigate these risks, we recommend breaking the large migration scope into manageable pieces – vertical slices – to focus on business outcomes – applications / BI Services.
- Vertical slicing reduces risk and improves quality with frequent feedback
Vertical slicing includes end-user involvement, meeting their expectations, and building trust in data and services in the cloud. This is a critical enabler for being able to decommission as-is applications/BI services. - Vertical slicing enables customers to optimise performance, reduce complexity & cost
Lift and optimise – In some cases, migration can leverage the current code base and design patterns (lift and shift), but we recommend even then adding optimisation of their execution on the target DBMS. Optimisation ensures that the migrated workloads are meeting the desired results – reduced latency, improved performance, and predictable scalability. In vertical slicing technical debt or other challenges become visible, transparent, and small fixes can be designed and implemented immediately. These optimisations help to reduce complexity, cost of unproductive data distribution/ movement, latency, and management. - Vertical slicing with DataOps automation enables accelerated & scalable delivery
Migration is an opportunity to introduce a new way of working to build a modern DW using metadata-driven DataOps automation. DataOps enables scalability and adaptability because it centralizes knowledge into a common metadata repository.
Automate migration with Migration Mill
Migration Mill is a Solita migration framework that helps customers to deliver incremental value with a repeatable and consistent migration approach like a migration factory.
We recommend automating the migration effort, which effectively allows us to do more work, with fewer people, with higher quality than traditional methods.
- Metadata discovery automation enables migration planning based on accurate & auditable architecture blueprint of as-is DW to be migrated. This can reduce migration scope, cost, risk, and time.
- Accelerate migration to the to-be DW by automating common, time-consuming tasks like translating code, maintaining security standards, and onboarding new users.
- Improve migration quality by automating highly manual, error-prone processes like code conversion, ensures consistent results, with fewer errors.
- Reduce migration costs – Automation also reduces the overall costs of the migration. It allows doing more work with fewer people.
We believe that vertical slicing coupled with optimisation and automation is the best way to migrate an as-is DW into the cloud. Using prioritised vertical slices, the migration delivers incremental value. Optimisations enable customers to “clean the closet” – avoid carrying any technical debt from the as-is to to-be DW. They help to reduce complexity, cost of unproductive data distribution/ movement, latency, and management.
Let us know if you are interested to hear more about the approach and the Solita migration framework – Migration Mill.
