Data Mesh Architecture: Experimentation to Industrialisation
Lecturers: Sunny Jaisinghani and Simon Massey 27.2.2022
Discover what happened when a large financial service organisation who were already underway with a DevOps and Agile transformation went from a Monolithic Data Lake architecture, onto a federated self-service Data Mesh on Google Cloud Platform (GCP).
The key driver from the transformation was to reduce Lead times and improve the Flow Efficiency for Business Change. The typical approaches to transformation demonstrated substantial efficiencies across the core operational platforms but no material impact was seen on the downstream Data Publishing and Data Analytics platforms. These were faced with more fundamental blockers around lack of autonomy, monolithic architecture and proxy ownership of the data, compounded by legacy tech estate of on-prem data warehouses, data marts, data lakes, etc. End to end solutions required coordination between specialised teams working in silos leading to extended lead times.
This required a paradigm shift on both the systems architecture and Ways of Working.
In this session, we’ll explore the key driving principles for the Data Mesh from MVP, to productionisation to industrialisation.
The Data Mesh was built to be an Open Self Service platform whereby the various tenants can contribute to the features themselves alongside using the Core Platform self-service features. The success of the Data Mesh led to buy-in across the business and the Data Mesh Adoption accelerated exponentially. During the talk, we’ll highlight some of the key outcomes and business value delivered through the Data Mesh including:
- Rapid business values delivered to many ongoing programmes building ML models, MI Dashboards, cross-domain analytics, Data Provider APIs, Enquire and Reporting apps, etc.
- Teams were able to react to fast changing business and client demand with lead times dropping from months to days.
- New business models identified.
- The Data Mesh brought parity across the varying levels of technology maturity and skills within the organisation.
The Data Mesh is now a de facto part of the downstream data publishing, reporting and analytics for the organisation.
Who should watch?
Anyone who wants to understand how Data Mesh can help businesses achieve their organisational objectives.
What you’ll learn?
- What is Data Mesh.
- The key driving principles.
- How the hyper-new concept delivers business value.
- How Data Mesh works across different programmes.
Lecture Langauge: English
Sunny Jaisinghani- Data Mesh Platform Owner
Simon Massey- Data Mesh Lead Technologist
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty: