Skai’s challenges in developing and operating a data platform at scale. Skai’s data platform ingests on a daily basis data from over 200,000 tables distributed over 650 servers, microservices and SaaS applications. Managing more than 2PB of data. We will talk about the bottlenecks of managing this kind of a data platform on Hadoop and the reasons why we looked for a better technology. We will deep dive into how to manage a migration project of this magnitude in a live and continuously growing platform without lowering SLA. We will wrap up with what we have gained so far and how Snowflake helps us move faster.
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty: