Skai’s challenges in developing and operating a data platform at scale. Skai’s data platform ingests on a daily basis data from over 200,000 tables distributed over 650 servers, microservices and SaaS applications. Managing more than 2PB of data. We will talk about the bottlenecks of managing this kind of a data platform on Hadoop and the reasons why we looked for a better technology. We will deep dive into how to manage a migration project of this magnitude in a live and continuously growing platform without lowering SLA. We will wrap up with what we have gained so far and how Snowflake helps us move faster.
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Based on a poll of over 1,600 data professionals in Reddit’s data engineering, this meetup will use real data to rank the thorniest issues in data engineering, and provide tangible, accessible solutions for each problem. Find out how to build out flexible, repeatable solutions for all the issues that drive you crazy, including: Version control Observability Cleansing & preparing data Maintaining 3rd party APIs
Video
English
Hebrew
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Introduction – what is Vertica, Query-Optimized Storage, Automatic Database Design, Automatic Data Marts, Pre-Aggregation for Cubes, Freedom from Underlying Storage, Deployment Options, All You Need for IoT & Clickstream, Complex Data Type Support, End-to-End ML Workflow Support, The Vertica Academy, Operator and Automation Tools for Kubernetes and Vertica Accelerator
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Looker is Google Cloud’s enterprise platform for business intelligence, data applications, and embedded analytics. Join this 101 session to understand the basics of what makes Looker a unique technology in today’s business intelligence landscape. Expect to come out of this session with an understanding of how Looker works and why its approach is fundamentally different to those of tools that have come before it
Video
Slides
Looker 201 Session
In this 201 session we will start witnessing the power of LookML (Looker’s proprietary modelling language) in action. Learn all the basics in a 30-minute, “Database to Dashboard” demo. Targeted at those who have at least some experience with SQL.
Video
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Object storage platforms offer unrivalled scalability and performance, but are notoriously difficult to manage. Over time an analytics team will start to spend more time fighting with the technology, instead of deriving useful insights from their data.
In this session, we will showcase how open-source project lakeFS prevents this from happening by enabling git-like operations over an object store. It provides a branching and committing model that scales to exabytes of data and makes your object storage ACID compliant. Learn how by using versioning concepts, you can work in an entirely new way, experiment safely, develop faster and ingest data seamlessly.
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty: