architecture, Snowflake

How Skai leverages Snowflake to move faster

How Skai leverages Snowflake to move faster

Lecturer: Pablo Roth, 22.2.2022

Skai’s challenges in developing and operating a data platform at scale.
Skai’s data platform ingests on a daily basis data from over 200,000 tables distributed over 650 servers, microservices and SaaS applications. Managing more than 2PB of data.
We will talk about the bottlenecks of managing this kind of a data platform on Hadoop and the reasons why we looked for a better technology.
We will deep dive into how to manage a migration project of this magnitude in a live and continuously growing platform without lowering SLA.
We will wrap up with what we have gained so far and how Snowflake helps us move faster.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

Data Engineering

Data Engineering Challenges Demystified

Data Engineering Challenges Demystified

Lecturer: Omid Vahdaty, 11.1.2022+24.5.2022

Based on a poll of over 1,600 data professionals in Reddit’s data engineering, this meetup will use real data to rank the thorniest issues in data engineering, and provide tangible, accessible solutions for each problem.
Find out how to build out flexible, repeatable solutions for all the issues that drive you crazy, including:
Version control
Observability
Cleansing & preparing data
Maintaining 3rd party APIs

Video

English

Hebrew

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

Databases

Vertica Coolest Features 

Vertica Coolest Features in 2021

Lecturer: Moshe Goldberg, 21.12.2021

Introduction – what is Vertica, Query-Optimized Storage, Automatic Database Design,
Automatic Data Marts, Pre-Aggregation for Cubes, Freedom from Underlying Storage,
Deployment Options, All You Need for IoT & Clickstream, Complex Data Type Support,
End-to-End ML Workflow Support, The Vertica Academy, Operator and Automation Tools for Kubernetes
and Vertica Accelerator

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

BI

Looker Demystified | Part 1

Looker Demystified | Part 1+2

Lecturer: Jonny Dungay November/December 2021

Looker 101 Session

Looker is Google Cloud’s enterprise platform for business intelligence, data applications, and embedded analytics. Join this 101 session to understand the basics of what makes Looker a unique technology in today’s business intelligence landscape. Expect to come out of this session with an understanding of how Looker works and why its approach is fundamentally different to those of tools that have come before it

Video

Slides

Looker 201 Session

In this 201 session we will  start witnessing the power of LookML (Looker’s proprietary modelling language) in action.
Learn all the basics in a 30-minute, “Database to Dashboard” demo. Targeted at those who have at least some experience with SQL.

Video


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

architecture

Version Control for Your Object Storage

Version Control for Your Object Storage

Lecturer: Yoni Augarten 3.11.2021

Object storage platforms offer unrivalled scalability and performance, but are notoriously difficult to manage. Over time an analytics team will start to spend more time fighting with the technology, instead of deriving useful insights from their data.

In this session, we will showcase how open-source project lakeFS prevents this from happening by enabling git-like operations over an object store. It provides a branching and committing model that scales to exabytes of data and makes your object storage ACID compliant. Learn how by using versioning concepts, you can work in an entirely new way, experiment safely, develop faster and ingest data seamlessly.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty: