Blog

architecture, Big Query, cost reduction, GCP Big Data Demystified, superQuery

80% Cost Reduction in Google Cloud BigQuery

The second in series of lectures GCP Big Data Demystified. In this lecture I will share with how I saved 80% of BigQuery monthly billing of investing.com. Lectures slides:

Videos from the meetup:

Link to previous lecture GCP Big Data Demystified #1

——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

https://www.linkedin.com/in/omid-vahdaty/

Data Engineering

Data Engineering Challenges Demystified

Data Engineering Challenges Demystified

Lecturer: Omid Vahdaty, 11.1.2022

Based on a poll of over 1,600 data professionals in Reddit’s data engineering, this meetup will use real data to rank the thorniest issues in data engineering, and provide tangible, accessible solutions for each problem.
Find out how to build out flexible, repeatable solutions for all the issues that drive you crazy, including:
Version control
Observability
Cleansing & preparing data
Maintaining 3rd party APIs

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

Databases

Vertica Coolest Features 

Vertica Coolest Features in 2021

Lecturer: Moshe Goldberg, 21.12.2021

Introduction – what is Vertica, Query-Optimized Storage, Automatic Database Design,
Automatic Data Marts, Pre-Aggregation for Cubes, Freedom from Underlying Storage,
Deployment Options, All You Need for IoT & Clickstream, Complex Data Type Support,
End-to-End ML Workflow Support, The Vertica Academy, Operator and Automation Tools for Kubernetes
and Vertica Accelerator

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

BI

Looker Demystified | Part 1

Looker Demystified | Part 1+2

Lecturer: Jonny Dungay November/December 2021

Looker 101 Session

Looker is Google Cloud’s enterprise platform for business intelligence, data applications, and embedded analytics. Join this 101 session to understand the basics of what makes Looker a unique technology in today’s business intelligence landscape. Expect to come out of this session with an understanding of how Looker works and why its approach is fundamentally different to those of tools that have come before it

Video

Slides

Looker 201 Session

In this 201 session we will  start witnessing the power of LookML (Looker’s proprietary modelling language) in action.
Learn all the basics in a 30-minute, “Database to Dashboard” demo. Targeted at those who have at least some experience with SQL.

Video


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

architecture

Version Control for Your Object Storage

Version Control for Your Object Storage

Lecturer: Yoni Augarten 3.11.2021

Object storage platforms offer unrivalled scalability and performance, but are notoriously difficult to manage. Over time an analytics team will start to spend more time fighting with the technology, instead of deriving useful insights from their data.

In this session, we will showcase how open-source project lakeFS prevents this from happening by enabling git-like operations over an object store. It provides a branching and committing model that scales to exabytes of data and makes your object storage ACID compliant. Learn how by using versioning concepts, you can work in an entirely new way, experiment safely, develop faster and ingest data seamlessly.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

architecture

Advanced ETL Demystified

Advanced ETL Demystified

Lecturer: Omid Vahdaty 18.10.2021

PySpark advantages over traditional ETL. Advanced techniques of parsing largest scales, JSONs based data sets at scale.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty: