In this lecture I will share with how I saved 80% of BigQuery monthly billing of investing.com. How to reduce costs using GCP big Query? what should we pay attention to? We are going to cover all of google best practices while working with BigQuery.
Video
Slides
27.10.2019
Video
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
In the data-driven dream everyone knows where the data is, what it means, and how to use it. When you wake up, managing truth and enabling data is hard work fraught with data errors and miscommunication. Data teams tackle this challenge daily – some with tools, some with process and some by building. Nothing seems to scale. Can “Semantic Layers” help? * There were SAP Business Objects, Then the cloud happened. * Sharing the truth: How ETL and catalogs fail us. * Emergence of the “Metric Store” in Airbnb and Uber. * Semantics for mortals: is it just having a better process? * The new semantic layers: dbt metrics, Looker/Malloy. * What’s next: will the Semantic Layer rise from the ashes? In this talk we will go over the history of the semantic layer, how it looks today in different data teams, and where the industry might be headed. We will elaborate on the advantages of that direction.
Lecturer: David Krakov, CEO & Co-Founder of Honeydew. Previously, the co-founder of Verada, a data acceleration engine acquired by Starburs.
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
In this lecture, I will teach you the principles of how to build a big data architecture that stores and processes 1 PB worth of data using different technologies (Redshift, Databricks, BigQuery, SQream).
What will you learn? 1. Challenges with such a big data scale. 2. Consideration set of the use case. 3. The architecture of each technology, relevant features for the use case, and pro/cons analysis. 4. Cost estimation. 5. Performance challenges.
Lecturer: Omid Vahdaty, CTO at Jutomate, a company that specializes in end-to-end data solutions
Part 1
Video
Slides
Part 2
Video
Slides
Part 3
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Architectural Evolution of Amazon internal Data Platform
Lecturer: Filipp Fediakov 5.9.2023
In this talk, we will delve into how our team built a self-service platform PXT DaaS. PXT DaaS is an internal serverless data platform built on top of AWS. PXT DaaS facilitates data exchange between software and analytical teams in Amazon HR. To scale our platform to support 100s of datasets our team decided to build self-service portal. We will discuss the challenges faced by the team during each step of this project and how we overcame them.
Lecturer- Filipp Fediakov, a Senior Software Development Engineer at Amazon. I have been a lead engineer in a team of 12 building an internal data platform for software, analysts, data engineering teams in Amazon People Experience Technology. Now I work in a team that works on intranet search to help Amazon employees find the information they need to stay productive. Apart from work at Amazon I also write blogs and contribute to AWS CDK.
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Recently, there has been a lot of buzz surrounding Gnerative AI and LLMs, with many discussions focusing on the huge impacts they have on the world. How will these changes influence our approach to data and analytics? How can we leverage the capabilities of GPT effectively to optimize our data processes, save time, and streamline our efforts?
In the upcoming meetup, we will delve into GPT and how it will transform our relationship with data. We will discuss the importance of semantic layer in unlocking the full power of visual data exploration and gaining deeper insights, enabling us to make better informed decisions with the same data. Furthermore, we will shed light on the expanding utilization of graph technology, which plays a significant role in data analysis, and we will emphasize the importance of data governance. Lastly, we will discuss how organizations capitalize on technological advancements to foster innovation in a data driven world.
The lecture is for anyone interested in GenerativeAI and Data, suitable for beginners and experts alike.
Lecturer: Inna Tokarev Sela, The founder and CEO of illumex, the unified Business Data Language for Enterprises with proprietary Generative AI for automated Data and Analytics Interpretation. Throughout her career leading Data Products and as a data stakeholder, Inna recognized the oxymoron of our domain: despite huge investments in data and analytics, most business decisions are still not based on data or insights. Therefore, she uses her vast industry and academic experience to solve this conundrum and help data and business teams scale trust-based decision-making. Before embarking on her entrepreneurial journey, Inna was VP of AI at Sisense and Senior Director of Machine Learning at SAP. Additionally, She authored several patents on knowledge graphs, natural language, and deep learning, is a frequent speaker at leading D&A and AI conferences, and is Head of the Women in Data Israel chapter.
Video
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty:
Designed for organizations migrating from SQL to NoSQL or optimizing any NoSQL data model, this talk will assist practitioners looking to advance their understanding of data modeling for NoSQL Databases. We’ll cover various topics, from building a solid foundation on NoSQL to correcting your course if you’re heading down a dangerous path.
You will learn how to:
Determine what NoSQL data modeling techniques will be most performant for your specific usage patterns.
Apply common key-value and key-key-value (e.g., Amazon DynamoDB, Google BigTable, Apache Cassandra, ScyllaDB) data modeling strategies in the context of a sample application.
Apply NoSQL data modeling best practices to avoid mistakes that have troubled some of the world’s top engineering teams in production.
About the lecturer: Guy Shtub is Head of Training at ScyllaDB and holds a B.SC. degree in Software Engineering from Ben Gurion University. He co-founded two start-ups and is experienced in creating products that people love.
Video
Slides
—————————————————————————————————————————— I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,
feel free to contact me via LinkedIn – Omid Vahdaty: