NoSQL

NoSQL Data Modeling and the LSM Tree data structure

NoSQL Data Modeling and the LSM Tree data structure

Lecturer: Guy Shtub 16.5.2023

In this talk, I’ll speak about basic data modeling and the LSM Tree, which is the underlying design for many NoSQL databases, including ScyllaDB and Apache Cassandra. I’ll cover topics such as Architecture, Primary Key Selection, Clustering key, and Compaction. This is an intermediate to advanced level talk.

About the lecturer:
Guy Shtub is Head of Training at ScyllaDB and holds a B.SC. degree in Software Engineering from Ben Gurion University. He co-founded two start-ups and is experienced in creating products that people love.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

Data Science

How to design ML Observability for high-risk AI use cases

How to design ML Observability for high-risk AI use cases

Vinay Kumar Sankarapu 1.5.2023

MLOps simplified the baseline processes making it easy to build models at scale today. But there has little or no focus on ML acceptance. Any AI/ML model can fail, models are not explainable by design, models can carry the risk of usage during production and model auditing is very complex. Deploying AI for mission-critical use cases requires additional layers like explainability, monitoring, auditability, data privacy and risk mitigation to ensure the AI solution is acceptable to all stakeholders.
Agenda:

  1. Introducing ML Observability
  2. Using ML Observability for model monitoring, model explainability and auditing.
  3. Designing the policy layers to manage model usage risk in ML Observability.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

architecture

Data integration, ETL, ELT, … challenges, and complexities

Data integration, ETL, ELT, ... challenges, and complexities

Lecturer: Meryem Chafry 4.4.2023

In today’s fast-paced business world, companies face a variety of challenges when it comes to collecting, integrating, and managing data from different sources. These challenges include dealing with legacy systems, managing data security, and handling data in real-time. At DatAtlas, we understand the complexity of these challenges and have created a SaaS platform that delivers data from various sources in real-time, securely and at scale.

During my presentation, I will discuss the importance of data integration, the challenges that companies face when dealing with data integration, I will also provide real-world examples of how businesses have successfully overcome data integration challenges using our platform.

Lecturer: Meryem Chafry, Founder & CEO @DatAtlas.
I worked for 3 years as a data scientist and data engineer in the Telco industry before launching my own start-up, that helps businesses manage, move, and make decisions on the freshest data in seconds. Since then, I have had the opportunity to work with other sectors to automate their data flows and achieve their business goals with data.

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

NoSQL

High Performance, Low Latency Database Architecture

High Performance, Low Latency Database Architecture

Lecturer: Guy Shtub 14.3.2023

In this talk, I’ll speak about modern, distributed, high-performance databases. I’ll cover topics like architecture, consistency, high availability, replication, and scaling. As an example, I’ll use ScyllaDB however the concepts hold for Apache Cassandra as well as other Column Family databases based on the BigTable paper published in 2006.

Lecturer: Guy Shtub is Head of Training at ScyllaDB

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty:

architecture

Data Mesh: Experimentation to Industrialisation

Data Mesh Architecture: Experimentation to Industrialisation

Lecturers: Sunny Jaisinghani and Simon Massey 27.2.2022

Discover what happened when a large financial service organisation who were already underway with a DevOps and Agile transformation went from a Monolithic Data Lake architecture, onto a federated self-service Data Mesh on Google Cloud Platform (GCP).

The key driver from the transformation was to reduce Lead times and improve the Flow Efficiency for Business Change. The typical approaches to transformation demonstrated substantial efficiencies across the core operational platforms but no material impact was seen on the downstream Data Publishing and Data Analytics platforms. These were faced with more fundamental blockers around lack of autonomy, monolithic architecture and proxy ownership of the data, compounded by legacy tech estate of on-prem data warehouses, data marts, data lakes, etc. End to end solutions required coordination between specialised teams working in silos leading to extended lead times.

This required a paradigm shift on both the systems architecture and Ways of Working.

In this session, we’ll explore the key driving principles for the Data Mesh from MVP, to productionisation to industrialisation.

The Data Mesh was built to be an Open Self Service platform whereby the various tenants can contribute to the features themselves alongside using the Core Platform self-service features. The success of the Data Mesh led to buy-in across the business and the Data Mesh Adoption accelerated exponentially. During the talk, we’ll highlight some of the key outcomes and business value delivered through the Data Mesh including:

  • Rapid business values delivered to many ongoing programmes building ML models, MI Dashboards, cross-domain analytics, Data Provider APIs, Enquire and Reporting apps, etc.
  • Teams were able to react to fast changing business and client demand with lead times dropping from months to days.
  • New business models identified.
  • The Data Mesh brought parity across the varying levels of technology maturity and skills within the organisation.

    The Data Mesh is now a de facto part of the downstream data publishing, reporting and analytics for the organisation.

     

    Who should watch?
    Anyone who wants to understand how Data Mesh can help businesses achieve their organisational objectives.

    What you’ll learn?

    • What is Data Mesh.
    • The key driving principles.
    • How the hyper-new concept delivers business value.
    • How Data Mesh works across different programmes.


    Lecture Langauge: English


    Speakers:

    Sunny Jaisinghani- Data Mesh Platform Owner
    Simon Massey- Data Mesh Lead Technologist

Video

Slides


——————————————————————————————————————————
I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way.
If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn – Omid Vahdaty: