Architecture

What is a good Big Data Architecture?

A good data lake enables fast querying of data, in a positive ROI and low maintenance.
Basically, Faster Cheaper and simpler should be the  motto for a good Architecture.

Our Architecture Blogs

  1. JutoLake – Analytics with no boundaries
  2. Big Data Architecture Best Practices
  3. 16 Tips to reduce costs on AWS SQL Athena
  4. DFP Data Transfer Files Use Case | Airflow and BigQuery 93% Cost Reduction
  5. Big Data Jargon | FAQ’s and everything you wanted to know and didn’t ask about Big Data …
  6. Recommendations for a new AWS account
  7. Access management in AWS SageMaker
  8. Questions and answers on AWS EMR Jupiter
  9. How to work with maximize resource allocation and Spark dynamic allocation [ AWS EMR Spark ]
  10. When should we use EMR and When should we use Redshift? EMR VS Redshift
  11. Flume VS Kafka , basic comparison
  12. AWS EMR and Hadoop Demystified – Comprehensive training program suggestion for Data Engineers in 200KM/h

 

Meetups which includes Architecture

English

  1. Data Mesh: Experimentation to Industrialisation
  2. Airflow Distributed work loads vs AWS Lambda vs Multi Threaded Python Script
  3. Fast Analytics with Presto and Alluxio
  4. A gentle Introduction to Alluxio
  5. Advanced ETL Demystified
  6. An Intro to Rivery Through Kits – Plug and Play Data Models
  7. 80% Cost Reduction in Google Cloud BigQuery
  8. AWS Big Data Demystified – Part 1
  9. AWS Big Data Demystified – Part 4
  10. GCP Big Data Demystified | Investing.com
  11. Serverless Data Pipelines 
  12. Unify Data Analytics: Any Stack Any Cloud
  13. Data Pipeline Observability
  14. Kafka + MQTT : make IoT data available for processing

Hebrew

  1. The lack of communication between data consumers
  2. Alluxio – The good, the bad and the ugly
  3. My First Architecture Parts 1-3
  4. How Skai leverages Snowflake to move faster
  5. When was the last time you analyzed 10 trillion records?
  6. Data Engineering Challenges Demystified
  7. Version Control for Your Object Storage
  8. From Block storage to Object storage
  9. How to create your own SaaS
  10. Data Engineering Demystified
  11. Data Engineering Use Cases Demystified
  12. 200KM/h overview on Big Data in AWS | Part 1
  13. 200KM/h overview on Big Data in AWS | Part 2
  14. AWS Big Data in 200KM/h 
  15. AWS Big Data Demystified – Part 2
  16. AWS Big Data Demystified – Part 3
  17. Walla’s Migration to AWS Cloud
  18. Automate all your EMR related activities

Top Videos Hebrew

Top video English