AWS EMR

What is EMR?

Fully managed Hadoop cluster platform, that simplifies running big data frameworks.

EMR advantages:

  1. EMR is very fast to deploy, just a few minutes to configure and you are ready to go.
  2. It contains a variety of Apache open source projects such as Hive, Spark and Presto.  

EMR use cases:

Transformation of data and modeling. 

EMR antipattern use cases:

  1. BI
  2. Operational Database.

Our EMR Blogs

Core Hadoop technologies

  1. Jupyter Demystified
  2. Installing an AWS EMR cluster tutorial
  3. Cherry pick source files in Hive external table example
  4. registering EMR master node as target to ALB via cloudFormation or CLI
  5. Bootstrapping EMR from 0900 to 1700 on each work day with AWS Cloud Formation and AWS Data Pipelines
  6. EMR and watch dog: service-nanny?
  7. How to increase disk space on master node root partition in EMR
  8. How to restart AWS EMR Hive Metastore
  9. AWS S3 caching while working with Hive spark SQL and External table | LLAP
  10. Securing Spark JDBC + thrift connection (SSL) @ AWS EMR (demystified)
  11. How to connect via JDBC to Spark SQL EMR on AWS
  12. How to export data from Google Big Query into AWS S3 + EMR hive or AWS Athena
  13. Bootstrapping Oozie

Optional Web Client

  1. Questions and answers on AWS EMR Jupiter
  2. Bootstrapping Zeppelin EMR
  3. accessing EMR zeppelin api via ssl https
  4. EMR Zeppelin Security
  5. adding HTTPS SSL to Hue on AWS EMR Hue
  6. Working Cloud Formation example with EMR – adding volumes to Core group and Master group and increasing root partition

EMR Realated Architectures Blogs

  1. AWS EMR Presto Demystified | Everything you wanted to know about Presto
  2. 39 Tips to reduce costs on AWS EMR
  3. AWS Big Data Demystified #2 | AWS Athena, Spectrum, EMR, Hive
  4. When should we use EMR and When should we use Redshift? EMR VS Redshift
  5. AWS EMR and Hadoop Demystified – Comprehensive training program suggestion for Data Engineers in 200KM/h
  6. How to work with maximize resource allocation and Spark dynamic allocation [ AWS EMR Spark ]
  7. Spark performance tuning demystified

EMR Related Meetups

English

  1. AWS EMR Demystified Part 1-4
  2. AWS Big Data in 200KM/h
  3. AWS Big Data Demystified – Part 1 [English]

Hebrew

  1. 200KM/h overview on Big Data in AWS | Part 1
  2. 200KM/h overview on Big Data in AWS | Part 2

Top Video English

Top Video Hebrew