Blog

What we’re about

A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just look at the picture in this articles, it only covers a small fraction of the technologies in the Big Data industry…

Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS infrastructure to answer the basic questions of anyone starting their way in the big data world.

how to transform data (TXT, CSV, TSV, JSON) into Parquet, ORCwhich technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL?how to handle streaming?how to manage costs?Performance tips?Security tip?Cloud best practices tips?

Some of our online materials:

Website:

https://big-data-demystified.ninja/

Youtube channels:

https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber

https://www.youtube.com/channel/UCMSdNB0fGmX5dXI7S7Y_LFA?view_as=subscriber

Meetup:

https://www.meetup.com/AWS-Big-Data-Demystified/

https://www.meetup.com/Big-Data-Demystified

Facebook Group :

https://www.facebook.com/groups/amazon.aws.big.data.demystified/

Facebook page (https://www.facebook.com/Amazon-AWS-Big-Data-Demystified-1832900280345700/)

Audience:

Data Engineers
Data Science
DevOps Engineers
Big Data Architects
Solution Architects
CTO
VP R&D

AWS Big Data Demystified

Tel Aviv-Yafo, IL
729 Members

A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The techn…

Check out this Meetup Group →

Some good reads about Apache Drill. drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. For example, you can join a user profile collection in MongoDB with a directory of event logs in Hadoop.

Drill’s datastore-aware optimizer automatically restructures a query plan to leverage the datastore’s internal processing capabilities. In addition, Drill supports data locality, so it’s a good idea to co-locate Drill and the datastore on the same nodes.

Traditional query engines demand significant IT intervention before data can be queried. Drill gets rid of all that overhead so that users can just query the raw data in-situ. There’s no need to load the data, create and maintain schemas, or transform the data before it can be processed. Instead, simply include the path to a Hadoop directory, MongoDB collection or S3 bucket in the SQL query.

Drill leverages advanced query compilation and re-compilation techniques to maximize performance without requiring up-front schema knowledge.

Drill features a JSON data model that enables queries on complex/nested data as well as rapidly evolving structures commonly seen in modern applications and non-relational datastores. Drill also provides intuitive extensions to SQL so that you can easily query complex data.

Drill is the only columnar query engine that supports complex data. It features an in-memory shredded columnar representation for complex data which allows Drill to achieve columnar speed with the flexibility of an internal JSON document model.

https://drill.apache.org/docs/architecture-introduction/

SQL parser: Drill uses Calcite, the open source SQL parser framework, to parse incoming queries. The output of the parser component is a language agnostic, computer-friendly logical plan that represents the query.

What You Will Learn:

Collecting data is easy; getting real value from that data is more difficult. The first step in creating value-adding data is having a structured business plan- for both analytics and BI – that will guide all future data operations.

This event aims to practically break down the steps in formulating a data analytics and BI strategy. The talks will broaden your own understanding of how to use a strategy to integrate data into your overall business goals, and will provide you with actionable steps to both think about and create your own strategy.

MAKING YOUR ANALYTICS TALK BUSINESS

Aligning your analysis to the business is fundamental for all types of analytics (digital or product analytics, business intelligence, etc) and is vertical- and tool agnostic. In this talk we will build on the discussion that was started in the previous meetup, and will discuss how analysts can learn to derive their stakeholders’ expectations, how to shift from metrics to “real” KPIs, and how to approach an analysis in order to create real impact.

This session is primarily geared towards those starting out into analytics, practitioners who feel that they are still struggling to prove their value in the organization or simply folks who want to power up their reporting and recommendation skills. If you are already a master at aligning your analysis to the business, you’re most welcome as well: join us to share your experiences so that we can all learn from each other and improve!

Bios:

Eliza Savov – Eliza is the team lead of the Customer Experience and Analytics team at Clicktale, the worldwide leader in behavioral analytics. She has extensive experience working with data analytics, having previously worked at Clicktale as a senior customer experience analyst, and as a product analyst at Seeking Alpha.

Structuring a Strategy: Creating a BI & Analytics Business Plan

Sunday, Jul 21, 2019, 6:00 PM

Google for Startups Campus
Ha-Umanim St 12 Tel Aviv-Yafo, IL

154 Members Went

Learn how to make a structured business plan for your analytics and BI operations that will add value to your bottom line. Agenda: 18:00 PM – Networking 18:30 PM – MAKING YOUR ANALYTICS TALK BUSINESS, by Eliza Savov, Team Lead, Customer Experience and Analytics at Clicktale 19:15 – Break 19:30 PM – BI STRATEGY FROM A BIRD’S EYE VIEW, by Omri Halak,…

Check out this Meetup →

In the talk we will discuss how to break down the company’s overall goals all the way to your BI team’s daily activities in 3 simple stages:

1. Understanding the path to success – Creating a revenue model
2. Gathering support and strategizing – Structuring a team
3. Executing – Tracking KPIs

Bios:

Omri Halak -Omri is the director of business operations at Logz.io, an intelligent and scalable machine data analytics platform built on ELK & Grafana that empowers engineers to monitor, troubleshoot, and secure mission-critical applications more effectively. In this position, Omri combines actionable business insights from the BI side with fast and effective delivery on the Operations side. Omri has ample experience connecting data with business, with previous positions at SimilarWeb as a business analyst, at Woobi as finance director, and as Head of State Guarantees at Israel Ministry of Finance.

Structuring a Strategy: Creating a BI & Analytics Business Plan

Sunday, Jul 21, 2019, 6:00 PM

Google for Startups Campus
Ha-Umanim St 12 Tel Aviv-Yafo, IL

154 Members Went

Learn how to make a structured business plan for your analytics and BI operations that will add value to your bottom line. Agenda: 18:00 PM – Networking 18:30 PM – MAKING YOUR ANALYTICS TALK BUSINESS, by Eliza Savov, Team Lead, Customer Experience and Analytics at Clicktale 19:15 – Break 19:30 PM – BI STRATEGY FROM A BIRD’S EYE VIEW, by Omri Halak,…

Check out this Meetup →