analytics, meetup, Uncategorised

BI STRATEGY FROM A BIRD’S EYE VIEW | BI and Analytics Demystified |Omri Halak, Director of Business Operations at Logz.io

In the talk we will discuss how to break down the company’s overall goals all the way to your BI team’s daily activities in 3 simple stages:

1. Understanding the path to success – Creating a revenue model
2. Gathering support and strategizing – Structuring a team
3. Executing – Tracking KPIs

Bios:

Omri Halak -Omri is the director of business operations at Logz.io, an intelligent and scalable machine data analytics platform built on ELK & Grafana that empowers engineers to monitor, troubleshoot, and secure mission-critical applications more effectively. In this position, Omri combines actionable business insights from the BI side with fast and effective delivery on the Operations side. Omri has ample experience connecting data with business, with previous positions at SimilarWeb as a business analyst, at Woobi as finance director, and as Head of State Guarantees at Israel Ministry of Finance.

Structuring a Strategy: Creating a BI & Analytics Business Plan

Sunday, Jul 21, 2019, 6:00 PM

Google for Startups Campus
Ha-Umanim St 12 Tel Aviv-Yafo, IL

154 Members Went

Learn how to make a structured business plan for your analytics and BI operations that will add value to your bottom line. Agenda: 18:00 PM – Networking 18:30 PM – MAKING YOUR ANALYTICS TALK BUSINESS, by Eliza Savov, Team Lead, Customer Experience and Analytics at Clicktale 19:15 – Break 19:30 PM – BI STRATEGY FROM A BIRD’S EYE VIEW, by Omri Halak,…

Check out this Meetup →

——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

https://www.linkedin.com/in/omid-vahdaty/

Uncategorised

Big Data Demystified |Using AI to optimize BigQuery

“Using AI to optimize BigQuery”
As we kick off 2019, one thing is certain: the “Serverless Revolution” has hit the mainstream. However, whereas a majority of serverless discussions tend to focus on its utility in software development, there is a parallel paradigm shift occurring in big data analytics.

With fully-managed data warehouses like Google BigQuery and Amazon Athena, users can start querying petabytes of data at unprecedented speed, paying only for the compute and storage used. Small startups to companies as large as Spotify and Home Depot are migrating to this new way of doing analytics every day.

However, it’s on-demand pricing can be a double-edged sword with data analysts and teams. Scan more data than is necessary, and your company will wind up with an expectedly large bill at the end of the month. Worst of all, with no real-time price transparency while querying, team’s don’t learn about this until it’s too late. As a result, companies set usage limits on their analysts.

This is not an efficient approach for data analytics services as powerful as BigQuery or Athena, as it sets a ceiling to an analyst’s potential contribution. The solution? SQL optimization.

Avi Zloof and Eben du Toit, CEO and Chief Data Scientist at superQuery, respectively, will discuss building ML models to optimize SQL queries and how that removes the need for any restrictions on data analysts.

https://www.youtube.com/watch?v=cXq2tffYQ-A
about the lecturers:
—————————-

Avi Zloof:
Avi has spent the past 20 years leading R&D and innovation teams, managing big data and full stack development initiatives.
Prior to founding superQuery he worked for six years at TradAir leading cloud and big data projects. Big data tools he’s developed are being used by Tier-1 banks daily to trade and analyze billions in currencies.

Ido Vollf:

Ido is a serial entrepreneur who previously founded Sleeve (Edtech), EmbraceMe (IoT), and ChaChange (FinTech).
Ido worked as a Technical and Business Evangelist at Microsoft and was in charge of communicating the value of Microsoft Azure and Windows 10 to Israel’s startup ecosystem.
He’s an active mentor in the Israeli startup ecosystem, helping early stage startups grow fast.

Eben du Toit: Storyteller, data scientist, tech geek and computer engineer.

I’m a storyteller, data scientist, tech geek, computer engineer and control systems expert. My data skills were honed conceptualising, implementing and leading the efforts in building data science stacks. During my engineering career I’ve designed, coded, installed and tested full-stack large-scale IT engineering infrastructure projects at several power plants across South Africa and developed software for both mobile back-end platforms and power industry applications. I have over 14 years experience in engineering and grew up as a child of the internet. Currently chief data scientist at superQuery.

 

contact me for more details or questions (would love to help)

Want more big data quality content? Join our meetup, subscribe to youtube channels

For more information about Superquery:
https://web.superquery.io/?camp=BDD

——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

https://www.linkedin.com/in/omid-vahdaty/

Uncategorised

setting the default interpreter of zeppelin for bootstrapping

When you bootstrap a new EMR zeppelin, once you open the notebook, you will be asked to save the default interpreter. in transient cluster you may want to set the default interpreter automatically.

To set the default interpreter, check /etc/zeppelin/conf/interpreter.json and look for something like:

...
{
  "name": "spark",
  "class": "org.apache.zeppelin.spark.SparkInterpreter",
  "defaultInterpreter": true,
  "editor": {
    "language": "scala",
    "editOnDblClick": false
  }
},
...
{
  "name": "pyspark",
  "class": "org.apache.zeppelin.spark.PySparkInterpreter",
  "defaultInterpreter": false,
  "editor": {
    "language": "python",
    "editOnDblClick": false
  }
}

Now everything seems trivial. We just need to change the defaultInterpreter of spark to false, and defaultInterpreter of pyspark to true.

And then restart the zeppelin

(sudo stop zeppelin; sudo start zeppelin).

 

——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

https://www.linkedin.com/in/omid-vahdaty/

Uncategorised

Running complete notebooks Zeppelin API https via java

here is a small code snippet example to allow any java user to login into zeppelin and run complete notebooks.

 

this is the main function

public void zeppelinApiRunNotebook() throws ClientProtocolException, IOException, AuthenticationException, NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException

{

LogUtil.logUtilWriteInfo(“”);

DefaultHttpClient httpclient = new DefaultHttpClient();

String cookieId = zeppelinApiLogin(httpclient);

String postData = https://”+ZeppelinURL+“/api/notebook/job/”+ZeppelinNotebookId;

HttpPost postRequest = new HttpPost(postData);

List<NameValuePair> formparams = new ArrayList<NameValuePair>();

formparams.add(new BasicNameValuePair(“JSESSIONID”, cookieId));

postRequest.setEntity(new UrlEncodedFormEntity(formparams, “UTF-8”));

CloseableHttpResponse response1 = httpclient.execute(postRequest);

System.out.println(response1);

response1.close();

httpclient.close();

}

smaller function, to login to zeppelin, and to cookie ID

private static String zeppelinApiLogin(DefaultHttpClient httpclient) throws ClientProtocolException, IOException, AuthenticationException, NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException

{

String urlLogin = https://&#8221;+ZeppelinURL+“/api/login”;

    HttpPost postRequest = new HttpPost(urlLogin);

    List<NameValuePair> formparams = new ArrayList<NameValuePair>();

    formparams.add(new BasicNameValuePair(“userName”, Prop.getProperty(“user_name”)));

    formparams.add(new BasicNameValuePair(“password”,Prop.getProperty(“password”)));

    postRequest.setEntity(new UrlEncodedFormEntity(formparams, “UTF-8”));

    CloseableHttpResponse response = httpclient.execute(postRequest);

    System.out.println(response);

    response.close();

    return parseSessionID(response);

     

}

private static String parseSessionID(HttpResponse response) {

LogUtil.logUtilWriteDebug(“”);

    String nid= “”;

   

try {

       

        Header[] header = response.getAllHeaders();

        for (int i = 0; i < header.length; i++)

        {

        String value = header[i].getValue();

   

        if (value.contains(“JSESSIONID”)) {

            int index = value.indexOf(“JSESSIONID=”);

            int endIndex = value.indexOf(“;”, index);

            String sessionID = value.substring(

                    index + “JSESSIONID=”.length(), endIndex);

            nid=sessionID;

     

        }

        }

       

    } catch (Exception e) {

    LogUtil.logUtilWriteError(“can’t read cookie from zeppelin login”);

    }

       return nid;

    }