Big Query, superQuery

Quick Start on GCP BigQuery – 5 rules of thumb

1. Install BQ mate – plugin for chrome in order to see cost per query in dollars before running.

2. log in to your BigQuery console to get started, consider using superQuery IDE for more advanced features over BigQuery, Increased productivity and cost reduction.

3. run sample query , ALWAYS use a date in the where clause, assuming the table is partitioned, this will save you costs by only scanning the relevant data as oppose to the entire table.

select * from 
where date = '2019-11-01'

4. AS rule of thumb, NEVER DO the below:

select * from t
or
select * from t limit 100

as this will incur unintended full scans of table data and thus create huge billing costs. Instead, use:

Select ColName1, ColName2 from T

and Preview table instead of LIMIT


5. as a rule of thumb , ALWAYS
:

if you are running a query more than 1$ Рyou are most probably doing something wrong.

Some official GCP BigQuery tutorials:

——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

https://www.linkedin.com/in/omid-vahdaty/

Leave a Reply