How to connect to Google Ad Manager via Airflow

Author: Omid Vahdaty 27.2.2020

  1. To simply authentication, You need a G Suite gmail user with admin permission to GAM console.
  2. make sure API access is enable in Google Ad Manager
  3. You need a network ID from google ad manager.
  4. Install a GCE machine with debian, Notice your need python 3.6 for any version higher than v201911
sudo apt update
sudo apt -y install python-pip python3-venv python3-pip
sudo apt-get -y install git
git clone https://github.com/googleads/googleads-python-lib.git
pip3 install googleads
pip install google-auth-oauthlib

and run example

cd ~/googleads-python-lib/examples/ad_manager/v201911/report_service
python3 run_reach_report.py

This will provide an authentication error:

  File "/home/analyticsynet/.local/lib/python3.5/site-packages/googleads/common.py", line 248, in LoadFromStorage
    'Given yaml file, %s, could not be opened.' % path)
googleads.errors.GoogleAdsValueError: Given yaml file, /home/omid/googleads.yaml, could not be opened.

you need to create OAuth json and get cilentID and ClientSecret, follow these instructions:

https://github.com/googleads/googleads-python-lib#getting-started

Notice OAuth should be blank for all fields and application type is “other” as stated here

and test authentication via:

cd ~/googleads-python-lib/examples/ad_manager/authentication
python generate_refresh_token.py --client_id INSERT_CLIENT_ID --client_secret INSERT_CLIENT_SECRET

if you are getting an error like the below, this means you have not chosen application type “other” in OAuth:

Issue:
The redirect URI in the request, urn:ietf:wg:oauth:2.0:oob, can only be used by a Client ID for native application. It is not allowed for the WEB client type. You can create a Client ID for native application at

Once you do that – you can now authenticate via the the googleads.yaml file :

  1. Copy the googleads.yaml file to your home directory.
  2. This will be used to store credentials and other settings that can be loaded to initialize a client.
  3. update client and secret client inside the YAML
  4. You also need to get network code in GAM.

If this does not work – try the services account options this manul to get the high level process:

  1. create a service account in https://console.developers.google.com/ , don’t forget to choose json method and download the json private key – this can only happen once. this will create an email like: omid-test1@myproject.iam.gserviceaccount.com
  2. Add service account in google ad manger console. use the email above section.
  3. confirm the user is in Active status before continuing.
  4. copy the json private key to the machine home folder. Icalled it my_gcp_private_key_service_account.json
  5. setup the googleads.yaml file:
ad_manager:
  application_name: INSERT_APPLICATION_NAME_HERE
  network_code: INSERT_NETWORK_CODE_HERE
  path_to_private_key_file: INSERT_PATH_TO_FILE_HERE

You can test quickly via adding json path of the private key of the above services account. update the KEY_FILE (your json key), APPLICATION_NAME (your application)

~/googleads-python-lib/examples/ad_manager/authentication
nano create_ad_manager_client_with_service_account.py 

run the script:

python3 create_ad_manager_client_with_service_account.py 

Expected output

This library is being run by an unsupported Python version (3.5.3). In order to benefit from important security improvements and ensure compatibility with this libr
ary, upgrade to Python 3.6 or higher.
Network with network code "1234" and display name "myRealAppName" was found.

a sample report example with this services account connection in our Git:

#!/usr/bin/env python
#
# Copyright 2014 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Initializes a AdManagerClient using a Service Account."""
from googleads import ad_manager
from googleads import oauth2
from googleads import errors
import tempfile
# OAuth2 credential information. In a real application, you'd probably be
# pulling these values from a credential storage.
KEY_FILE = '/home/omid/111bd7ea6ae8534.json'
# Ad Manager API information.
APPLICATION_NAME = 'myApp'
NETWORK_CODE='6690'
def main(key_file, application_name):
  oauth2_client = oauth2.GoogleServiceAccountClient(
      key_file, oauth2.GetAPIScope('ad_manager'))
  client = ad_manager.AdManagerClient(
      oauth2_client, application_name, NETWORK_CODE)
  #networks = ad_manager_client.GetService('NetworkService').getAllNetworks()
  #for network in networks:
  #  print('Network with network code "%s" and display name "%s" was found.'
  #        % (network['networkCode'], network['displayName']))
  # Initialize a DataDownloader.
  report_downloader = client.GetDataDownloader(version='v201911')
 # Create report job.
  report_job = {
      'reportQuery': {
          'dimensions': ['DATE', 'AD_UNIT_NAME'],
          'adUnitView': 'HIERARCHICAL',
          'columns': ['AD_SERVER_IMPRESSIONS', 'AD_SERVER_CLICKS',
                      'ADSENSE_LINE_ITEM_LEVEL_IMPRESSIONS',
                      'ADSENSE_LINE_ITEM_LEVEL_CLICKS',
                      'TOTAL_LINE_ITEM_LEVEL_IMPRESSIONS',
                      'TOTAL_LINE_ITEM_LEVEL_CPM_AND_CPC_REVENUE'],
          'dateRangeType': 'LAST_WEEK'
      }
  }
  try:
    # Run the report and wait for it to finish.
    report_job_id = report_downloader.WaitForReport(report_job)
  except errors.AdManagerReportError as e:
    print('Failed to generate report. Error was: %s' % e)
 
 # Change to your preferred export format.
  export_format = 'CSV_DUMP'
  report_file = tempfile.NamedTemporaryFile(suffix='.csv.gz', delete=False)
  # Download report data.
  report_downloader.DownloadReportToFile(
      report_job_id, export_format, report_file)
  report_file.close()
  
    # Display results.
  print('Report job with id "%s" downloaded to:\n%s' % (
      report_job_id, report_file.name))
if __name__ == '__main__':
  main(KEY_FILE, APPLICATION_NAME)

You could you date range enums to in ReportService.DateRangeType and change the date range, but that would not fit into your airflow plan. Think Dynamic operators like we did in similarweb airflow blog. you may also need to overwrite partitions of dates.

Lets now you wish to add to python script command line argument of start date and end date. full example of google ad manager python with date range committed to out git and is also below:

import sys, getopt
from datetime import datetime
from datetime import timedelta
from googleads import ad_manager
from googleads import oauth2
from googleads import errors
import tempfile
# OAuth2 credential information. In a real application, you'd probably be
# pulling these values from a credential storage.
KEY_FILE = '/home/omid/my_gcp_private_key_service_account.json'
# Ad Manager API information.
APPLICATION_NAME = 'jutomate'
NETWORK_CODE='6690'
def report(key_file, application_name,startDate,endDate):
  oauth2_client = oauth2.GoogleServiceAccountClient(
      key_file, oauth2.GetAPIScope('ad_manager'))
  client = ad_manager.AdManagerClient(
      oauth2_client, application_name, NETWORK_CODE)
  #networks = ad_manager_client.GetService('NetworkService').getAllNetworks()
  #for network in networks:
  #  print('Network with network code "%s" and display name "%s" was found.'
  #        % (network['networkCode'], network['displayName']))
  # Initialize a DataDownloader.
  report_downloader = client.GetDataDownloader(version='v201911')
  # Set the start and end dates of the report to run (past 0 days, you can change to what u need).
  end_date = datetime.strptime( startDate, "%Y-%m-%d").date()
  
  start_date = datetime.strptime( endDate, "%Y-%m-%d").date()
  
  print ('start_date: ', start_date)
  print ('end_date: ', end_date)
  
  report_filename_prefix='report_example_using_service_account_with_date_range'
  
 # Create report job.
  report_job = {
      'reportQuery': {
          'dimensions': ['DATE', 'AD_UNIT_NAME'],
          'adUnitView': 'HIERARCHICAL',
          'columns': ['AD_SERVER_IMPRESSIONS', 'AD_SERVER_CLICKS',
                      'ADSENSE_LINE_ITEM_LEVEL_IMPRESSIONS',
                      'ADSENSE_LINE_ITEM_LEVEL_CLICKS',
                      'TOTAL_LINE_ITEM_LEVEL_IMPRESSIONS',
                      'TOTAL_LINE_ITEM_LEVEL_CPM_AND_CPC_REVENUE'],
          'dateRangeType': 'CUSTOM_DATE',
          'startDate': start_date,
          'endDate': end_date
      }
  }
  try:
    # Run the report and wait for it to finish.
    report_job_id = report_downloader.WaitForReport(report_job)
  except errors.AdManagerReportError as e:
    print('Failed to generate report. Error was: %s' % e)
 
 # Change to your preferred export format.
  export_format = 'CSV_DUMP'
  report_file = tempfile.NamedTemporaryFile(suffix='_'+report_filename_prefix+'_'+startDate+'__'+endDate+'.csv.gz', delete=False)
  # Download report data.
  report_downloader.DownloadReportToFile(
      report_job_id, export_format, report_file)
  report_file.close()
  
    # Display results.
  print('Report job with id "%s" downloaded to:\n%s' % (
      report_job_id, report_file.name))
def main(argv):
   startDate = ''
   endDate = ''
   try:
      opts, args = getopt.getopt(argv,"hi:o:",["start=","end="])
   except getopt.GetoptError:
      print ('example_python_command_line_arguments.py -s  -e ')
      sys.exit(2)
   for opt, arg in opts:
      if opt == '-h':
         print ('example_python_command_line_arguments.py -s  -e ')
         sys.exit()
      elif opt in ("-s", "--start"):
         startDate = arg
      elif opt in ("-e", "--end"):
         endDate = arg
   print ('start date is ', startDate)
   print ('end   date is ', endDate)
   report(KEY_FILE, APPLICATION_NAME,startDate,endDate)
   
if __name__ == '__main__':
  main(sys.argv[1:])
© 2020 GitHub, Inc.
Terms
Privacy
Security
Status

——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with,

feel free to contact me via LinkedIn:

1 thought on “How to connect to Google Ad Manager via Airflow”

Leave a Reply