AWS EMR, open source

Bootstrapping Oozie

Bootstrap Oozie

  1. define admin user 
    cd /usr/lib/hue
    sudo build/env/bin/hue  createsuperuser

    sudo build/env/bin/hue changepassword userName

    Username (leave blank to use 'root'): <enter the super user name>
    Email address: <your email id>
    Password: <password with one upper case, number, and special character>
    Password (again): 
    Superuser created successfully.
  2. copy files from source EMR

    hadoop fs -get /user/hue/oozie/ /mnt/

    aws s3 mv deployments s3://walla-oozie/ –recursive

    aws s3 cp s3://walla-oozie/ oozie/ –recursive

    hadoop fs -put oozie/workspaces/ /user/hue/oozie

  3. imports /export jsons

    1. source emr
      1. cd /usr/lib/hue
      2. sudo ./build/env/bin/hue dumpdata desktop.Document2 –indent 2 –natural > /home/hadoop/data.json

      3. aws s3 cp /home/hadoop/data.json s3://walla-oozie/

    2. import emr

      1. aws s3 cp s3://walla-oozie/data.json /home/hadoop/

      2. cd /usr/lib/hue
      3. ./build/env/bin/hue loaddata /home/hadoop/data.json

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s