AWS EMR, open source

Bootstrapping Oozie

Omid

Bootstrap Oozie

  1. define admin user  
    cd /usr/lib/hue
sudo build/env/bin/hue  createsuperuser

    sudo build/env/bin/hue changepassword userName

    

Username (leave blank to use 'root'): <enter the super user name>
Email address: <your email id>
Password: <password with one upper case, number, and special character>
Password (again): 
Superuser created successfully.
  2. copy files from source EMR

    hadoop fs -get /user/hue/oozie/ /mnt/

    aws s3 mv deployments s3://walla-oozie/ –recursive

    aws s3 cp s3://walla-oozie/ oozie/ –recursive

    hadoop fs -put oozie/workspaces/ /user/hue/oozie

  3. imports /export jsons http://gethue.com/export-and-import-your-oozie-workflows/

    1. source emr
      1. cd /usr/lib/hue

      2. sudo ./build/env/bin/hue dumpdata desktop.Document2 –indent 2 –natural > /home/hadoop/data.json

      3. aws s3 cp /home/hadoop/data.json s3://walla-oozie/

    2. import emr

      1. aws s3 cp s3://walla-oozie/data.json /home/hadoop/

      2. cd /usr/lib/hue
      3. ./build/env/bin/hue loaddata /home/hadoop/data.json



——————————————————————————————————————————

I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

https://www.linkedin.com/in/omid-vahdaty/



Leave a Reply