Here is a bash script I used a couple of years ago for creating Hadoop users from CLI (or batch). It might be useful for someone.
The script does the following:
- creates a Linux user
- generates keys
- creates home directory in HDFS
- adds user to a group
- allocates HDFS space quota
- gives access in Ambari Views
#!/bin/bash NEW_USER="$1" DEPT_NAME="$2" NAMENODE="t-namenode1" AMBARI="t-ambari" # echo "Creating user "$NEW_USER #Creating user with no password with user's folder sudo adduser --disabled-password --gecos "" $NEW_USER #Create Linux user on the namenode ssh -i /home/ubuntu/.ssh/key $NAMENODE 'sudo adduser --disabled-password --gecos "" $NEW_USER && sudo chown $NEW_USER:$NEW_USER /home/$NEW_USER' #Prepare .ssh folder cd /user/$NEW_USER sudo mkdir .ssh sudo chown $NEW_USER:$NEW_USER .ssh/ sudo chmod 700 .ssh #Create private and public key sudo -u $NEW_USER ssh-keygen -t rsa -f $NEW_USER-key #Copy public key to the authorized_keys sudo -u $NEW_USER cp $NEW_USER-key.pub .ssh/authorized_keys sudo -u $NEW_USER chmod 600 .ssh/authorized_keys #######HDFS echo "Create system folder for user" sudo -u hdfs hadoop fs -mkdir /user/$NEW_USER echo "Change owner of the system folder" sudo -u hdfs hadoop fs -chown $NEW_USER:hdfs /user/$NEW_USER #Defining HDFS space quota echo "Allocate 100g of space on HDFS for the user" sudo -su hdfs hdfs dfsadmin -setSpaceQuota 100g /department/$DEPT_NAME/users/$NEW_USER #Access to Ambari Views curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{"Users/user_name": "$USER_NAME", "Users/password": "$USER_NAME", "Users/active": true, "Users/admin": false }' http://$AMBARI:8080/api/v1/users #Add user to a group in Ambari Views curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '[{"MemberInfo/user_name":"$NEW_USER", "MemberInfo/group_name":"$DEPT_NAME"}]' http://$AMBARI:8080/api/v1/groups/$DEPT_NAME/members echo "User's folder on the client:" ls -l /user/$NEW_USER echo "User's system folder on HDFS:" sudo -u $HDFS hadoop fs -ls /user/$NEW_USER
hi markobigdata i ran your script with some issues maybe i dont understand something else
i need change this parameters?
NEW_USER=”$1″ in this need repalce the “NEW_USER?
DEPT_NAME=”$2″
NAMENODE=”t-namenode1″ i n this i need repalce the name of namenode?
AMBARI=”t-ambari” in this why need repalce?
and i not have ubuntu have centos this line need replace?
#Create Linux user on the namenode
ssh -i /home/ubuntu/.ssh/key $NAMENODE ‘sudo adduser –disabled-password –gecos “” $NEW_USER && sudo chown $NEW_USER:$NEW_USER /home/$NEW_USER’
LikeLike
$1 is the first parameter of shell script that adds new user. $2 is the second parameter. First one gives username, second one gives groups.
path to key is your path to the private key (default is id_rsa)
LikeLike