Here is a bash script I used a couple of years ago for creating Hadoop users from CLI (or batch). It might be useful for someone.
The script does the following:
- creates a Linux user
- generates keys
- creates home directory in HDFS
- adds user to a group
- allocates HDFS space quota
- gives access in Ambari Views
#!/bin/bash
NEW_USER="$1"
DEPT_NAME="$2"
NAMENODE="t-namenode1"
AMBARI="t-ambari"
#
echo "Creating user "$NEW_USER
#Creating user with no password with user's folder
sudo adduser --disabled-password --gecos "" $NEW_USER
#Create Linux user on the namenode
ssh -i /home/ubuntu/.ssh/key $NAMENODE 'sudo adduser --disabled-password --gecos "" $NEW_USER && sudo chown $NEW_USER:$NEW_USER /home/$NEW_USER'
#Prepare .ssh folder
cd /user/$NEW_USER
sudo mkdir .ssh
sudo chown $NEW_USER:$NEW_USER .ssh/
sudo chmod 700 .ssh
#Create private and public key
sudo -u $NEW_USER ssh-keygen -t rsa -f $NEW_USER-key
#Copy public key to the authorized_keys
sudo -u $NEW_USER cp $NEW_USER-key.pub .ssh/authorized_keys
sudo -u $NEW_USER chmod 600 .ssh/authorized_keys
#######HDFS
echo "Create system folder for user"
sudo -u hdfs hadoop fs -mkdir /user/$NEW_USER
echo "Change owner of the system folder"
sudo -u hdfs hadoop fs -chown $NEW_USER:hdfs /user/$NEW_USER
#Defining HDFS space quota
echo "Allocate 100g of space on HDFS for the user"
sudo -su hdfs hdfs dfsadmin -setSpaceQuota 100g /department/$DEPT_NAME/users/$NEW_USER
#Access to Ambari Views
curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{"Users/user_name": "$USER_NAME", "Users/password": "$USER_NAME", "Users/active": true, "Users/admin": false }' http://$AMBARI:8080/api/v1/users
#Add user to a group in Ambari Views
curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '[{"MemberInfo/user_name":"$NEW_USER", "MemberInfo/group_name":"$DEPT_NAME"}]' http://$AMBARI:8080/api/v1/groups/$DEPT_NAME/members
echo "User's folder on the client:"
ls -l /user/$NEW_USER
echo "User's system folder on HDFS:"
sudo -u $HDFS hadoop fs -ls /user/$NEW_USER
hi markobigdata i ran your script with some issues maybe i dont understand something else
i need change this parameters?
NEW_USER=”$1″ in this need repalce the “NEW_USER?
DEPT_NAME=”$2″
NAMENODE=”t-namenode1″ i n this i need repalce the name of namenode?
AMBARI=”t-ambari” in this why need repalce?
and i not have ubuntu have centos this line need replace?
#Create Linux user on the namenode
ssh -i /home/ubuntu/.ssh/key $NAMENODE ‘sudo adduser –disabled-password –gecos “” $NEW_USER && sudo chown $NEW_USER:$NEW_USER /home/$NEW_USER’
LikeLike
$1 is the first parameter of shell script that adds new user. $2 is the second parameter. First one gives username, second one gives groups.
path to key is your path to the private key (default is id_rsa)
LikeLike