Add Flume in Ambari
- Click on Aded Service from the Ambari interface.
- Flume service available in HDP is 1.5.2. Choose this service to be installed.
- Pick where to install the Flume service. In this case, Flume is added to the namenode. The services can be moved to another node by using Ambari.
- In step Customize Services, Flume agent can be configured. This can be done after the service is installed. For now, let it be empty.
- In step Review, click on Deploy
- After the install, the service is started and tested. If everything goes well, the green progress bar shows up
- The summary warns you that some services would have to be restarted so that Flume can function properly. This is a generic message. In case of installing only Flume, no restart of existing services is needed.
Work in Linux
- User flume is added automatically by Ambari and it belongs to group hadoop.
Work in HDFS
- In order for user flume to work properly on HDFS, flume folder has to be created under /user in HDFS. For example, in case of deleting files in HDFS as user flume, deleted files are moved to /user/flume.Create /user/flume in HDFS.
sudo -u hdfs hadoop fs -mkdir /user/flume
Give ownership to user flume.
sudo -u hdfs hadoop fs -chown flume /user/flume
Give read, write and execute to flume and flume’s HDFS group – hdfs.
sudo -u flume hadoop fs -chmod 770 /user/flume