Hadoop clusters require passwordless shh between nodes for proper communication.
This is all done on the instance you wish to connect FROM!
The recipe how I made paswordless shh work between two instances is the following:
- create ec2 instances – they should be in the same subnet and have the same security group
- Open ports between them – make sure instances can communicate to each other. Use the default security group which has one rule relevant for this case:
- Type: All Traffic
- Source: Custom – id of the security group
- Log in to the instance you want to connect from to the other instance
ssh-keygen -t rsa -N "" -f /home/ubuntu/.ssh/id_rsa
to generate a new rsa key.
- Copy your private AWS key as ~/.ssh/my.key (or whatever name you want to use)
- Make sure you change the permission to 600
chmod 600 .ssh/my.key
- Copy the public key to the instance you wish to connect to passwordless
cat ~/.ssh/id_rsa.pub | ssh -i ~/.ssh/my.key email@example.com.X "cat >> ~/.ssh/authorized_keys"
If you test the passwordless ssh to the other machine, it should work.