If you are using the Ubuntu Linux DSVM edition, there is a requirement to do a one-time setup step to enable a local single node Hadoop HDFS and YARN instance. By default, Hadoop services are installed but disabled on the DSVM. In order to enable it, it is necessary to run the following commands as root the first time:
echo -e 'y\n' | ssh-keygen -t rsa -P '' -f ~hadoop/.ssh/id_rsa
cat ~hadoop/.ssh/id_rsa.pub >> ~hadoop/.ssh/authorized_keys
chmod 0600 ~hadoop/.ssh/authorized_keys
chown hadoop:hadoop ~hadoop/.ssh/id_rsa
chown hadoop:hadoop ~hadoop/.ssh/id_rsa.pub
chown hadoop:hadoop ~hadoop/.ssh/authorized_keys
systemctl start hadoop-namenode hadoop-datanode hadoop-yarn
You can stop the Hadoop-related services when you do not need them by executing the following command:
systemctl stop hadoop-namenode hadoop-datanode...
