hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kartashov, Andy" <Andy.Kartas...@mpac.ca>
Subject RE: starting Daemons remotely, through ssh - SOLVED
Date Tue, 06 Nov 2012 20:46:00 GMT
Forrest,

I must admit. I am very new myself.  Two months ago I didn't know what "$sudo" was. :) So,
do not take my comments too seriously. I could be wrong about many issues.


There are 5 daemon scripts  in /etc/init.d/. I am using MRv1, so mine are:

hadoop-hdfs-datanode
hadoop-hdfs-namenode
hadoop-hdfs-secondarynamenode
<specific to MRv1>
hadoop-0.20-mapreduce-jobtracker
hadoop-0.20-mapreduce-tasktracker

For scalability I chose to:

-           only have NN/JT on my master node so I need to start|stop|restart only hadoop-hdfs-namenode/hadoop-0.20-mapreduce-jobtracker

-          different node for SNN so I need to only start|stop|restart hadoop-hdfs-secondarynamenode

-          all of my DNs - hadoop-hdfs-datanode/hadoop-0.20-mapreduce-tasktracker
My script is VERY simple for now:

#!/bin/bash
# myHadoop.sh
dir=/etc/init.d
if [ $# -lt 1 ]; then
  echo "USAGE: $`basename $0` [start|stop|restart]"
  exit 1
fi
sudo ${dir}/hadoop-hdfs-namenode  $1
sudo ${dir}/hadoop-0.20-mapreduce-jobtracker  $1
exit 0

for SNN I altered script to run instead:
sudo ${dir}/hadoop-hdfs-secondarynamenode  $1

      and for DNs:
sudo ${dir}/hadoop-hdfs-datanode  $1
sudo ${dir}/hadoop-0.20-mapreduce-tasktracker $1

p.s. Criticism on my post is welcomed.

AK47


From: Forrest Aldrich [mailto:forrie@gmail.com]
Sent: Tuesday, November 06, 2012 3:09 PM
To: user@hadoop.apache.org
Subject: Re: starting Daemons remotely, through ssh - SOLVED

Andy, I noticed you are splitting up your /etc/init.d startup -- we're new-ish to Hadoop and
found that the supplied init.d script doesn't necessarily stop services properly, so I end
up doing it manually.   If you could share your info that would be great... I can see where
they can be time-delayed in the startup with chkconfig (RHEL).


Thanks.
On 11/6/12 3:07 PM, Kartashov, Andy wrote:
Harsh/Ravi,

I wrote my own scripts to [start|stop|restart]  [hdfs|mapred] daemons. The content of the
script is

$sudo service /etc/init.d/hadoop-[hdfs-*|mapred-*]

I have no problem starting Daemons locally on each node. I have sudo-access granted. I run
script and enter password.
I have passwordless ssh enabled. But when I am trying to start slave Daemons from NN I get
"<remothsot>:  sudo: sorry, you must have a tty to run sudo".

I just solved this issue by:

a.       Modifying slaves.sh script to add -t after ssh "-t" <host-ip> <command>


a1. Someone suggested to:

$sudo sudeors and commenting out Defaults requiretty

But a. worked just fine.



Cheers,

AK47

From: Ravi Mutyala [mailto:ravi@hortonworks.com]
Sent: Tuesday, November 06, 2012 2:31 PM
To: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: Re: starting Daemons remotely, through ssh

Andy,

Which specific scripts are these? As far as I know, all daemons on the hadoop side can be
started with hadoop/hdfs or any non-sudo username. You will require passwordless ssh for that
user if you are using slaves/regionservers configured to start the daemons remotely from the
NameNode/HBase Master.


On Tue, Nov 6, 2012 at 1:25 PM, Kartashov, Andy <Andy.Kartashov@mpac.ca<mailto:Andy.Kartashov@mpac.ca>>
wrote:
Hadoopers,

How does one start Daemons remotely when scripts normally require root user to start them?
Do you modify scripts?

Thanks,
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

Mime
View raw message