hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kartashov, Andy" <Andy.Kartas...@mpac.ca>
Subject cluster set-up / a few quick questions
Date Fri, 26 Oct 2012 16:40:29 GMT

- do you put Master's node <hostname> under fs.default.name in core-site.xml on the
slave machines or slaves' hostnames?
- do you need to run "sudo -u hdfs hadoop namenode -format" and create /tmp /var folders on
the HDFS of the slave machines that will be running only DN and TT or not? Do you still need
to create hadoop/dfs/name folder on the slaves?

In hdfs-site.xml for dfs.name.dir & dfs.data.dir properties  we specify  /hadoop/dfs/name
/hadoop/dfs/data  being  local linux NFS directories by running command "mkdir -p /hadoop/dfs/data"
but mapred.system.dir  property is to point to HDFS and not NFS  since we are running "sudo
-u hdfs hadoop fs -mkdir /tmp/mapred/system"??
If so and since it is exactly the same format  /far/boo/baz how does hadoop know which directory
is local on NFS or HDFS?

Would you please kindly reconfirm.

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
View raw message