hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohamed Riadh Trad <Mohamed.t...@inria.fr>
Subject Problem Setting Cluster with no public nodes adresses
Date Fri, 22 Jul 2011 10:53:56 GMT
Dear all,

I am trying to set up hadoop on a 18 node cluster.

The problem is that cluster nodes do not have public IPs, such as, in order to access node1,
I had first to connect to the frontal node.

Let a be the ip address of my frontal node.

I have configured my .ssh/config with proxycommand:

# Cluster
Host node*.a
ProxyCommand ssh -q a /usr/bin/nc `echo %h | cut -d"." -f1` 22
#

and added 3 nodes in the masters conf/slave:
node001.a
node002.a
node003.a

then I have formatted my namenode and started dfs.

the issue, is that when I look for datanodes, there is only one instead of 3... the displayed
datanode name changes every time that I want to display the datanodes lit.

Any Idea, how to resolve this issue??

I am suspecting the framework not differentiating hosts since I used the proxycommand.

Bests,



Trad Mohamed Riadh, M.Sc, Ing.
PhD. student
INRIA-TELECOM PARISTECH - ENPC School of International Management

Office: 11-15
Phone: (33)-1 39 63 59 33
Fax: (33)-1 39 63 56 74
Email: riadh.trad@inria.fr
Home page: http://www-rocq.inria.fr/who/Mohamed.Trad/


Mime
View raw message