hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "khalil honsali" <mailists.hons...@gmail.com>
Subject multi-node setup, ipc client problem with addressing, non-mailisted
Date Fri, 23 Nov 2007 12:57:51 GMT
Greetings;

I followed the excellent tutorials on the wiki, everything worked fine for
the single node version,
but for the multi-node setup (four nodes, including master), I had to use ip
addresses instead of fully qualified domain names in the
hadoop-site.xml(see appendix)
otherwise I get the error "Retrying connect to server" from both datanode
and task tracker's ipc.Client.

Connectivity is ok, ssh, host, traceroute, iptables all ok
I upgraded to hadoop-1.15 and removed ipv6 to kill the doubt , still the
problem is there...

What I am suspicious about, is the address retrieved by the ipcClient, which
is printed in the datanode info log as follow:

2007-11-22 14:52:09,734 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: speech02.matlab.nitech.ac.jp/133.68.15.85:54310. Already tried 10
time(s).
2007-11-22 14:52:10,742 INFO org.apache.hadoop.ipc.RPC: Server at
speech02.matlab.nitech.ac.jp/133.68.15.85:54310 not available yet, Zzzzz...

as you can see , it is a combination of  fully qualified domain name + / +
ip address, i wonder if it is used as is....
even if I use strict ip addressing in the setup , hadoop still manages to
use the fully qualified name in the log (how?)  like here:

133.68.15.85: starting tasktracker, logging to
/PFS/hadoop-0.15.0/bin/../logs/hadoop-hadoopsan-tasktracker-
speech02.matlab.nitech.ac.jp.out




Many thanks in advance for your time

K. Honsali



// PS \\  On a 'healthy' setup,  I also got 'false' results from the {
test.jar mapredtest 5 50 }
Mismatch!  Pos=1, i=0, val=1, dist[i]=9
Mismatch!  Pos=2, i=1, val=2, dist[i]=9
Mismatch!  Pos=3, i=2, val=3, dist[i]=11
Mismatch!  Pos=4, i=3, val=4, dist[i]=7
Cannot read entry 4
Original sum: 50
Recomputed sum: 10
Success=false

Appendix:
---------------
healthy hadoop-site.xml excerpt (using ip)
<property>
  <name> fs.default.name</name>
  <value>hdfs://133.68.15.85:54310</value>
</property>
//*all the rest is default , apart from map tasks = 4*10 and reduce tasks =
4*2
<property>
  <name> mapred.job.tracker</name>
  <value>133.68.15.85:54311</value>
</property>

unhealthy hadoop-site.xml:
<property>
  <name>fs.default.name</name>
  <value>hdfs://speech02.matlab.nitech.ac.jp:54310</value>
</property>
<property>
  <name>mapred.job.tracker</name>
  <value>speech02.matlab.nitech.ac.jp:54311</value>
</property>
--


PS2: 3 posts were sent but not delivered, the title was changed.
previous title: Problem: [multi-node setup] addresss + DNS +
ipc.client {query mailist = {!0}

Mime
View raw message