Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of eslam.elnikety@gmail.com
 designates 209.85.216.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=XGCS+lG3aZjWWrvov/IRxRIe0/9ZJAw7a3ZCIqj6qlHVSkEGHyJXRiqytbntq6lhKA
         k0L2x8ynE8NTksFBzJuDY8Vm75RsWIyG8rC3i+OE8UIxTnWACA5CYvnczH3pdZeHYWLX
         pUSVMKtkrH3jxITKAHNSPeWVAsSD/XWi2/+Ek=
MIME-Version: 1.0
Date: Sat, 5 Feb 2011 13:19:23 +0300
Message-ID: <AANLkTimF4uj88pGB2P=+9XRgVV+XMrXUaENF=-C=QR3O@mail.gmail.com>
Subject: Reducer-Mapper Communication
From: Eslam Elnikety <eslam.elnikety@gmail.com>
To: mapreduce-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0023544714b42910cc049b86551a

--0023544714b42910cc049b86551a
Content-Type: text/plain; charset=ISO-8859-1

Dear all,

I need to pass data from a reducer task to a mapper task. Currently, I am
testing Hadoop in a pseudo-distributed mode.

The reducer (org.apache.hadoop.mapred.ReduceTask) executes the following
code:

 InetAddress address = InetAddress.*getByName*("localhost");

 serverSocket = *new* ServerSocket(port, 0, address);

 socket = serverSocket.accept();

 outStream = *new* ObjectOutputStream(socket.getOutputStream());

where the mapper (org.apache.hadoop.mapred.MapTask) executes:

    InetAddress address = InetAddress.*getByName*("localhost");

 socket = *new* Socket(address, port);

 inStream = *new* ObjectInputStream(socket.getInputStream());

The variable port has the same value in both mapper and reducer, and is
dynamically assigned after scanning for a free port on localhost.

The scenario goes like this:
1) The reducer listens correctly on the port (I checked it with netstat)
2) The mapper throws a java.net.ConnectException: Connection refused

I can connect to that open port using a test program from both
localhost/remote machine using the same code of the mapper while the
ReduceTask is waiting on serverSocket.accept(). It only fails when this code
is executed by the mapper. I have tried to replace localhost with loopback
(127.0.0.1), and eth0 IP address, but I just get the same behavior as
described above. Any suggestions what might be causing the problem? Thanks!

Regards,
Elnikety

--0023544714b42910cc049b86551a--