hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean (Contractor)" <dean.hil...@broadridge.com>
Subject RE: what is the map reduce fetcher??(and why is it binding to localhost when TT binds to my host???)
Date Sun, 02 Jan 2011 21:12:26 GMT
Well, I am running colinux and I have to make sure all ports I need routed to my datanode/tasknode
are in the config file so the ephemeral doesn't work and I locked it down to fix it....I have
been trying to hook up eclipse to debug but just found out that child processes are spun up.
 I think that part is working now, but zookeeper isn't though I should be able to figure out
from debugging it I think if I manage to hook up the eclipse debugger to my task node with
that cool IsolationRunner class.

Dean

-----Original Message-----
From: Harsh J [mailto:qwertymaniac@gmail.com] 
Sent: Sunday, January 02, 2011 1:27 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: what is the map reduce fetcher??(and why is it binding to localhost when TT binds
to my host???)

Hello Dean,

On Mon, Jan 3, 2011 at 1:07 AM, Hiller, Dean  (Contractor)
<dean.hiller@broadridge.com> wrote:
> 2011-01-02 12:32:26,230 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events
fetcher for all reduce tasks on tracker_DENVER-DHILLER.jsq.bsg.ad.adp.com:127.0.0.1/127.0.0.1:34835

This is normal if am right. A TaskTracker's service by default binds
to the loopback as it only needs to communicate with its child Task
JVMs and has no communication from outside.

> EVEN WORSE...This port 34835 keeps changing on startup so when using colinux,
> I get screwed in that I can't telnet into the port(ie. No programs can
> connect to that port) so this bound port becomes useless as no one will be
> able to talk to it.  Is there any way

The TaskTracker binding port by default is an ephemeral one.

> 1.       To fix it to an address?

For Hadoop 0.20.2, the "Expert mode (aka, think twice before
changing)" configuration you're looking for is:
mapred.task.tracker.report.address

That property by default is set to the string "127.0.0.1:0",
explaining the above behavior. I haven't seen it cause trouble really
-- what issue are you facing, and are you absolutely sure it is cause
of this?

-- 
Harsh J
www.harshj.com
This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.


Mime
View raw message