hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joey Krabacher <jkrabac...@gmail.com>
Subject Re: Map Task Capacity Not Changing
Date Thu, 29 Dec 2011 16:32:57 GMT
To follow up on what I have found:

I opened up some of the logs on the datanodes and found this message:
"Can not start task tracker because java.net.BindException: Address
already in use"

It was using the default port setting from mapred-default.xml, which was 50060.
I decided to try an add

  <property>
    <name>mapred.task.tracker.http.address</name>
    <value>0.0.0.0:0</value>
  </property>

to mapred-site.xml so that the first open port would be selected.
This works and it also, allows for the tasktracker to start normally
which in turn allows the mapred.tasktracker.map.tasks.maximum setting
to take effect.

Thanks for all who helped.

--Joey

On Sat, Dec 17, 2011 at 1:42 AM, Joey Krabacher <jkrabacher@gmail.com> wrote:
> pid files are there, I checked for running processes with the sameID's
> and they all checked out.
> --Joey
> On Fri, Dec 16, 2011 at 5:40 PM, Rahul Jain <rjain7@gmail.com> wrote:
>> You might be suffering from HADOOP-7822; I'd suggest you verify your pid
>> files and fix the problem by hand if it is the same issue.
>>
>> -Rahul
>>
>> On Fri, Dec 16, 2011 at 2:40 PM, Joey Krabacher <jkrabacher@gmail.com>wrote:
>>
>>> Turns out my tasktrackers(on the datanodes) are not starting properly....
>>>
>>> so I guess they are taking some alternate route??
>>>
>>> because they are up and running...even though when I run
>>> stop-mapred.sh it says "data01: no tasktracker to stop"
>>>
>>> --Joey
>>>
>>> On Thu, Dec 15, 2011 at 5:37 PM, James Warren
>>> <james.warren@stanfordalumni.org> wrote:
>>> > (moving to mapreduce-user@, bcc'ing common-user@)
>>> >
>>> > Hi Joey -
>>> >
>>> > You'll want to change the value on all of your servers running
>>> tasktrackers
>>> > and then restart each tasktracker to reread the configuration.
>>> >
>>> > cheers,
>>> > -James
>>> >
>>> > On Thu, Dec 15, 2011 at 3:30 PM, Joey Krabacher <jkrabacher@gmail.com
>>> >wrote:
>>> >
>>> >> I have looked up how to up this value on the web and have tried all
>>> >> suggestions to no avail.
>>> >>
>>> >> Any help would be great.
>>> >>
>>> >> Here is some background:
>>> >>
>>> >> Version: 0.20.2, r911707
>>> >> Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo
>>> >>
>>> >> Nodes: 5
>>> >> Current Map Task Capacity : 10  <--- this is what I want to increase.
>>> >>
>>> >> What I have tried :
>>> >>
>>> >> Adding
>>> >>   <property>
>>> >>    <name>mapred.tasktracker.map.tasks.maximum</name>
>>> >>    <value>8</value>
>>> >>    <final>true</final>
>>> >>  </property>
>>> >> to mapred-site.xml on NameNode.  I also added this to one of the
>>> >> datanodes for the hell of it and that didn't work either.
>>> >>
>>> >> Thanks.
>>> >>
>>>

Mime
View raw message