Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Date: Sat, 11 Jun 2011 18:39:58 +0000 (UTC)
From: "Harsh J (JIRA)" <jira@apache.org>
To: mapreduce-dev@hadoop.apache.org
Message-ID: 
 <73935481.14179.1307817598965.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <539809132.7349.1300304793717.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Resolved] (MAPREDUCE-2390) JobTracker and TaskTrackers fail
 with a misleading error if one of the mapreduce.cluster.dir has unusable
 permissions / is unavailable.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Harsh J resolved MAPREDUCE-2390.
--------------------------------

       Resolution: Duplicate
    Fix Version/s: 0.22.0
     Release Note:   (was: Ignore bad locations in mapreduce.cluster.dir / mapred.local.dir directories in a proper fashion.)

Resolving as Duplicate of MAPREDUCE-1382 (Which is fixed in 0.22)

> JobTracker and TaskTrackers fail with a misleading error if one of the mapreduce.cluster.dir has unusable permissions / is unavailable.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2390
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2390
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, tasktracker
>    Affects Versions: 0.20.2
>         Environment: CDH3 and Apache 0.20 || Linux
>            Reporter: Harsh J
>            Assignee: Harsh J
>              Labels: configuration, directory-permissions, mapreduce
>             Fix For: 0.22.0
>
>
> To reproduce, have a mapred.local.dir property set to a few directories. Before starting up the JT, set one of these directories' permission as 'd---------', and then start the JT/TT. The JT, although it tries to ignore this directory, fails with an odd and misleading message claiming that its configured address in use.
> Fixing the permission clears this issue!
> This was also reported in the mailing lists by Ted Yu, quite a few months ago. But I had forgotten about filing a bug for it here. Still seems to happen. A log is attached below.
> {code}
> 2011-03-17 00:40:32,321 WARN org.apache.hadoop.mapred.JobTracker: Error starting tracker: java.io.IOException: Cannot create toBeDeleted in /home/hack/.tmplocalz/2
>         at org.apache.hadoop.util.MRAsyncDiskService.<init>(MRAsyncDiskService.java:86)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2189)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2022)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:276)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:268)
>         at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4712)
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list
> 2011-03-17 00:40:33,350 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker with owner as hack
> 2011-03-17 00:40:33,351 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to localhost/127.0.0.1:8021 : Address already in use
>         at org.apache.hadoop.ipc.Server.bind(Server.java:227)
>         at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:314)
>         at org.apache.hadoop.ipc.Server.<init>(Server.java:1411)
>         at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:510)
>         at org.apache.hadoop.ipc.RPC.getServer(RPC.java:471)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2112)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2022)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:276)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:268)
>         at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4712)
> Caused by: java.net.BindException: Address already in use
>         at sun.nio.ch.Net.bind(Native Method)
>         at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126)
>         at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
>         at org.apache.hadoop.ipc.Server.bind(Server.java:225)
>         ... 9 more
> 2011-03-17 00:40:33,352 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG: 
> /************************************************************
> SHUTDOWN_MSG: Shutting down JobTracker at QDuo/127.0.0.1
> ************************************************************/
> {code}
> The list conversation in context, at {{search-hadoop.com}}:
> http://search-hadoop.com/m/FzN7iqreL/problem+starting+cdh3b2+jobtracker&subj=problem+starting+cdh3b2+jobtracker
> I'll try to investigate and post the exact problem / solution soon.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira