hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-2390) JobTracker and TaskTrackers fail with a misleading error if one of the mapreduce.cluster.dir has unusable permissions / is unavailable.
Date Sat, 11 Jun 2011 18:39:58 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J resolved MAPREDUCE-2390.
--------------------------------

       Resolution: Duplicate
    Fix Version/s: 0.22.0
     Release Note:   (was: Ignore bad locations in mapreduce.cluster.dir / mapred.local.dir
directories in a proper fashion.)

Resolving as Duplicate of MAPREDUCE-1382 (Which is fixed in 0.22)

> JobTracker and TaskTrackers fail with a misleading error if one of the mapreduce.cluster.dir
has unusable permissions / is unavailable.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2390
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2390
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, tasktracker
>    Affects Versions: 0.20.2
>         Environment: CDH3 and Apache 0.20 || Linux
>            Reporter: Harsh J
>            Assignee: Harsh J
>              Labels: configuration, directory-permissions, mapreduce
>             Fix For: 0.22.0
>
>
> To reproduce, have a mapred.local.dir property set to a few directories. Before starting
up the JT, set one of these directories' permission as 'd---------', and then start the JT/TT.
The JT, although it tries to ignore this directory, fails with an odd and misleading message
claiming that its configured address in use.
> Fixing the permission clears this issue!
> This was also reported in the mailing lists by Ted Yu, quite a few months ago. But I
had forgotten about filing a bug for it here. Still seems to happen. A log is attached below.
> {code}
> 2011-03-17 00:40:32,321 WARN org.apache.hadoop.mapred.JobTracker: Error starting tracker:
java.io.IOException: Cannot create toBeDeleted in /home/hack/.tmplocalz/2
>         at org.apache.hadoop.util.MRAsyncDiskService.<init>(MRAsyncDiskService.java:86)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2189)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2022)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:276)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:268)
>         at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4712)
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Updating the current master key for generating delegation tokens
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Updating the current master key for generating delegation tokens
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured
with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks)
(-1, -1, -1, -1)
> 2011-03-17 00:40:33,322 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts
(include/exclude) list
> 2011-03-17 00:40:33,350 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker
with owner as hack
> 2011-03-17 00:40:33,351 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException:
Problem binding to localhost/127.0.0.1:8021 : Address already in use
>         at org.apache.hadoop.ipc.Server.bind(Server.java:227)
>         at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:314)
>         at org.apache.hadoop.ipc.Server.<init>(Server.java:1411)
>         at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:510)
>         at org.apache.hadoop.ipc.RPC.getServer(RPC.java:471)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2112)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2022)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:276)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:268)
>         at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4712)
> Caused by: java.net.BindException: Address already in use
>         at sun.nio.ch.Net.bind(Native Method)
>         at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126)
>         at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
>         at org.apache.hadoop.ipc.Server.bind(Server.java:225)
>         ... 9 more
> 2011-03-17 00:40:33,352 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG: 
> /************************************************************
> SHUTDOWN_MSG: Shutting down JobTracker at QDuo/127.0.0.1
> ************************************************************/
> {code}
> The list conversation in context, at {{search-hadoop.com}}:
> http://search-hadoop.com/m/FzN7iqreL/problem+starting+cdh3b2+jobtracker&subj=problem+starting+cdh3b2+jobtracker
> I'll try to investigate and post the exact problem / solution soon.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message