hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2785) MiniMR cluster thread crashes if no hadoop log dir set
Date Tue, 20 Nov 2012 22:00:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501526#comment-13501526
] 

Matt Foley commented on MAPREDUCE-2785:
---------------------------------------

moved target version to 1.2.0 upon publishing 1.1.1 RC.
                
> MiniMR cluster thread crashes if no hadoop log dir set
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-2785
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2785
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.203.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>         Attachments: MAPREDUCE-2785.patch
>
>
> I'm marking this as minor as it is most obvious in the MiniMRCluster, but the root cause
is in the JT. 
> If you instantiate an MiniMR Cluster without setting {{hadoop.job.history.location}}
in the configuration and the system property {{hadoop.log.dir}} unset, then the JobHistory
throws an NPE. In production, that would be picked up as a failure to start the JT. In the
MiniMRCluster, all it does is crash the JT thread -which isn't noticed by the MiniMR cluster.
You see the logged error, but the tests will just timeout waiting for things to come up
> 2011/08/08 17:46:26:427 CEST [ERROR][Thread-44] org.apache.hadoop.mapred.MiniMRCluster
- Job tracker crashed <java.lang.NullPointerException>         java.lang.NullPointerException
>  	at java.io.File.<init>(File.java:222)
>  	at org.apache.hadoop.mapred.JobHistory.initLogDir(JobHistory.java:531)
>  	at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:499)
>  	at org.apache.hadoop.mapred.JobTracker$2.run(JobTracker.java:2316)
>  	at org.apache.hadoop.mapred.JobTracker$2.run(JobTracker.java:2313)
>  	at java.security.AccessController.doPrivileged(Native Method)
>  	at javax.security.auth.Subject.doAs(Subject.java:396)
>  	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>  	at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2313)
>  	at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2171)
>  	at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:300)
>  	at org.apache.hadoop.mapred.MiniMRCluster$JobTrackerRunner$1.run(MiniMRCluster.java:114)
>  	at org.apache.hadoop.mapred.MiniMRCluster$JobTrackerRunner$1.run(MiniMRCluster.java:112)
>  	at java.security.AccessController.doPrivileged(Native Method)
>  	at javax.security.auth.Subject.doAs(Subject.java:396)
>  	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>  	at org.apache.hadoop.mapred.MiniMRCluster$JobTrackerRunner.run(MiniMRCluster.java:112)
>  	at java.lang.Thread.run(Thread.java:662)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message