hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
Date Mon, 11 Jan 2016 22:36:40 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092830#comment-15092830
] 

Jason Lowe commented on MAPREDUCE-6554:
---------------------------------------

Thanks for updating the patch, Bibin!  The patch will fix the specific NPE reported in the
summary, but it won't address the other case you reported with the Avro exception.  How about
if we check for a null schema _and_ translate any AvroRuntimeException thrown by the parser
into an IOException?  I think that would be a bit safer than assuming every exception should
be a parse error I/O exception.

> MRAppMaster servicestart failing  with NPE in MRAppMaster#parsePreviousJobHistory
> ---------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6554
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>         Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main] org.apache.hadoop.service.AbstractService: Service
org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; cause: java.lang.NullPointerException
> java.lang.NullPointerException
>         at java.io.StringReader.<init>(StringReader.java:50)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:917)
>         at org.apache.avro.Schema.parse(Schema.java:966)
>         at org.apache.hadoop.mapreduce.jobhistory.EventReader.<init>(EventReader.java:75)
>         at org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
>         at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler:
Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
>  this.version = in.readLine();
> ...
>     Schema myschema = new SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
>     this.schema = Schema.parse(in.readLine());
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message