hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5114) Subsequent AM attempt can crash trying to read prior AM attempt information
Date Fri, 14 Nov 2014 16:09:33 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14212419#comment-14212419
] 

Jason Lowe commented on MAPREDUCE-5114:
---------------------------------------

Note that this can also occur in JobHistoryCopyService.  In this case it looks like the previous
jhist file was empty:

{noformat}
2014-11-14 11:51:28,627 INFO [main] org.apache.hadoop.service.AbstractService: Service JobHistoryCopyService
failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
Incompatible event log version: null
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException: Incompatible
event log version: null
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService.serviceStart(JobHistoryCopyService.java:78)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1081)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1494)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1490)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1423)
Caused by: java.io.IOException: Incompatible event log version: null
        at org.apache.hadoop.mapreduce.jobhistory.EventReader.<init>(EventReader.java:71)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:99)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService.parse(JobHistoryCopyService.java:93)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService.serviceStart(JobHistoryCopyService.java:76)
        ... 10 more
2014-11-14 11:51:28,630 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster
failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
Incompatible event log version: null
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException: Incompatible
event log version: null
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService.serviceStart(JobHistoryCopyService.java:78)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1081)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1494)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1490)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1423)
Caused by: java.io.IOException: Incompatible event log version: null
        at org.apache.hadoop.mapreduce.jobhistory.EventReader.<init>(EventReader.java:71)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:99)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService.parse(JobHistoryCopyService.java:93)
        at org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService.serviceStart(JobHistoryCopyService.java:76)
        ... 10 more
{noformat}

> Subsequent AM attempt can crash trying to read prior AM attempt information
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5114
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5114
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am
>    Affects Versions: 2.0.3-alpha, 0.23.6
>            Reporter: Jason Lowe
>
> Saw the second AM attempt of a job fail early during startup because it tried to read
the AMInfos from the previous attempt's history file and hit an error that wasn't an IOException.
 Stack trace to follow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message