hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lujie (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-7178) NPE while YarnChild shudown
Date Sat, 26 Jan 2019 04:59:00 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-7178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

lujie updated MAPREDUCE-7178:
-----------------------------
    Attachment: MR-9226_1.patch
        Status: Patch Available  (was: Open)

> NPE while YarnChild shudown
> ---------------------------
>
>                 Key: MAPREDUCE-7178
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7178
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: lujie
>            Priority: Major
>         Attachments: MR-9226_1.patch
>
>
> In YarnChild.main
> {code:java}
> try{
>  logSyncer = TaskLog.createLogSyncer();//line 168
>  ....
>  taskFinal.run(job, umbilical); //line 178
> }catch (Exception exception) {//line 187
>   LOG.warn("Exception running child : "
>    + StringUtils.stringifyException(exception));
>    .................
>    task.taskCleanup(umbilical);// line 200
> }{code}
> At line 178. it will initialize the task.committer, but the line168 may throw exception,
it will skip  initialize the task.committer, hence task.committer == null. Line 187 will
catch this exception and do clean up(line 200), code line 200 will use  task.committer without
null check, hence NPE happens
> {code:java}
> 2019-01-23 16:59:42,864 INFO [main] org.apache.hadoop.mapred.YarnChild: Exception cleaning
up: java.lang.NullPointerException
> at org.apache.hadoop.mapred.Task.taskCleanup(Task.java:1458)
> at org.apache.hadoop.mapred.YarnChild$3.run(YarnChild.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:197)
> {code}
> So why  line168 may throw exception, below log give a example:
> {code:java}
> 2019-01-23 16:59:42,857 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running
child : java.lang.IllegalStateException: Shutdown in progress, cannot add a shutdownHook
> at org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:299)
> at org.apache.hadoop.mapred.TaskLog.createLogSyncer(TaskLog.java:340)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message