hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lohit vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1771) streaming hang when IOException in MROutputThread. (NPE)
Date Tue, 09 Oct 2007 23:13:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

lohit vijayarenu updated HADOOP-1771:

    Attachment: h-1771-3.patch

Thanks to Koji, while debugging a similar problem, we saw similar case where MROutputThread
was done, but  the map task was still hung
Upon killing the streaming map executable, we saw there was a problem with the thread so it
had terminated. 
Looks like the streaming map executable was still trying to write and was hung. 

The problem could be that clientIn_ and clientErr_ are not closed when their thread is done,
which causes this problem.
Attached is an updated patch, which makes sure it will close the streams when their threads

Please could anyone review/comment.

> streaming hang when IOException in MROutputThread. (NPE)
> --------------------------------------------------------
>                 Key: HADOOP-1771
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1771
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.13.1
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>            Priority: Minor
>             Fix For: 0.15.0
>         Attachments: h-1771-2.patch, h-1771-3.patch, h-1771.patch
> One streaming task hang and had stderr userlog as follows.
> {code}
> Exception in thread "Thread-5" java.lang.NullPointerException
>          at java.lang.Throwable.printStackTrace(Throwable.java:460)
>          at org.apache.hadoop.streaming.PipeMapRed$MROutputThread.run(PipeMapRed.java:352)
> {code}
> In PipeMapRed.java
> {code}
> 351       } catch (IOException io) {
> 352         io.printStackTrace(log_);
> 353         outerrThreadsThrowable = io;
> {code}
> I guess log_ is  Null... Should call logStackTrace.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message