hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5746) Errors encountered in MROutputThread after the last map/reduce call can go undetected
Date Wed, 10 Jun 2009 06:15:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amar Kamat updated HADOOP-5746:
-------------------------------

    Release Note: If the child (streaming) process returns successfully and the MROutputThread
throws an error, there was no way to detect that as all the IOExceptions was ignored. Such
issues can occur when DFS clients were closed etc. Now a check for errors (in threads) is
made before finishing off the task and an exception is thrown that fails he task.

> Errors encountered in MROutputThread after the last map/reduce call can go undetected
> -------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5746
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5746
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/streaming
>            Reporter: Devaraj Das
>            Assignee: Amar Kamat
>             Fix For: 0.20.1
>
>         Attachments: 5746-reproduce.1.patch, 5746-testcase.patch, 5746.1.patch, 5746.6.patch
>
>
> The framework map/reduce bridge methods make a check at the beginning of the respective
methods whether _MROutputThread_ encountered an exception while writing keys/values that the
streaming process emitted. However, if the exception happens in _MROutputThread_ after the
last call to the map/reduce method, the exception goes undetected. An example of such an exception
is an exception from the _DFSClient_ that fails to write to a file on the HDFS.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message