hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-444) In streaming with a NONE reducer, you get duplicate files if a mapper fails, is restarted, and succeeds next time.
Date Wed, 20 Sep 2006 22:33:25 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-444?page=all ]

Doug Cutting updated HADOOP-444:
--------------------------------

           Status: Resolved  (was: Patch Available)
    Fix Version/s: 0.7.0
       Resolution: Fixed

This was fixed as a part of HADOOP-542.

> In streaming with a NONE reducer, you get duplicate files if a mapper fails, is restarted,
and succeeds next time.
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-444
>                 URL: http://issues.apache.org/jira/browse/HADOOP-444
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.5.0
>            Reporter: Dick King
>         Assigned To: Michel Tourn
>             Fix For: 0.7.0
>
>
> When the dust settled after a streaming run, the directory ended up looking like this:
>   /user/dking/<project-name>/K-HTML-UTF8-2006-08-09-rescued-abstracted/task_0026_m_007384_0
<r 3>	10563406
>   /user/dking/<project-name>/K-HTML-UTF8-2006-08-09-rescued-abstracted/task_0026_m_007384_1
<r 3>	10563406
> Future processing will receive duplicated data.
> -dk

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message