hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-728) Map-reduce task does not produce correct results when -reducer NONE is specified through streaming
Date Wed, 29 Nov 2006 17:45:23 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-728?page=comments#action_12454386 ] 
            
Owen O'Malley commented on HADOOP-728:
--------------------------------------

Currently, the PhasedFileSystem assumes it is running in a map/reduce context. It gets the
task id out of the JobConf to make the unique name and counts on the TaskTracker cleaning
up the storage if the process dies. So currently the mapred package is right in my opinion.
Arguably, the map/reduce dependencies should be removed to make it a more general class, but
that would be a different bug.

I haven't run this bug, but it looks like the right direction for streaming. Streaming needs
to become much more unified with the rest of Hadoop.

> Map-reduce task does not produce correct results when -reducer NONE is specified through
streaming
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-728
>                 URL: http://issues.apache.org/jira/browse/HADOOP-728
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>            Reporter: dhruba borthakur
>         Assigned To: Sanjay Dahiya
>         Attachments: Hadoop-728.patch
>
>
> a) a file is create for the output instead of a directory.
> b) there is no way to understand what is going on from the client output
> I can produce an example for you, if you like -- but the behavior is consistent, so $HSTREAM
-mapper /bin/cat -reducer NONE should show the problem
> ~

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message