hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Dahiya (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-728) Map-reduce task does not produce correct results when -reducer NONE is specified through streaming
Date Tue, 28 Nov 2006 12:33:25 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-728?page=comments#action_12453904 ] 
            
Sanjay Dahiya commented on HADOOP-728:
--------------------------------------

The reason of keeping PhasedFileSystem in mapred package was that it works only for tasks
in a mapred job as of now. it used jobid/tipid/taskid in "mapred.system.dir" for temporary
storage to avoid any possible conflicts among task outputs. Is it ok to have this in fs package
with this dependency ? 
Also currently it extends FileSystem abstract base class for API compatibility, it takes another
pre initialized file system as constructor argument and channels calls to that, Just for this
there doesnt seem to be a need for another FIlterFileSystem. 



> Map-reduce task does not produce correct results when -reducer NONE is specified through
streaming
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-728
>                 URL: http://issues.apache.org/jira/browse/HADOOP-728
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>            Reporter: dhruba borthakur
>         Assigned To: Sanjay Dahiya
>         Attachments: Hadoop-728.patch
>
>
> a) a file is create for the output instead of a directory.
> b) there is no way to understand what is going on from the client output
> I can produce an example for you, if you like -- but the behavior is consistent, so $HSTREAM
-mapper /bin/cat -reducer NONE should show the problem
> ~

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message