hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-403) close method in a Mapper should be provided with OutputCollector and a Reporter
Date Mon, 14 Aug 2006 20:06:15 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-403?page=comments#action_12427956 ] 
            
Doug Cutting commented on HADOOP-403:
-------------------------------------

I don't think we should make all future io.Closeable implementations deal with a mapred.OutputCollector.
 I would prefer any such methods were added to a new interface, perhaps mapred.Output.

One can easily workaround this problem by specifying an alternate MapRunnable.  Is this issue
really so common that this workaround, or Runping's, do not suffice?

> close method in a Mapper should be provided with OutputCollector and a Reporter
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-403
>                 URL: http://issues.apache.org/jira/browse/HADOOP-403
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.5.0
>         Environment: all
>            Reporter: Milind Bhandarkar
>         Assigned To: Milind Bhandarkar
>             Fix For: 0.6.0
>
>
> For mappers with side-effects, or mappers that work as aggregators (i.e. no output on
individual key-value pairs, but an aggregate output at the end of all key-value pairs), output
should be performed in the close method. For this purpose, we need to supply output collector
and reporter to the close method of Mapper. This involves interface change, though. Thoughts
?

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message