hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frank Luo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3520) Allow for metrics to be exchanged between mapps and reduces
Date Sat, 17 Aug 2013 16:00:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742962#comment-13742962
] 

Frank Luo commented on MAPREDUCE-3520:
--------------------------------------

any progress on this ticket, or there is a better way to do it now?
                
> Allow for metrics to be exchanged between mapps and reduces
> -----------------------------------------------------------
>
>                 Key: MAPREDUCE-3520
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3520
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2
>    Affects Versions: 0.24.0
>            Reporter: Robert Joseph Evans
>
> Some people on the mailing lists have requested having read access in reducers to counters
that were set by mappers.  I propose that we provide a new interface for doing this.  The
counter data in addition to being set periodically to the Application Master would be put
into the intermediate output of the mapper for each reducer and be specially marked as such.
 When the reducers fetch that data they can combine it together and provide a read only interface
to it.
> This allows users to potentially optimize their code on the reducer side if they know
some special metadata before hand.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message