hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jiaqi Tan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6013) Add reduce ID to shuffle clienttrace
Date Fri, 12 Jun 2009 21:09:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718980#action_12718980
] 

Jiaqi Tan commented on HADOOP-6013:
-----------------------------------

Destination reduceID of shuffle needed in Chukwa for building causal tracing between maps
and reduces; this saves the effort of having to parse task logs for shuffle information, and
greatly reduces the number of adaptors needed (eliminates need for per-task adaptors).

> Add reduce ID to shuffle clienttrace
> ------------------------------------
>
>                 Key: HADOOP-6013
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6013
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Jiaqi Tan
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-6013.patch
>
>
> Current clienttrace messages from shuffles note only the destination map ID but not the
source reduce ID. Having both source and destination ID of each shuffle enables full tracing
of execution. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message