hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5876) Shuffling information logged to userlogs/attempt_####_###_r_###_#/syslogs
Date Wed, 20 May 2009 18:28:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711277#action_12711277
] 

Arun C Murthy commented on HADOOP-5876:
---------------------------------------

What kind of shuffle information are you looking for? 

Overall it doesn't make sense to start pushing task-logs to the tasktracker logs - it's just
easier to collect userlogs. Even more, it makes absolutely no sense to push it to job-history,
seems to me that we should be using some metrics at best.

> Shuffling information logged to userlogs/attempt_####_###_r_###_#/syslogs
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-5876
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5876
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.20.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Priority: Critical
>
> Shuffling information is currently logged to userlogs, it would be ideal to have this
information consolidated in task tracker log or job history log file for down stream log collection
and analysis program (Chukwa) to pickup.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message