chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-267) Collect shuffling information from userlogs/syslog
Date Thu, 28 May 2009 23:41:45 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714229#action_12714229
] 

Cheng commented on CHUKWA-267:
------------------------------

Suppose two threads reach the point of "synchronized (chukwaLock)" at the same time. After
one thread quits from the sync block, another thread will take turn. It means the code inside
the sync block will be executed twice. It means the file will be added more than one time;
ClientFinalizer will be created more than one time, etc.

> Collect shuffling information from userlogs/syslog
> --------------------------------------------------
>
>                 Key: CHUKWA-267
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-267
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: data collection
>    Affects Versions: 0.2.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: CHUKWA-267.patch
>
>
> Hadoop uses a customized version of log4j appender (org.apache.hadoop.mapred.TaskLogAppender)
for tasks.  Shuffling information is written to userlogs/syslog by TaskLogAppender.  For chukwa
to collect shuffling information, we need to write a new TaskLogAppender to do this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message