chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (CHUKWA-267) Collect shuffling information from userlogs/syslog
Date Thu, 28 May 2009 23:13:45 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714213#action_12714213
] 

Cheng edited comment on CHUKWA-267 at 5/28/09 4:13 PM:
-------------------------------------------------------

Not sure if I understand the code correctly, below are some questions I have.

1. chukwaClientIsNull and chukwaLock. I guess the correct order might be
<pre>
        synchronized (chukwaLock) {
            chukwaClientIsNull = false;
            if (chukwaClientIsNull) {
                 ..............
            }
        }
</pre>
  instead of,

      if (chukwaClientIsNull) {
         synchronized (chukwaLock) {
             chukwaClientIsNull = false;
             ...........
          }
       }


2. Currently in the patch, the chukwaClient is created in method TaskLogAppender.subAppend.
If the agent is not running, will the block inside "synchronized (chukwaLock)" called many
times? If yes, our appender may impact the task track and then we may need a background thread
to create chukwaClient. If no, can we move the code into constructor?

Thanks,
Cheng

-Cheng


      was (Author: zhangyongjiang):
    Not sure if I understand the code correctly, below are some questions I have.

1. chukwaClientIsNull and chukwaLock. I guess the correct order might be

        synchronized (chukwaLock) {
            chukwaClientIsNull = false;
            if (chukwaClientIsNull) {
                 ..............
            }
        }

  instead of,

      if (chukwaClientIsNull) {
         synchronized (chukwaLock) {
             chukwaClientIsNull = false;
             ...........
          }
       }


2. Currently in the patch, the chukwaClient is created in method TaskLogAppender.subAppend.
If the agent is not running, will the block inside "synchronized (chukwaLock)" called many
times? If yes, our appender may impact the task track and then we may need a background thread
to create chukwaClient. If no, can we move the code into constructor?

Thanks,
Cheng

-Cheng

  
> Collect shuffling information from userlogs/syslog
> --------------------------------------------------
>
>                 Key: CHUKWA-267
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-267
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: data collection
>    Affects Versions: 0.2.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: CHUKWA-267.patch
>
>
> Hadoop uses a customized version of log4j appender (org.apache.hadoop.mapred.TaskLogAppender)
for tasks.  Shuffling information is written to userlogs/syslog by TaskLogAppender.  For chukwa
to collect shuffling information, we need to write a new TaskLogAppender to do this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message