Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-dev@hadoop.apache.org
Message-ID: <1710664285.1243552425570.JavaMail.jira@brutus>
Date: Thu, 28 May 2009 16:13:45 -0700 (PDT)
From: "Cheng (JIRA)" <jira@apache.org>
To: chukwa-dev@hadoop.apache.org
Subject: [jira] Issue Comment Edited: (CHUKWA-267) Collect shuffling
 information from userlogs/syslog
In-Reply-To: <797354348.1243471785937.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CHUKWA-267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714213#action_12714213 ] 

Cheng edited comment on CHUKWA-267 at 5/28/09 4:13 PM:
-------------------------------------------------------

Not sure if I understand the code correctly, below are some questions I have.

1. chukwaClientIsNull and chukwaLock. I guess the correct order might be
<pre>
        synchronized (chukwaLock) {
            chukwaClientIsNull = false;
            if (chukwaClientIsNull) {
                 ..............
            }
        }
</pre>
  instead of,

      if (chukwaClientIsNull) {
         synchronized (chukwaLock) {
             chukwaClientIsNull = false;
             ...........
          }
       }


2. Currently in the patch, the chukwaClient is created in method TaskLogAppender.subAppend. If the agent is not running, will the block inside "synchronized (chukwaLock)" called many times? If yes, our appender may impact the task track and then we may need a background thread to create chukwaClient. If no, can we move the code into constructor?

Thanks,
Cheng

-Cheng


      was (Author: zhangyongjiang):
    Not sure if I understand the code correctly, below are some questions I have.

1. chukwaClientIsNull and chukwaLock. I guess the correct order might be

        synchronized (chukwaLock) {
            chukwaClientIsNull = false;
            if (chukwaClientIsNull) {
                 ..............
            }
        }

  instead of,

      if (chukwaClientIsNull) {
         synchronized (chukwaLock) {
             chukwaClientIsNull = false;
             ...........
          }
       }


2. Currently in the patch, the chukwaClient is created in method TaskLogAppender.subAppend. If the agent is not running, will the block inside "synchronized (chukwaLock)" called many times? If yes, our appender may impact the task track and then we may need a background thread to create chukwaClient. If no, can we move the code into constructor?

Thanks,
Cheng

-Cheng

  
> Collect shuffling information from userlogs/syslog
> --------------------------------------------------
>
>                 Key: CHUKWA-267
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-267
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: data collection
>    Affects Versions: 0.2.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: CHUKWA-267.patch
>
>
> Hadoop uses a customized version of log4j appender (org.apache.hadoop.mapred.TaskLogAppender) for tasks.  Shuffling information is written to userlogs/syslog by TaskLogAppender.  For chukwa to collect shuffling information, we need to write a new TaskLogAppender to do this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.