Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <86803648.1219943324730.JavaMail.jira@brutus>
Date: Thu, 28 Aug 2008 10:08:44 -0700 (PDT)
From: "Runping Qi (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3748) Flag to make tasks to send counter
 information only at the end of the task
In-Reply-To: <141217348.1215745411637.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626650#action_12626650 ] 

Runping Qi commented on HADOOP-3748:
------------------------------------


I think we need to be more pragmatic about performance related contributions.
I think any improvement is worth to consider. It is reasonable that we  weight performance improvement brought in by a patch against the additional code complexity it introduces.
However, it is too strong to state that such a patch will not be committed unless it brought significant performance gain.
BTW, how do you characterize significant performance gain?
 

> Flag to make tasks to send counter information only at the end of the task
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-3748
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3748
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>         Environment: all
>            Reporter: Alejandro Abdelnur
>
> Currently counters are streaming from the task to the jobtracker as the task progresses. If the number of counters is large this has a significant impact on the network traffic as well as in the JobTracker load.
> The should be a flag, for example by counter-group, that indicates that the counters are to be reported at the end of the task. By default this flag should be set to false for all counter-groups maintaining the current behavior.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.