Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <832254696.1220373344833.JavaMail.jira@brutus>
Date: Tue, 2 Sep 2008 09:35:44 -0700 (PDT)
From: "Owen O'Malley (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Updated: (HADOOP-3748) Flag to make tasks to send counter
 information only at the end of the task
In-Reply-To: <141217348.1215745411637.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley updated HADOOP-3748:
----------------------------------

    Resolution: Won't Fix
        Status: Resolved  (was: Patch Available)

I think this is a really bad plan. The application can already do this by keeping the counters local until the map or reduce close method is called. We already have broken code in the task that tries to only send counters occasionally that should be removed.

> Flag to make tasks to send counter information only at the end of the task
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-3748
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3748
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>         Environment: all
>            Reporter: Alejandro Abdelnur
>         Attachments: [HADOOP-3748].patch
>
>
> Currently counters are streaming from the task to the jobtracker as the task progresses. If the number of counters is large this has a significant impact on the network traffic as well as in the JobTracker load.
> The should be a flag, for example by counter-group, that indicates that the counters are to be reported at the end of the task. By default this flag should be set to false for all counter-groups maintaining the current behavior.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.