Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 53955 invoked from network); 1 Mar 2007 19:41:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 1 Mar 2007 19:41:17 -0000 Received: (qmail 44622 invoked by uid 500); 1 Mar 2007 19:41:21 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 44581 invoked by uid 500); 1 Mar 2007 19:41:21 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 44564 invoked by uid 99); 1 Mar 2007 19:41:21 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 11:41:21 -0800 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO brutus.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 11:41:11 -0800 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 6E1587141F8 for ; Thu, 1 Mar 2007 11:40:51 -0800 (PST) Message-ID: <5275981.1172778051448.JavaMail.jira@brutus> Date: Thu, 1 Mar 2007 11:40:51 -0800 (PST) From: "David Bowen (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-492) Global counters In-Reply-To: <5641100.1156887324032.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12477063 ] David Bowen commented on HADOOP-492: ------------------------------------ True. Since this issue is closed, I could add this feature to the HADOOP-1041 patch. OK? > Global counters > --------------- > > Key: HADOOP-492 > URL: https://issues.apache.org/jira/browse/HADOOP-492 > Project: Hadoop > Issue Type: New Feature > Components: mapred > Reporter: arkady borkovsky > Assigned To: David Bowen > Fix For: 0.12.0 > > Attachments: counters1.patch, counters2.patch, counters3.patch > > > It would be nice to have map / reduce job keep aggregated counts for arbitrary events occuring in its tasks -- the numer of records processed, the numer of exceptions of a specific type, the number of sentences in passive voice, whatever the jobs finds useful. > This can be implemented by tasks periodically sending pairs to the jobtracker (in some implementations such messages are piggy-backed on the heartbeats), so that the job tracker stores all the latests values from each task and aggregates them on a request. It should also make the aggregated values available at the job end. The value for a task would be flushed when the task fails. > #491 and #490 may be related to this one. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.