Return-Path: 
 <mapreduce-issues-return-40235-apmail-hadoop-mapreduce-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 1B13D9AAA
	for <apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org>;
 Mon,  6 Feb 2012 19:52:22 +0000 (UTC)
Received: (qmail 38028 invoked by uid 500); 6 Feb 2012 19:52:21 -0000
Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org
Received: (qmail 37957 invoked by uid 500); 6 Feb 2012 19:52:21 -0000
Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-issues@hadoop.apache.org>
List-Id: <mapreduce-issues.hadoop.apache.org>
Reply-To: mapreduce-issues@hadoop.apache.org
Delivered-To: mailing list mapreduce-issues@hadoop.apache.org
Received: (qmail 37830 invoked by uid 99); 6 Feb 2012 19:52:21 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Feb 2012 19:52:21 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=5.0
	tests=ALL_TRUSTED,T_RP_MATCHES_RCVD
X-Spam-Check-By: apache.org
Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Feb 2012 19:52:19 +0000
Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116])
	by hel.zones.apache.org (Postfix) with ESMTP id E6966CC072
	for <mapreduce-issues@hadoop.apache.org>;
 Mon,  6 Feb 2012 19:51:59 +0000 (UTC)
Date: Mon, 6 Feb 2012 19:51:59 +0000 (UTC)
From: "Tom White (Updated) (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <453590475.3199.1328557919945.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <125884873.56285.1326985480334.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (MAPREDUCE-3697) Hadoop Counters API limits
 Oozie's working across different hadoop versions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


     [ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated MAPREDUCE-3697:
---------------------------------

    Attachment: apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh

I reviewed the patch for compatibility with Hadoop 1.x. I ran a SigTest compatibility test (like in HADOOP-7738) to see what the differences between the mapred counters are between 1.0.0 and trunk with this patch (see attached). Most are false positives due to the inheritance hierarchy changing. Here are the ones that I think need fixing:

* Counters.getGroupNames returns an Iterable<String> not a Collection<String> as it does in 1.x. I think we should change it back to return Collection<String> - we can always add a getGroupNamesIterable() later if needed.
* method public java.lang.String org.apache.hadoop.mapred.Counters.makeCompactString() is missing
* public static int org.apache.hadoop.mapred.Counters.MAX_COUNTER_LIMIT is missing (it's not used in 23, but we should still define it for compatibility)
                
> Hadoop Counters API limits Oozie's working across different hadoop versions
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3697
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1
>            Reporter: John George
>            Assignee: Mahadev konar
>            Priority: Blocker
>         Attachments: MAPREDUCE-3697-facade.patch, MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh
>
>
> Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
> hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
> parent class AbstractCounters implements it. This is different from hadoop20X.
> As a result, Oozie compiled with either hadoop version does not work with the
> other version.
> A specific scenario, Oozie compiled with .23 and run against 205, does not
> update job status owing to a Counters API exception.
> Will explicit re-compilation against the relevant hadoop jars be required each
> time? This will prevent launching a uniform Oozie version across different
> clusters.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira