hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3697) Hadoop Counters API limits Oozie's working across different hadoop versions
Date Fri, 03 Feb 2012 18:47:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199928#comment-13199928
] 

Mahadev konar commented on MAPREDUCE-3697:
------------------------------------------

@Alejandro,
 I have a similar patch which tries to do something like this but I am running into some issues
with it when I do the same thing with Group. Looks like some issue where the write fields
doesnt match the readfields for TaskStatus (where Counters are used) and the tasks hang. Am
still debugging that. After doing this for a couple of days, I am actually not averse to copying
over the code to mapred.Counters :).
                
> Hadoop Counters API limits Oozie's working across different hadoop versions
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3697
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1
>            Reporter: John George
>            Assignee: Mahadev konar
>            Priority: Blocker
>         Attachments: MAPREDUCE-3697-facade.patch, MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch,
MAPREDUCE-3697.patch, MAPREDUCE-3697.patch
>
>
> Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
> hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
> parent class AbstractCounters implements it. This is different from hadoop20X.
> As a result, Oozie compiled with either hadoop version does not work with the
> other version.
> A specific scenario, Oozie compiled with .23 and run against 205, does not
> update job status owing to a Counters API exception.
> Will explicit re-compilation against the relevant hadoop jars be required each
> time? This will prevent launching a uniform Oozie version across different
> clusters.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message