hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5108) Review support for Binary Compatibility for MR applications via YARN
Date Wed, 03 Apr 2013 17:33:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13621085#comment-13621085

Arun C Murthy commented on MAPREDUCE-5108:

[~tomwhite] / [~tucu00] - I thought we fixed Counters with MAPREDUCE-3697. Can you please
share details on what issues you are seeing with Oozie? Thanks.
> Review support for Binary Compatibility for MR applications via YARN
> --------------------------------------------------------------------
>                 Key: MAPREDUCE-5108
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5108
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 2.0.3-alpha
>            Reporter: Arun C Murthy
>            Assignee: Zhijie Shen
>            Priority: Blocker
>             Fix For: 2.0.5-beta
> As we get ready to ship out a beta/stable version of hadoop-2, it makes sense to spend
time reviewing support for existing MR applications (hadoop-1) to migrate seamlessly.
> We've done various pieces of work over time, let's track progress and document things
clearly. [~zjshen] has done a bunch of testing and results look very promising so far.
> The aim is to support applications using org.apache.hadoop.mapred.* api in a binary compatible
manner in hadoop-2 - thus, users can just take existing MR applications jars, point them at
YARN clusters and things just work.
> Clearly, we might have some corner cases (haven't seen many so far), including semantics
(not just apis); however the intent is to, at least, document them throughly if not actually
fix them as feasible.
> Also, it's clear that we will *not* be able to support org.apache.hadoop.mapreduce api
in a *binary compatible* manner due to the interface changes we made in hadoop-0.21 (sigh),
and hence, users using the _new_ apis will have to re-compile (i.e. source compatible only).

> Net, given that vast majority of users use the org.apache.hadoop.mapred api, it's a very
reasonable way to ease migration to hadoop-2.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message