hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5108) Review support for Binary Compatibility for MR applications via YARN
Date Wed, 17 Apr 2013 08:23:17 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633897#comment-13633897
] 

Steve Loughran commented on MAPREDUCE-5108:
-------------------------------------------

I got a stack trace trying to submit a pig job; the submission API has changed from the pig
0.10 library

                
> Review support for Binary Compatibility for MR applications via YARN
> --------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5108
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5108
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 2.0.3-alpha
>            Reporter: Arun C Murthy
>            Assignee: Zhijie Shen
>            Priority: Blocker
>             Fix For: 2.0.5-beta
>
>         Attachments: Binary Backward Compatibility.pdf, mr1_mr2_api_diff.tar.gz
>
>
> As we get ready to ship out a beta/stable version of hadoop-2, it makes sense to spend
time reviewing support for existing MR applications (hadoop-1) to migrate seamlessly.
> We've done various pieces of work over time, let's track progress and document things
clearly. [~zjshen] has done a bunch of testing and results look very promising so far.
> The aim is to support applications using org.apache.hadoop.mapred.* api in a binary compatible
manner in hadoop-2 - thus, users can just take existing MR applications jars, point them at
YARN clusters and things just work.
> Clearly, we might have some corner cases (haven't seen many so far), including semantics
(not just apis); however the intent is to, at least, document them throughly if not actually
fix them as feasible.
> Also, it's clear that we will *not* be able to support org.apache.hadoop.mapreduce api
in a *binary compatible* manner due to the interface changes we made in hadoop-0.21 (sigh),
and hence, users using the _new_ apis will have to re-compile (i.e. source compatible only).

> Net, given that vast majority of users use the org.apache.hadoop.mapred api, it's a very
reasonable way to ease migration to hadoop-2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message