hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radhe Radhe <radhe.krishna.ra...@live.com>
Subject RE: Doubt regarding Binary Compatibility\Source Compatibility with old *mapred* APIs and new *mapreduce* APIs in Hadoop
Date Tue, 15 Apr 2014 19:44:37 GMT
Thanks John for your comments,
I believe MRv2 has support for both the old *mapred* APIs and new *mapreduce* APIs.
I see this way:[1.]  One may have binaries i.e. jar file of the M\R program that used old
*mapred* APIsThis will work directly on MRv2(YARN).
[2.]  One may have the source code i.e. Java Programs of the M\R program that used old *mapred*
APIsFor this I need to recompile and generate the binaries i.e. jar file. Do I have to change
the old *org.apache.hadoop.mapred* APIs to new *org.apache.hadoop.mapreduce* APIs? or No code
changes are needed?
-RR
> Date: Mon, 14 Apr 2014 10:37:56 -0400
> Subject: Re: Doubt regarding Binary Compatibility\Source Compatibility with old *mapred*
APIs and new *mapreduce* APIs in Hadoop
> From: john.meagher@gmail.com
> To: user@hadoop.apache.org
> 
> Also, "Source Compatibility" also means ONLY a recompile is needed.
> No code changes should be needed.
> 
> On Mon, Apr 14, 2014 at 10:37 AM, John Meagher <john.meagher@gmail.com> wrote:
> > Source Compatibility = you need to recompile and use the new version
> > as part of the compilation
> >
> > Binary Compatibility = you can take something compiled against the old
> > version and run it on the new version
> >
> > On Mon, Apr 14, 2014 at 9:19 AM, Radhe Radhe
> > <radhe.krishna.radhe@live.com> wrote:
> >> Hello People,
> >>
> >> As per the Apache site
> >> http://hadoop.apache.org/docs/r2.3.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html
> >>
> >> Binary Compatibility
> >> ====================
> >> First, we ensure binary compatibility to the applications that use old
> >> mapred APIs. This means that applications which were built against MRv1
> >> mapred APIs can run directly on YARN without recompilation, merely by
> >> pointing them to an Apache Hadoop 2.x cluster via configuration.
> >>
> >> Source Compatibility
> >> ====================
> >> We cannot ensure complete binary compatibility with the applications that
> >> use mapreduce APIs, as these APIs have evolved a lot since MRv1. However, we
> >> ensure source compatibility for mapreduce APIs that break binary
> >> compatibility. In other words, users should recompile their applications
> >> that use mapreduce APIs against MRv2 jars. One notable binary
> >> incompatibility break is Counter and CounterGroup.
> >>
> >> For "Binary Compatibility" I understand that if I had build a MR job with
> >> old *mapred* APIs then they can be run directly on YARN without and changes.
> >>
> >> Can anybody explain what do we mean by "Source Compatibility" here and also
> >> a use case where one will need it?
> >>
> >> Does that mean code changes if I already have a MR job source code written
> >> with with old *mapred* APIs and I need to make some changes to it to run in
> >> then I need to use the new "mapreduce* API and generate the new  binaries?
> >>
> >> Thanks,
> >> -RR
> >>
> >>
 		 	   		  
Mime
View raw message