hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Meagher <john.meag...@gmail.com>
Subject Re: Doubt regarding Binary Compatibility\Source Compatibility with old *mapred* APIs and new *mapreduce* APIs in Hadoop
Date Mon, 14 Apr 2014 14:37:56 GMT
Also, "Source Compatibility" also means ONLY a recompile is needed.
No code changes should be needed.

On Mon, Apr 14, 2014 at 10:37 AM, John Meagher <john.meagher@gmail.com> wrote:
> Source Compatibility = you need to recompile and use the new version
> as part of the compilation
> Binary Compatibility = you can take something compiled against the old
> version and run it on the new version
> On Mon, Apr 14, 2014 at 9:19 AM, Radhe Radhe
> <radhe.krishna.radhe@live.com> wrote:
>> Hello People,
>> As per the Apache site
>> http://hadoop.apache.org/docs/r2.3.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html
>> Binary Compatibility
>> ====================
>> First, we ensure binary compatibility to the applications that use old
>> mapred APIs. This means that applications which were built against MRv1
>> mapred APIs can run directly on YARN without recompilation, merely by
>> pointing them to an Apache Hadoop 2.x cluster via configuration.
>> Source Compatibility
>> ====================
>> We cannot ensure complete binary compatibility with the applications that
>> use mapreduce APIs, as these APIs have evolved a lot since MRv1. However, we
>> ensure source compatibility for mapreduce APIs that break binary
>> compatibility. In other words, users should recompile their applications
>> that use mapreduce APIs against MRv2 jars. One notable binary
>> incompatibility break is Counter and CounterGroup.
>> For "Binary Compatibility" I understand that if I had build a MR job with
>> old *mapred* APIs then they can be run directly on YARN without and changes.
>> Can anybody explain what do we mean by "Source Compatibility" here and also
>> a use case where one will need it?
>> Does that mean code changes if I already have a MR job source code written
>> with with old *mapred* APIs and I need to make some changes to it to run in
>> then I need to use the new "mapreduce* API and generate the new  binaries?
>> Thanks,
>> -RR

View raw message