hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: Hadoop Code of Incompatible Changes
Date Tue, 29 Jul 2014 21:14:21 GMT
Eli pointed out to me that this is the up-to-date compatibility guide:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html

Thanks,
Sandy


On Tue, Jul 29, 2014 at 9:39 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Hi Zhijie,
>
> The Hadoop compatibility guide mentions this as "semantic compatibility":
> http://wiki.apache.org/hadoop/Compatibility
>
> My interpretation of the section is that we can't change the behavior of
> public APIs unless we're fixing buggy behavior.  If the change could break
> an existing application that's behaving reasonably with respect to the old
> API, it's an incompatible change.
>
> -Sandy
>
>
>
> On Tue, Jul 29, 2014 at 9:26 AM, Zhijie Shen <zshen@hortonworks.com>
> wrote:
>
>> Hi folks,
>>
>> Recently we have a conversation on YARN-2209 about the incompatible
>> changes
>> over releases. For those API changes that will break binary compatibility,
>> source compatibility towards the existing API users, we've already had a
>> rather clear picture about what we should do. However, YARN-2209 has
>> introduced another case which I'm not quite sure about, which is kind of
>> *logic
>> incompatibility*.
>>
>> In detail, an ApplicationMasterProtocol API is going to throw an exception
>> which is not expected before. The exception is a sub-class of
>> YarnException, such that it doesn't need any method signature change, and
>> won't break any binary/source compatibility. However, the exception is not
>> expected before, but needs to be treated specially at the AM side. Not
>> being aware of the newly introduced exception, the existing YARN
>> applications' AM may not handle it exception properly, and is at the risk
>> of being broken on a new YARN cluster after this change.
>>
>> An additional thought around this problem is that the change of what
>> exception is to throw under what situation may be considered as a *soft
>> *method
>> signature change, because we're supposed to write this javadoc to tell the
>> users (though we didn't it well in Hadoop), and users refer to it to guide
>> how to handle the exception.
>>
>> In a more generalized form, let's assume we have a method, which behaves
>> as
>> A, in release 1.0. However, in release 2.0, the method signature has kept
>> the same, while its behavior is altered from A to B. A and B are different
>> behaviors. In this case, do we consider it as an incompatible change?
>>
>> I think it's somewhat a common issue, such that I raise it on the mailing
>> list. Please share your ideas.
>>
>> Thanks,
>> Zhijie
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message