hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Agarwal <aagar...@hortonworks.com>
Subject Re: Hadoop Code of Incompatible Changes
Date Tue, 29 Jul 2014 21:24:31 GMT
I cleared out the wiki page and left a forwarding link to the site docs.
>From a quick scan all the content is included in the site docs.


On Tue, Jul 29, 2014 at 2:14 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Eli pointed out to me that this is the up-to-date compatibility guide:
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html
>
> Thanks,
> Sandy
>
>
> On Tue, Jul 29, 2014 at 9:39 AM, Sandy Ryza <sandy.ryza@cloudera.com>
> wrote:
>
> > Hi Zhijie,
> >
> > The Hadoop compatibility guide mentions this as "semantic compatibility":
> > http://wiki.apache.org/hadoop/Compatibility
> >
> > My interpretation of the section is that we can't change the behavior of
> > public APIs unless we're fixing buggy behavior.  If the change could
> break
> > an existing application that's behaving reasonably with respect to the
> old
> > API, it's an incompatible change.
> >
> > -Sandy
> >
> >
> >
> > On Tue, Jul 29, 2014 at 9:26 AM, Zhijie Shen <zshen@hortonworks.com>
> > wrote:
> >
> >> Hi folks,
> >>
> >> Recently we have a conversation on YARN-2209 about the incompatible
> >> changes
> >> over releases. For those API changes that will break binary
> compatibility,
> >> source compatibility towards the existing API users, we've already had a
> >> rather clear picture about what we should do. However, YARN-2209 has
> >> introduced another case which I'm not quite sure about, which is kind of
> >> *logic
> >> incompatibility*.
> >>
> >> In detail, an ApplicationMasterProtocol API is going to throw an
> exception
> >> which is not expected before. The exception is a sub-class of
> >> YarnException, such that it doesn't need any method signature change,
> and
> >> won't break any binary/source compatibility. However, the exception is
> not
> >> expected before, but needs to be treated specially at the AM side. Not
> >> being aware of the newly introduced exception, the existing YARN
> >> applications' AM may not handle it exception properly, and is at the
> risk
> >> of being broken on a new YARN cluster after this change.
> >>
> >> An additional thought around this problem is that the change of what
> >> exception is to throw under what situation may be considered as a *soft
> >> *method
> >> signature change, because we're supposed to write this javadoc to tell
> the
> >> users (though we didn't it well in Hadoop), and users refer to it to
> guide
> >> how to handle the exception.
> >>
> >> In a more generalized form, let's assume we have a method, which behaves
> >> as
> >> A, in release 1.0. However, in release 2.0, the method signature has
> kept
> >> the same, while its behavior is altered from A to B. A and B are
> different
> >> behaviors. In this case, do we consider it as an incompatible change?
> >>
> >> I think it's somewhat a common issue, such that I raise it on the
> mailing
> >> list. Please share your ideas.
> >>
> >> Thanks,
> >> Zhijie
> >>
> >> --
> >> Zhijie Shen
> >> Hortonworks Inc.
> >> http://hortonworks.com/
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or entity
> >> to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> >> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> >> immediately
> >> and delete it from your system. Thank You.
> >>
> >
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message