hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: Hadoop Code of Incompatible Changes
Date Tue, 29 Jul 2014 21:26:08 GMT
Thanks Arpit!


On Tue, Jul 29, 2014 at 2:24 PM, Arpit Agarwal <aagarwal@hortonworks.com>
wrote:

> I cleared out the wiki page and left a forwarding link to the site docs.
> From a quick scan all the content is included in the site docs.
>
>
> On Tue, Jul 29, 2014 at 2:14 PM, Sandy Ryza <sandy.ryza@cloudera.com>
> wrote:
>
> > Eli pointed out to me that this is the up-to-date compatibility guide:
> >
> >
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html
> >
> > Thanks,
> > Sandy
> >
> >
> > On Tue, Jul 29, 2014 at 9:39 AM, Sandy Ryza <sandy.ryza@cloudera.com>
> > wrote:
> >
> > > Hi Zhijie,
> > >
> > > The Hadoop compatibility guide mentions this as "semantic
> compatibility":
> > > http://wiki.apache.org/hadoop/Compatibility
> > >
> > > My interpretation of the section is that we can't change the behavior
> of
> > > public APIs unless we're fixing buggy behavior.  If the change could
> > break
> > > an existing application that's behaving reasonably with respect to the
> > old
> > > API, it's an incompatible change.
> > >
> > > -Sandy
> > >
> > >
> > >
> > > On Tue, Jul 29, 2014 at 9:26 AM, Zhijie Shen <zshen@hortonworks.com>
> > > wrote:
> > >
> > >> Hi folks,
> > >>
> > >> Recently we have a conversation on YARN-2209 about the incompatible
> > >> changes
> > >> over releases. For those API changes that will break binary
> > compatibility,
> > >> source compatibility towards the existing API users, we've already
> had a
> > >> rather clear picture about what we should do. However, YARN-2209 has
> > >> introduced another case which I'm not quite sure about, which is kind
> of
> > >> *logic
> > >> incompatibility*.
> > >>
> > >> In detail, an ApplicationMasterProtocol API is going to throw an
> > exception
> > >> which is not expected before. The exception is a sub-class of
> > >> YarnException, such that it doesn't need any method signature change,
> > and
> > >> won't break any binary/source compatibility. However, the exception is
> > not
> > >> expected before, but needs to be treated specially at the AM side. Not
> > >> being aware of the newly introduced exception, the existing YARN
> > >> applications' AM may not handle it exception properly, and is at the
> > risk
> > >> of being broken on a new YARN cluster after this change.
> > >>
> > >> An additional thought around this problem is that the change of what
> > >> exception is to throw under what situation may be considered as a
> *soft
> > >> *method
> > >> signature change, because we're supposed to write this javadoc to tell
> > the
> > >> users (though we didn't it well in Hadoop), and users refer to it to
> > guide
> > >> how to handle the exception.
> > >>
> > >> In a more generalized form, let's assume we have a method, which
> behaves
> > >> as
> > >> A, in release 1.0. However, in release 2.0, the method signature has
> > kept
> > >> the same, while its behavior is altered from A to B. A and B are
> > different
> > >> behaviors. In this case, do we consider it as an incompatible change?
> > >>
> > >> I think it's somewhat a common issue, such that I raise it on the
> > mailing
> > >> list. Please share your ideas.
> > >>
> > >> Thanks,
> > >> Zhijie
> > >>
> > >> --
> > >> Zhijie Shen
> > >> Hortonworks Inc.
> > >> http://hortonworks.com/
> > >>
> > >> --
> > >> CONFIDENTIALITY NOTICE
> > >> NOTICE: This message is intended for the use of the individual or
> entity
> > >> to
> > >> which it is addressed and may contain information that is
> confidential,
> > >> privileged and exempt from disclosure under applicable law. If the
> > reader
> > >> of this message is not the intended recipient, you are hereby notified
> > >> that
> > >> any printing, copying, dissemination, distribution, disclosure or
> > >> forwarding of this communication is strictly prohibited. If you have
> > >> received this communication in error, please contact the sender
> > >> immediately
> > >> and delete it from your system. Thank You.
> > >>
> > >
> > >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message