hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Clarifying interface evolution freedom in patch releases (was: Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
Date Fri, 24 Apr 2015 23:56:44 GMT
On Thu, Apr 23, 2015 at 6:39 PM, Enis Söztutar <enis.soz@gmail.com> wrote:

> I would be in favor of going 2.6.0 and jackson upgrade in 1.1.
>

I am also +1 bumping to Hadoop 2.6.x and Jackson 1.9.x in HBase 1.1.0. This
seems like the most practical solution to the problems outlined here. Let
me explore the API surface area differences between Hadoop 2.5, 2.6 and see
what might crop up for users.

-n

On Thu, Apr 23, 2015 at 5:34 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > On Thu, Apr 23, 2015 at 4:13 PM, Stack <stack@duboce.net> wrote:
> >
> > > Does this 'admission' help with the which-hadoop thread too?
> > >
> >
> > Right -- "working toward semver".
> >
> > Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
> > have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.
> >
> > What about the jackson issue with 2.5 YARN runtime?
> >
> > Thanks Sean and Josh for being our community conscience on these issues.
> >
> > > Just want to make sure we're all on the same page (or find out if not).
> > > >
> > > > On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <enis@apache.org>
> > wrote:
> > > >
> > > > > Then let's get Andrew's proposal committed:
> > > > >
> > > > > + APIs available in a patch version will be available in all later
> > > > > + patch versions. However, new APIs may be added which will not be
> > > > > + available in earlier patch versions.
> > > > >
> > > > > I propose the following change to the "Client Binary compatibility"
> > > > section
> > > > > of Section 11.1:
> > > > >
> > > > > - Old client code can run unchanged (no recompilation needed)
> against
> > > new
> > > > > jars.
> > > > > + Client code written to APIs available in a given patch release
> > > > > + can run unchanged (no recompilation needed) against the new
> > > > > + jars of later patch versions.
> > > > >
> > > > > We can even claim that if you are not using new APIs, we are binary
> > > > compat
> > > > > for downgrade. But you have to make sure that your code compiles
> with
> > > > that
> > > > > downgraded version.
> > > > >
> > > > > Enis
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <busbey@cloudera.com>
> > > > wrote:
> > > > >
> > > > > > I'm fine with us adding methods to public APIs in patch releases
> so
> > > > long
> > > > > as
> > > > > > we stop claiming to follow semver. We can say we take the
> > principles
> > > of
> > > > > > semver as inspiration. This would reflect the current state
of
> the
> > > > world
> > > > > > WRT 1.0.1 and would still give us a reason keep the narrower
> > > definition
> > > > > of
> > > > > > "new features" in minor versions.
> > > > > >
> > > > > >
> > > > > > No longer claiming semver would also have the added benefit
of
> > making
> > > > it
> > > > > > for me to easier to explain our compatibility promises to people.
> > > Right
> > > > > now
> > > > > > I have to explain the difference between the things that get
> proper
> > > > > semver
> > > > > > treatment (e.g. public api, wire compat) and which things are
> > > > downgraded
> > > > > to
> > > > > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
> > > Starting
> > > > > > with the context of "we like semver" will be easier than "we
use
> > > > semver".
> > > > > >
> > > > > >
> > > > > > Like Josh, my main concern is that we accurately advertise what
> > we're
> > > > > > doing. There are few things I've found more frustrating than
> being
> > an
> > > > end
> > > > > > user of a project that claims to follow semver without
> > understanding
> > > > the
> > > > > > burden that carries (and subsequently not meeting it).
> > > > > >
> > > > > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <stack@duboce.net>
wrote:
> > > > > >
> > > > > > > I agree w/ the Enis characterization (so we need the callout
on
> > > > semvar)
> > > > > > but
> > > > > > > think we should practice what Seans says (patch is bug
fixes
> > only).
> > > > > > > St.Ack
> > > > > > >
> > > > > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <
> > busbey@cloudera.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Why don't we just focus development after a minor
release on
> > the
> > > > next
> > > > > > > minor
> > > > > > > > release instead of the next patch release?
> > > > > > > >
> > > > > > > > We could limit backports to the patch releases to
critical
> > bugs,
> > > > > which
> > > > > > > > would cut down on how often someone has to deal with
the pain
> > of
> > > > > making
> > > > > > > > sure we don't add to public APIs. It also reduces
the risk
> > > someone
> > > > > > going
> > > > > > > > through an upgrade has, since there are fewer changes.
> > > > > > > >
> > > > > > > > If someone fixes a bug and doesn't want to do the
work of
> > making
> > > > sure
> > > > > > it
> > > > > > > > doesn't add methods in a patch release, they just
don't
> > backport
> > > to
> > > > > > that
> > > > > > > > version and make a follow on e.g. "backport to 1.0.z"
ticket.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar <
> > > enis.soz@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 to the proposal.
> > > > > > > > >
> > > > > > > > > The problem is that we have a very big API surface
> especially
> > > > with
> > > > > > the
> > > > > > > > > coprocessors included in the report. Even simple
bug fixes
> > can
> > > > > > > introduce
> > > > > > > > > protected or public methods to base classes,
which makes
> > patch
> > > > > > releases
> > > > > > > > > very hard to maintain. I would not want to spend
the effort
> > to
> > > > > spend
> > > > > > > tons
> > > > > > > > > of time trying to make a patch not introduce
new methods in
> > > order
> > > > > to
> > > > > > > > > backport. That effort can be spent elsewhere
IMO.
> > > > > > > > >
> > > > > > > > > Looking at the report
> > > > > > > > >
> > > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > > > > > nothing
> > > > > > > > > strikes me as "new functionality". Going from
current 1.0.0
> > to
> > > > > > 1.0.1RC2
> > > > > > > > > should actually be as you would expect from upgrading
a
> patch
> > > > > > release.
> > > > > > > > >
> > > > > > > > > Yes, adding new API in patch releases will make
downgrading
> > > > harder,
> > > > > > > but I
> > > > > > > > > think that is an acceptable tradeoff. We can
document that
> if
> > > > your
> > > > > > > > > application compiles (meaning that you are not
using new
> API)
> > > > with
> > > > > > > 1.0.0,
> > > > > > > > > then you can swap your jars in a binary compat
manner.
> > > > > > > > >
> > > > > > > > > Enis
> > > > > > > > >
> > > > > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell
<
> > > > > > apurtell@apache.org>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Anyone disagree with the point of view put
forward by
> Josh
> > > and
> > > > > > Sean?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser
<
> > > > > josh.elser@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Andy -- I understood your intent, but
thanks for
> > > clarifying.
> > > > > (as
> > > > > > > well
> > > > > > > > > as
> > > > > > > > > > > taking the time to break this discussion
out in the
> first
> > > > > > place). I
> > > > > > > > > agree
> > > > > > > > > > > with your assessment.
> > > > > > > > > > >
> > > > > > > > > > > re: Sean's comments, if it wasn't clear
by me asking in
> > the
> > > > > first
> > > > > > > > > place,
> > > > > > > > > > I
> > > > > > > > > > > also think sticking as close as possible
to semver's
> > rules
> > > is
> > > > > the
> > > > > > > > best
> > > > > > > > > > > approach, although I'm getting the
impression that
> there
> > > have
> > > > > > been
> > > > > > > > some
> > > > > > > > > > > previous reservations to doing so (especially
by your
> > > comment
> > > > > > about
> > > > > > > > > > > backporting features if there is demand
is).
> > > > > > > > > > >
> > > > > > > > > > > I've found adhering to the bug-fix
release restrictions
> > can
> > > > be
> > > > > a
> > > > > > > very
> > > > > > > > > > > painful and time-consuming task, so
this is something
> to
> > > get
> > > > a
> > > > > > > > > > > representative sampling of those who
do the work to
> make
> > > sure
> > > > > > > > everyone
> > > > > > > > > is
> > > > > > > > > > > on board.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Sean Busbey wrote:
> > > > > > > > > > >
> > > > > > > > > > >> I'd much rather we stick with the
definitions used in
> > > > Semantic
> > > > > > > > > > Versioning.
> > > > > > > > > > >> Our use is already confusing enough
given our matrix
> of
> > > > > > > > > compatibilities
> > > > > > > > > > >> that don't get "major version for
breaking"
> protections.
> > > > > > > > > > >>
> > > > > > > > > > >> We've previously discussed how
we'll do additional
> minor
> > > > > > releases
> > > > > > > > when
> > > > > > > > > > >> there's sufficient interest in
the new features
> present
> > > > there.
> > > > > > > > What's
> > > > > > > > > > >> building that demand if any backwards
compatible
> change
> > > can
> > > > go
> > > > > > > back
> > > > > > > > > > into a
> > > > > > > > > > >> patch release?
> > > > > > > > > > >>
> > > > > > > > > > >> Would we have an easier time restraining
ourselves if
> we
> > > > had a
> > > > > > > > regular
> > > > > > > > > > >> schedule planned around new minor
versions?
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM,
Josh Elser<
> > > > > > josh.elser@gmail.com
> > > > > > > >
> > > > > > > > > > >> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>  While I can understand the desire
to want to add
> > things,
> > > I
> > > > do
> > > > > > > think
> > > > > > > > > it
> > > > > > > > > > >>> makes things harder for users
to reliably write code
> > > > against
> > > > > > > > versions
> > > > > > > > > > of
> > > > > > > > > > >>> HBase which (by their view)
should be completely
> > > compatible
> > > > > > with
> > > > > > > > one
> > > > > > > > > > >>> another.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Take this extremely hypothetical
situation: I'm new
> to
> > > > HBase
> > > > > > and
> > > > > > > > > start
> > > > > > > > > > >>> writing some code against HBase
1.0.1 which was just
> > > > deployed
> > > > > > at
> > > > > > > my
> > > > > > > > > > >>> $job. I
> > > > > > > > > > >>> don't _know_ what APIs are
new, I just know what
> exists
> > > and
> > > > > > treat
> > > > > > > > > that
> > > > > > > > > > as
> > > > > > > > > > >>> acceptable for me to be using.
Meanwhile in
> production,
> > > > some
> > > > > > > other
> > > > > > > > > > people
> > > > > > > > > > >>> find a bug with HBase 1.0.1
and roll back to 1.0.0
> > which
> > > > they
> > > > > > had
> > > > > > > > > been
> > > > > > > > > > >>> previously using. My reaction
would be "of course my
> > code
> > > > > > should
> > > > > > > > work
> > > > > > > > > > >>> with
> > > > > > > > > > >>> HBase 1.0.0, I only used the
public API" when in fact
> > > this
> > > > is
> > > > > > not
> > > > > > > > > true.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Personally, I think it's a
little bold to say semver
> is
> > > > even
> > > > > in
> > > > > > > use
> > > > > > > > > if
> > > > > > > > > > >>> this principal isn't being
followed as it doesn't
> > follow
> > > at
> > > > > all
> > > > > > > > with
> > > > > > > > > my
> > > > > > > > > > >>> understanding on the guarantees
defined by semver for
> > > > bug-fix
> > > > > > > > > releases.
> > > > > > > > > > >>>
> > > > > > > > > > >>> That being said, if the intent
*is* to allow
> ourselves
> > to
> > > > > make
> > > > > > > > these
> > > > > > > > > > >>> sorts
> > > > > > > > > > >>> of changes, I just think some
sort of disclaimer
> should
> > > be
> > > > > > > present:
> > > > > > > > > > >>>
> > > > > > > > > > >>> - HBase uses Semantic Versioning
for its release
> > > versioning
> > > > > > > > > > >>> + HBase uses Semantic Versioning
for its release
> > > versioning
> > > > > > with
> > > > > > > a
> > > > > > > > > > caveat
> > > > > > > > > > >>> that methods and members might
be added in newer
> > bug-fix
> > > > > > releases
> > > > > > > > > that
> > > > > > > > > > >>> were
> > > > > > > > > > >>> not present in the previous
bug-fix release.
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>> Andrew Purtell wrote:
> > > > > > > > > > >>>
> > > > > > > > > > >>>  [Subject changed]
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> On Tue, Apr 21, 2015 at
8:47 PM, Josh Elser<
> > > > > > > josh.elser@gmail.com>
> > > > > > > > > > >>>>  wrote:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>   I was a little surprised
when I noticed method
> > > additions
> > > > > to
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>> InterfaceAudience.Public
annotated classes. This
> > means
> > > > > that a
> > > > > > > > user
> > > > > > > > > > >>>>> could
> > > > > > > > > > >>>>> write code against
1.0.1 that would not work
> against
> > > > 1.0.0
> > > > > > > which
> > > > > > > > > > seems
> > > > > > > > > > >>>>> undesirable for a bugfix
release. I read over the
> > book
> > > > > > section
> > > > > > > on
> > > > > > > > > > >>>>> compatibility and didn't
see this addressed, so I
> > > thought
> > > > > I'd
> > > > > > > > ask.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>> Let's clarify this. It's
not the first time this
> > > question
> > > > > has
> > > > > > > been
> > > > > > > > > > >>>> asked.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> To get things moving:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> I propose the following
addition to the "Client API
> > > > > > > compatibility"
> > > > > > > > > > >>>> section
> > > > > > > > > > >>>> of Section 11.1:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> + APIs available in a patch
version will be
> available
> > in
> > > > all
> > > > > > > later
> > > > > > > > > > >>>> + patch versions. However,
new APIs may be added
> which
> > > > will
> > > > > > not
> > > > > > > be
> > > > > > > > > > >>>> + available in earlier
patch versions.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> I propose the following
change to the "Client Binary
> > > > > > > > compatibility"
> > > > > > > > > > >>>> section
> > > > > > > > > > >>>> of Section 11.1:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> - Old client code can run
unchanged (no
> recompilation
> > > > > needed)
> > > > > > > > > against
> > > > > > > > > > >>>> new
> > > > > > > > > > >>>> jars.
> > > > > > > > > > >>>> + Client code written to
APIs available in a given
> > patch
> > > > > > release
> > > > > > > > > > >>>> + can run unchanged (no
recompilation needed)
> against
> > > the
> > > > > new
> > > > > > > > > > >>>> + jars of later patch versions.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> What do you think?
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> If these changes are (mostly)
ok, then this
> clarifies
> > in
> > > > one
> > > > > > > > > > direction.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> If these changes are not
acceptable, I will propose
> > > edits
> > > > > that
> > > > > > > > > clarify
> > > > > > > > > > >>>> toward the opposite meaning.
​
> > > > > > > > > > >>>>
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message