Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: message received from 54.76.25.247 which
 is an MX secondary for dev@hbase.apache.org)
MIME-Version: 1.0
In-Reply-To: 
 <CADcMMgGWNZDppmF59EPpXZkANPzTKba3ypvWYdBeYMqhTP8c3A@mail.gmail.com>
References: 
 <CA+RK=_CkTUfa3nWPsy2A0PZt07h=1stP72rcazBrw5U0JFdgXA@mail.gmail.com>
 <5537D45D.3020207@gmail.com>
 <CAGHyZ6L3kMQgACjujFyGBOiKbHNXW=-hc=7+G-cE-+y5BDSakg@mail.gmail.com>
 <5537DBF5.6080204@gmail.com>
 <CA+RK=_AnCe-XwGjguw1=SyZObBe2EVLO=969C_Qy1ChEdiZfWQ@mail.gmail.com>
 <CAMUu0w_4YM-oM0u4KQ0AOdrxGf=qkGxdk5Z2PZ9=WS5Nw0xLvQ@mail.gmail.com>
 <CAGHyZ6LtU+CJP9fAtq+wSq-kFsXyPvFG2qPLPoHiOP+QhWSUWQ@mail.gmail.com>
 <CADcMMgGmfwSxDPMs3rR-JMzvn4bSyXZCyc+OHG_eb4s9aqFsOg@mail.gmail.com>
 <CAGHyZ6KukpNxXOgSXzOEvdhagdXQAfZ00knLor2jAyqAbR_A1Q@mail.gmail.com>
 <CAMUu0w9auTGgqACo_Eizv+q7jP8AMjK+omDdBLnECuFtAyzs-g@mail.gmail.com>
 <CA+RK=_BUt-xcytK_7mrHnaxO8KMRcH_0D=-dspttTh+4zEjJsQ@mail.gmail.com>
 <CADcMMgGWNZDppmF59EPpXZkANPzTKba3ypvWYdBeYMqhTP8c3A@mail.gmail.com>
From: Nick Dimiduk <ndimiduk@gmail.com>
Date: Thu, 23 Apr 2015 17:34:38 -0700
Message-ID: 
 <CANZa=Gsxaw5gsE7KL_FN2oigOVCwj0CUwJPYu6ib0N6k8kkOUQ@mail.gmail.com>
Subject: Re: Clarifying interface evolution freedom in patch releases (was:
 Re: [VOTE] Third release candidate for HBase 1.0.1 (RC2))
To: hbase-dev <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=f46d0442883ee5fc5e05146d9068

--f46d0442883ee5fc5e05146d9068
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Thu, Apr 23, 2015 at 4:13 PM, Stack <stack@duboce.net> wrote:

> Does this 'admission' help with the which-hadoop thread too?
>

Right -- "working toward semver".

Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still
have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved.

What about the jackson issue with 2.5 YARN runtime?

Thanks Sean and Josh for being our community conscience on these issues.

> Just want to make sure we're all on the same page (or find out if not).
> >
> > On Thu, Apr 23, 2015 at 2:59 PM, Enis S=C3=B6ztutar <enis@apache.org> w=
rote:
> >
> > > Then let's get Andrew's proposal committed:
> > >
> > > + APIs available in a patch version will be available in all later
> > > + patch versions. However, new APIs may be added which will not be
> > > + available in earlier patch versions.
> > >
> > > I propose the following change to the "Client Binary compatibility"
> > section
> > > of Section 11.1:
> > >
> > > - Old client code can run unchanged (no recompilation needed) against
> new
> > > jars.
> > > + Client code written to APIs available in a given patch release
> > > + can run unchanged (no recompilation needed) against the new
> > > + jars of later patch versions.
> > >
> > > We can even claim that if you are not using new APIs, we are binary
> > compat
> > > for downgrade. But you have to make sure that your code compiles with
> > that
> > > downgraded version.
> > >
> > > Enis
> > >
> > >
> > >
> > > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <busbey@cloudera.com>
> > wrote:
> > >
> > > > I'm fine with us adding methods to public APIs in patch releases so
> > long
> > > as
> > > > we stop claiming to follow semver. We can say we take the principle=
s
> of
> > > > semver as inspiration. This would reflect the current state of the
> > world
> > > > WRT 1.0.1 and would still give us a reason keep the narrower
> definition
> > > of
> > > > "new features" in minor versions.
> > > >
> > > >
> > > > No longer claiming semver would also have the added benefit of maki=
ng
> > it
> > > > for me to easier to explain our compatibility promises to people.
> Right
> > > now
> > > > I have to explain the difference between the things that get proper
> > > semver
> > > > treatment (e.g. public api, wire compat) and which things are
> > downgraded
> > > to
> > > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility).
> Starting
> > > > with the context of "we like semver" will be easier than "we use
> > semver".
> > > >
> > > >
> > > > Like Josh, my main concern is that we accurately advertise what we'=
re
> > > > doing. There are few things I've found more frustrating than being =
an
> > end
> > > > user of a project that claims to follow semver without understandin=
g
> > the
> > > > burden that carries (and subsequently not meeting it).
> > > >
> > > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <stack@duboce.net> wrote:
> > > >
> > > > > I agree w/ the Enis characterization (so we need the callout on
> > semvar)
> > > > but
> > > > > think we should practice what Seans says (patch is bug fixes only=
).
> > > > > St.Ack
> > > > >
> > > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <busbey@cloudera.com=
>
> > > > wrote:
> > > > >
> > > > > > Why don't we just focus development after a minor release on th=
e
> > next
> > > > > minor
> > > > > > release instead of the next patch release?
> > > > > >
> > > > > > We could limit backports to the patch releases to critical bugs=
,
> > > which
> > > > > > would cut down on how often someone has to deal with the pain o=
f
> > > making
> > > > > > sure we don't add to public APIs. It also reduces the risk
> someone
> > > > going
> > > > > > through an upgrade has, since there are fewer changes.
> > > > > >
> > > > > > If someone fixes a bug and doesn't want to do the work of makin=
g
> > sure
> > > > it
> > > > > > doesn't add methods in a patch release, they just don't backpor=
t
> to
> > > > that
> > > > > > version and make a follow on e.g. "backport to 1.0.z" ticket.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis S=C3=B6ztutar <
> enis.soz@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > +1 to the proposal.
> > > > > > >
> > > > > > > The problem is that we have a very big API surface especially
> > with
> > > > the
> > > > > > > coprocessors included in the report. Even simple bug fixes ca=
n
> > > > > introduce
> > > > > > > protected or public methods to base classes, which makes patc=
h
> > > > releases
> > > > > > > very hard to maintain. I would not want to spend the effort t=
o
> > > spend
> > > > > tons
> > > > > > > of time trying to make a patch not introduce new methods in
> order
> > > to
> > > > > > > backport. That effort can be spent elsewhere IMO.
> > > > > > >
> > > > > > > Looking at the report
> > > > > > >
> > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html,
> > > > > > nothing
> > > > > > > strikes me as "new functionality". Going from current 1.0.0 t=
o
> > > > 1.0.1RC2
> > > > > > > should actually be as you would expect from upgrading a patch
> > > > release.
> > > > > > >
> > > > > > > Yes, adding new API in patch releases will make downgrading
> > harder,
> > > > > but I
> > > > > > > think that is an acceptable tradeoff. We can document that if
> > your
> > > > > > > application compiles (meaning that you are not using new API)
> > with
> > > > > 1.0.0,
> > > > > > > then you can swap your jars in a binary compat manner.
> > > > > > >
> > > > > > > Enis
> > > > > > >
> > > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell <
> > > > apurtell@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Anyone disagree with the point of view put forward by Josh
> and
> > > > Sean?
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser <
> > > josh.elser@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Andy -- I understood your intent, but thanks for
> clarifying.
> > > (as
> > > > > well
> > > > > > > as
> > > > > > > > > taking the time to break this discussion out in the first
> > > > place). I
> > > > > > > agree
> > > > > > > > > with your assessment.
> > > > > > > > >
> > > > > > > > > re: Sean's comments, if it wasn't clear by me asking in t=
he
> > > first
> > > > > > > place,
> > > > > > > > I
> > > > > > > > > also think sticking as close as possible to semver's rule=
s
> is
> > > the
> > > > > > best
> > > > > > > > > approach, although I'm getting the impression that there
> have
> > > > been
> > > > > > some
> > > > > > > > > previous reservations to doing so (especially by your
> comment
> > > > about
> > > > > > > > > backporting features if there is demand is).
> > > > > > > > >
> > > > > > > > > I've found adhering to the bug-fix release restrictions c=
an
> > be
> > > a
> > > > > very
> > > > > > > > > painful and time-consuming task, so this is something to
> get
> > a
> > > > > > > > > representative sampling of those who do the work to make
> sure
> > > > > > everyone
> > > > > > > is
> > > > > > > > > on board.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Sean Busbey wrote:
> > > > > > > > >
> > > > > > > > >> I'd much rather we stick with the definitions used in
> > Semantic
> > > > > > > > Versioning.
> > > > > > > > >> Our use is already confusing enough given our matrix of
> > > > > > > compatibilities
> > > > > > > > >> that don't get "major version for breaking" protections.
> > > > > > > > >>
> > > > > > > > >> We've previously discussed how we'll do additional minor
> > > > releases
> > > > > > when
> > > > > > > > >> there's sufficient interest in the new features present
> > there.
> > > > > > What's
> > > > > > > > >> building that demand if any backwards compatible change
> can
> > go
> > > > > back
> > > > > > > > into a
> > > > > > > > >> patch release?
> > > > > > > > >>
> > > > > > > > >> Would we have an easier time restraining ourselves if we
> > had a
> > > > > > regular
> > > > > > > > >> schedule planned around new minor versions?
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser<
> > > > josh.elser@gmail.com
> > > > > >
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >>  While I can understand the desire to want to add things=
,
> I
> > do
> > > > > think
> > > > > > > it
> > > > > > > > >>> makes things harder for users to reliably write code
> > against
> > > > > > versions
> > > > > > > > of
> > > > > > > > >>> HBase which (by their view) should be completely
> compatible
> > > > with
> > > > > > one
> > > > > > > > >>> another.
> > > > > > > > >>>
> > > > > > > > >>> Take this extremely hypothetical situation: I'm new to
> > HBase
> > > > and
> > > > > > > start
> > > > > > > > >>> writing some code against HBase 1.0.1 which was just
> > deployed
> > > > at
> > > > > my
> > > > > > > > >>> $job. I
> > > > > > > > >>> don't _know_ what APIs are new, I just know what exists
> and
> > > > treat
> > > > > > > that
> > > > > > > > as
> > > > > > > > >>> acceptable for me to be using. Meanwhile in production,
> > some
> > > > > other
> > > > > > > > people
> > > > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 whic=
h
> > they
> > > > had
> > > > > > > been
> > > > > > > > >>> previously using. My reaction would be "of course my co=
de
> > > > should
> > > > > > work
> > > > > > > > >>> with
> > > > > > > > >>> HBase 1.0.0, I only used the public API" when in fact
> this
> > is
> > > > not
> > > > > > > true.
> > > > > > > > >>>
> > > > > > > > >>> Personally, I think it's a little bold to say semver is
> > even
> > > in
> > > > > use
> > > > > > > if
> > > > > > > > >>> this principal isn't being followed as it doesn't follo=
w
> at
> > > all
> > > > > > with
> > > > > > > my
> > > > > > > > >>> understanding on the guarantees defined by semver for
> > bug-fix
> > > > > > > releases.
> > > > > > > > >>>
> > > > > > > > >>> That being said, if the intent *is* to allow ourselves =
to
> > > make
> > > > > > these
> > > > > > > > >>> sorts
> > > > > > > > >>> of changes, I just think some sort of disclaimer should
> be
> > > > > present:
> > > > > > > > >>>
> > > > > > > > >>> - HBase uses Semantic Versioning for its release
> versioning
> > > > > > > > >>> + HBase uses Semantic Versioning for its release
> versioning
> > > > with
> > > > > a
> > > > > > > > caveat
> > > > > > > > >>> that methods and members might be added in newer bug-fi=
x
> > > > releases
> > > > > > > that
> > > > > > > > >>> were
> > > > > > > > >>> not present in the previous bug-fix release.
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>> Andrew Purtell wrote:
> > > > > > > > >>>
> > > > > > > > >>>  [Subject changed]
> > > > > > > > >>>>
> > > > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser<
> > > > > josh.elser@gmail.com>
> > > > > > > > >>>>  wrote:
> > > > > > > > >>>>
> > > > > > > > >>>>   I was a little surprised when I noticed method
> additions
> > > to
> > > > > > > > >>>>
> > > > > > > > >>>>> InterfaceAudience.Public annotated classes. This mean=
s
> > > that a
> > > > > > user
> > > > > > > > >>>>> could
> > > > > > > > >>>>> write code against 1.0.1 that would not work against
> > 1.0.0
> > > > > which
> > > > > > > > seems
> > > > > > > > >>>>> undesirable for a bugfix release. I read over the boo=
k
> > > > section
> > > > > on
> > > > > > > > >>>>> compatibility and didn't see this addressed, so I
> thought
> > > I'd
> > > > > > ask.
> > > > > > > > >>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>> Let's clarify this. It's not the first time this
> question
> > > has
> > > > > been
> > > > > > > > >>>> asked.
> > > > > > > > >>>>
> > > > > > > > >>>> To get things moving:
> > > > > > > > >>>>
> > > > > > > > >>>> I propose the following addition to the "Client API
> > > > > compatibility"
> > > > > > > > >>>> section
> > > > > > > > >>>> of Section 11.1:
> > > > > > > > >>>>
> > > > > > > > >>>> + APIs available in a patch version will be available =
in
> > all
> > > > > later
> > > > > > > > >>>> + patch versions. However, new APIs may be added which
> > will
> > > > not
> > > > > be
> > > > > > > > >>>> + available in earlier patch versions.
> > > > > > > > >>>>
> > > > > > > > >>>> I propose the following change to the "Client Binary
> > > > > > compatibility"
> > > > > > > > >>>> section
> > > > > > > > >>>> of Section 11.1:
> > > > > > > > >>>>
> > > > > > > > >>>> - Old client code can run unchanged (no recompilation
> > > needed)
> > > > > > > against
> > > > > > > > >>>> new
> > > > > > > > >>>> jars.
> > > > > > > > >>>> + Client code written to APIs available in a given pat=
ch
> > > > release
> > > > > > > > >>>> + can run unchanged (no recompilation needed) against
> the
> > > new
> > > > > > > > >>>> + jars of later patch versions.
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>> What do you think?
> > > > > > > > >>>>
> > > > > > > > >>>> If these changes are (mostly) ok, then this clarifies =
in
> > one
> > > > > > > > direction.
> > > > > > > > >>>>
> > > > > > > > >>>> If these changes are not acceptable, I will propose
> edits
> > > that
> > > > > > > clarify
> > > > > > > > >>>> toward the opposite meaning. =E2=80=8B
> > > > > > > > >>>>
> > >
> >
>

--f46d0442883ee5fc5e05146d9068--