Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of busbey@cloudera.com
 designates 209.85.216.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAMUu0w-J00yuMenZTvm8YD4NHfGmF2Q3s=VX3EQbWFw=rju9UQ@mail.gmail.com>
References: 
 <CAMUu0w_V-jdEKg4rx+_aQ-_wLNLKyoDHigEakvSZdE+YL5Hr_g@mail.gmail.com>
 <CAGHyZ6JpD0YXbt=trhn=0=Amm+w8prrU580jxB=Na5CKqnr32w@mail.gmail.com>
 <CAMUu0w9B1w2r7GuymhTJDqdb6OnsEFMz8_5S0wX_xXJprvkoxg@mail.gmail.com>
 <CAGHyZ6+MFw6HH-sSrBW+N=uEw51bJ75O2bbZ4We-DxEbsfGcKQ@mail.gmail.com>
 <CAMUu0w-J00yuMenZTvm8YD4NHfGmF2Q3s=VX3EQbWFw=rju9UQ@mail.gmail.com>
From: Sean Busbey <busbey@cloudera.com>
Date: Wed, 11 Mar 2015 17:26:02 -0500
Message-ID: 
 <CAGHyZ6KDjzXrcYXejDC1KM2czYQZ4hiH7OWqXgi6RbLsTP8srw@mail.gmail.com>
Subject: Re: [DISCUSS] Dependency compatibility
To: dev <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a11357630d2ab0305110ac196

--001a11357630d2ab0305110ac196
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Wed, Mar 11, 2015 at 5:19 PM, Enis S=C3=B6ztutar <enis.soz@gmail.com> wr=
ote:

> On Wed, Mar 11, 2015 at 3:07 PM, Sean Busbey <busbey@cloudera.com> wrote:
>
> > On Wed, Mar 11, 2015 at 4:49 PM, Enis S=C3=B6ztutar <enis.soz@gmail.com=
>
> wrote:
> >
> > > >
> > > > It's worth noting that if users follow our ref guide (which says to
> use
> > > > "hadoop jar"), then jobs don't fail. It's only when they attempt to
> > > launch
> > > > jobs using "hbase com.example.MyDriver" that things fail.
> > > >
> > > > Additionally, if we stick to telling users that only the "hadoop ja=
r"
> > > > version is supported, we can rely on the application classpath
> support
> > > > built into Hadoop 2.6+ to make it so jobs built on us get our
> > dependency
> > > > version and not the ones from Hadoop as it changes.
> > > >
> > >
> > > We have learned that the users do not read or follow documentation. A=
nd
> > it
> > > is a regression
> > > if launching job using hbase command does not work.
> > >
> > >
> > >
> > They do when things break. ;) An additional troubleshooting section tha=
t
> > shows the error and says "remember to use hadoop jar" would nicely help
> > catch searchers.
> >
> > Furthermore, "hadoop jar" is how you're supposed to launch YARN apps. I=
f
> we
> > say that doing things via the hbase command is acceptable, we're openin=
g
> > ourselves up to an expansion of what the hbase command has to do. (i.e.
> > perhaps it should detect if the passed class is a YARN driver and then
> use
> > the hadoop jar command? or should it always pass through to the hadoop
> jar
> > command?)
> >
>
> Traditionally, and in our documentation, HBase owned MR classes (CopyTabl=
e,
> Import, etc) are run
> with the hbase script, not the hadoop script. It is a regression in that
> sense still. Yes, there is a
> workaround, but why we bother where we can fix this easily.
>
>
All of our current ref guide examples use the "hadoop jar" command:

http://hbase.apache.org/book.html#hbase.mapreduce.classpath

http://hbase.apache.org/book.html#_bundled_hbase_mapreduce_jobs

They only rely on the hbase command to get things that need to be added to
hte hadop classpath.


>
> >
> >
> >
> > > >
> > > >
> > > >
> > > > > So, my proposal is:
> > > > >  - Commit HBASE-13149 to master and 1.1
> > > > >  - Either change the dependency compat story for minor versions t=
o
> > > false,
> > > > > or add a footnote saying that there may be exceptions because of
> the
> > > > > reasons listed above.
> > > > >
> > > >
> > > >
> > > > If we decide we need to do the jackson version bump, what about the
> > > > possibility of moving the code in branch-1 to be version 2.0.0 (and
> > > making
> > > > master 3.0.0). We could start the release process once the changes
> > Andrew
> > > > needs for Phoenix are in place and get it out the door.
> > > >
> > >
> > > I don't think this requires a major version bump. As I was mentioning
> in
> > > the other
> > > thread, HBase is not upgraded too frequently in production. Again, we
> do
> > > not want
> > > to inconvenience the user even further.
> > >
> > >
> > >
> > How would this inconvenience users further? Barring the change in versi=
on
> > numbers, it's the same upgrade they would be doing to move to what we'r=
e
> > currently calling HBase 1.1. Since version numbers under semver signal
> what
> > we understand about our changeset, it's just us acknowledging that we
> broke
> > some kind of compatibility. A release note that calls out the Jackson
> > dependency as the cause for that compatibility breakage makes the
> > evaluation easy.
> >
>
> The problem is boils down to "major versions are cheap" kind of argument,
> which have
> been discussed in Hadoop context. I do not buy it, because a major versio=
n
> upgrade implies
> (though do not have to be) a big change. I don't see why ever we would wa=
nt
> to bump
> our major version, where the said library only bumped their minor version=
.
> Jackson could
> have went with 2.0 for those changes between 1.8 and 1.9. Why would we wa=
nt
> to
> promise more than what our dependencies promise? It is not realistic.
>
>

But we _know_ that 1.8 -> 1.9 in jackson is not really a minor version in
the semver sense. we _know_ it's a breaking change. Their release notes
even call out breaking changes, so we know it isn't just a change in
internals.


--=20
Sean

--001a11357630d2ab0305110ac196--