hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerry He <jerry...@gmail.com>
Subject Re: [DISCUSS] Dependency compatibility
Date Fri, 20 Mar 2015 21:59:44 GMT
On Wed, Mar 18, 2015 at 7:38 AM, Jonathan Hsieh <jon@cloudera.com> wrote:

> We are now considered a mature project and as a "middle turtle" project we
> need to consider the projects that we will inconvenience because
> they depend on us. All the folks in this discussion are hbase devs and or
> sophisticated hbase users.  I agree with one of Sean's thoughts -- let us
> validate if this is a concern for users or not by have a discussion on
> user@
> to see how our users feel about it.  Maybe this is no big deal for them.


This probably can serve as a head-up for the user community as well.


>
>
Let's say that this is a concern for the users.  I like Lars's pragmatic
> critieria -- if the dep change doesn't cause problems for intra version
> interoperability then, let's let it in and let's update our policy.  If it
> does cause problem's let's keep it out and stick to our guns.
>
> To be more concrete there are two scenarios we could test:
> 1)  an mr job that uses hbase 1.0 client (and it's hadoop dep) needs to
> work out of the box against 1.1 hbase servers (and it's hadoop deps) or a
> mixed hbase 1.1- hbase 1.0  environment (e.g. assume we're in the middle of
> rolling upgrading hbase) without failing due to dependency compat checks.
>

See the coverage matrix below


> 2) an mr job that uses hbase 1.0 client (and it's hadoop dep) needs to work
> out of the box against 1.1 hbase servers while the hdfs before and after it
> is upgraded (either rolling or shutdown-restart upgrade)
> If both succeed despite dep changes, we are in in the clear. If it fails
> then we do not allow the dep change.
>

(hbase 1.0 client w/ jackson 1.8 + hadoop 2.4 )  go against (hbase 1.1
server w/ jackson 1.9 + hadoop 2.4)

(hbase 1.0 client w/ jackson 1.8 + hadoop 2.4 )  go against (hbase 1.1
server w/ jackson 1.9 + hadoop 2.6)

(hbase 1.0 client w/ jackson 1.8 + hadoop 2.6 )  go against (hbase 1.1
server w/ jackson 1.9 + hadoop 2.6)

(hbase 1.0 client w/ jackson 1.8 + hadoop 2.4 )  go against (mixed of hbase
1.0 w/ jackson 1.8 + hbase 1.1 server w/ Jackson 1.9 + hadoop 2.6)

Is this the coverage you meant?



> To avoid and simplify this problem on our internal builds, we have done the
> exercise of harmonizing all to one version of jackson/guava/etc or shading
> jars for the combination of hadoop and hbase (and other systems) that we be
> ship. I'd like to be in a world where this work is upstream.  At the
> moment, we don't know whether an upgrade to 2.6 hadoop including its
> Jackson 1.9 dep will have MR jobs fail or succeed because of classpath
> ordering issues.  It sounds like it is an may be an issue but we should
> test to make sure and decide based on that.
>
> Jon.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message