hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject [VOTE] hbase-thirdparty project, first release candidate 1.0.0RC0
Date Fri, 30 Jun 2017 22:07:00 GMT
This is an interesting vote. The vote is on an RC made out of a new hbase
project, hbase-thirdparty.

First the usual stuff.

The 1.0.0RC0 is available at:

 https://dist.apache.org/repos/dist/dev/hbase/hbase-thirdparty/1.0.0RC0/

Maven artifacts are in the following staging repository:

 https://repository.apache.org/content/repositories/orgapachehbase-1171

Artifacts are signed with 8ACC93D2 which is at the tail of our KEYS file
http://www-us.apache.org/dist/hbase/KEYS.

I tagged this RC as 1.0.0RC at 8ffaf3dd561052bcc71772148ecd04cdf9e224f3 in
the new hbase-thirdparty repository at [1].

Now to the unusual.

The artifact was made from a new repository, hbase-thirdparty[1].
hbase-thirdparty is a project overseen by the hbase PMC. It came of
discussion on the dev list in the thread '[DISCUSS] More Shading' [2]. Its
charge is the hosting of ornery thirdparty libs -- e.g. protobuf, guava,
netty -- that need patching and/or shading so we are no longer bound by
whatever the version of a lib Hadoop happens to ship (and we can change
versions w/o damaging out glorious downstreamers). In the past, when we've
been able, we've done this dirty patching and shading work as an
unacknowledged 'pre-build' hack step appended to our protobuf file
generation. Moving this mess out of our mainline build to live elsewhere,
pulled in as a legit 'dependency', simplifies our build particularly around
the shading of third-part libs; the narrower scope of a dedicated project
that produces transformed jars is easier to reason about, is less likely to
clash with existing dependencies whether transitive or otherwise, and it
makes for less spaghetti poms.

Other (minor) benefits include our being able to generate protobuf files
inline now; no need of the pre-build anymore nor checking in of generated
protobuf files (25MB).  There is a patch to remove them (predicated on this
hbase-thirdparty release): "HBASE-17056 Remove checked in PB generated
files".

Downside is having to explicitly reference our version in code as in
org.apache.hadoop.hbase.shaded.com.google.protobuf.Message rather than just
com.google.protobuf.Message and so on for guava references, etc. (We've
been doing this for a while now so hopefully it not too strange
developing)*.

There is little to this new project other than a few poms to do packaging
and a couple of patches we apply on build. It works by pulling down
dependencies and doing whatever shading, renaming, or patching we need.
Currently hbase-thirdparty hosts protobuf (3.1.0 => 3.3.1), guava (12.0 =>
22.0), gson, protobuf-util, and netty 4.1.12-FINAL. It publishes three jars
for hbase to consume: hbase-shaded-protobuf, hbase-shaded-netty, and then
hbase-shaded-miscellaneous for the remaining libs. Philosophy is if the lib
is troublesome, protobuf needs patching and netty includes an .so that
needs a rename, then give it its own module (See README in the release for
more). I've arrived at this layout for hbase-thirdparty after a bunch of
experimentation and messing with hbase dependencies (See sub-tasks under
HBASE-17898 Update dependencies). That said, as we head to hbase-2.0.0, we
might have to make a an odd new release of hbase-thirdparty but thereafter
I'd imagine it should settle down to be an infrequent update.

VOTE lasts 72 hours. Let +1 be an assent to the hbase-thirdparty project
and a vote for this RC.

Questions on how it will go w/ this project welcome on this thread. If
heavy-detail on how the shading works, I suggest ask on the '[DISCUSS] More
Shading' thread.

Thanks,
St.Ack
1. https://git-wip-us.apache.org/repos/asf/hbase-thirdparty
2. See the '[DISCUSS] More Shading' thread,
http://apache-hbase.679495.n3.nabble.com/DISCUSS-More-Shading-td4083025.html

* Doc on how to make use of hbase-thirdparty is coming for the refguide.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message