hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: [VOTE] hbase-thirdparty project, first release candidate 1.0.0RC0
Date Sat, 01 Jul 2017 18:38:17 GMT
+1 from me.
St.Ack

On Fri, Jun 30, 2017 at 3:07 PM, Stack <stack@duboce.net> wrote:

> This is an interesting vote. The vote is on an RC made out of a new hbase
> project, hbase-thirdparty.
>
> First the usual stuff.
>
> The 1.0.0RC0 is available at:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-thirdparty/1.0.0RC0/
>
> Maven artifacts are in the following staging repository:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1171
>
> Artifacts are signed with 8ACC93D2 which is at the tail of our KEYS file
> http://www-us.apache.org/dist/hbase/KEYS.
>
> I tagged this RC as 1.0.0RC at 8ffaf3dd561052bcc71772148ecd04cdf9e224f3
> in the new hbase-thirdparty repository at [1].
>
> Now to the unusual.
>
> The artifact was made from a new repository, hbase-thirdparty[1].
> hbase-thirdparty is a project overseen by the hbase PMC. It came of
> discussion on the dev list in the thread '[DISCUSS] More Shading' [2].
> Its charge is the hosting of ornery thirdparty libs -- e.g. protobuf,
> guava, netty -- that need patching and/or shading so we are no longer bound
> by whatever the version of a lib Hadoop happens to ship (and we can change
> versions w/o damaging out glorious downstreamers). In the past, when we've
> been able, we've done this dirty patching and shading work as an
> unacknowledged 'pre-build' hack step appended to our protobuf file
> generation. Moving this mess out of our mainline build to live elsewhere,
> pulled in as a legit 'dependency', simplifies our build particularly around
> the shading of third-part libs; the narrower scope of a dedicated project
> that produces transformed jars is easier to reason about, is less likely to
> clash with existing dependencies whether transitive or otherwise, and it
> makes for less spaghetti poms.
>
> Other (minor) benefits include our being able to generate protobuf files
> inline now; no need of the pre-build anymore nor checking in of generated
> protobuf files (25MB).  There is a patch to remove them (predicated on this
> hbase-thirdparty release): "HBASE-17056 Remove checked in PB generated
> files".
>
> Downside is having to explicitly reference our version in code as in
> org.apache.hadoop.hbase.shaded.com.google.protobuf.Message rather than
> just com.google.protobuf.Message and so on for guava references, etc.
> (We've been doing this for a while now so hopefully it not too strange
> developing)*.
>
> There is little to this new project other than a few poms to do packaging
> and a couple of patches we apply on build. It works by pulling down
> dependencies and doing whatever shading, renaming, or patching we need.
> Currently hbase-thirdparty hosts protobuf (3.1.0 => 3.3.1), guava (12.0 =>
> 22.0), gson, protobuf-util, and netty 4.1.12-FINAL. It publishes three jars
> for hbase to consume: hbase-shaded-protobuf, hbase-shaded-netty, and then
> hbase-shaded-miscellaneous for the remaining libs. Philosophy is if the lib
> is troublesome, protobuf needs patching and netty includes an .so that
> needs a rename, then give it its own module (See README in the release for
> more). I've arrived at this layout for hbase-thirdparty after a bunch of
> experimentation and messing with hbase dependencies (See sub-tasks under
> HBASE-17898 Update dependencies). That said, as we head to hbase-2.0.0, we
> might have to make a an odd new release of hbase-thirdparty but thereafter
> I'd imagine it should settle down to be an infrequent update.
>
> VOTE lasts 72 hours. Let +1 be an assent to the hbase-thirdparty project
> and a vote for this RC.
>
> Questions on how it will go w/ this project welcome on this thread. If
> heavy-detail on how the shading works, I suggest ask on the '[DISCUSS] More
> Shading' thread.
>
> Thanks,
> St.Ack
> 1. https://git-wip-us.apache.org/repos/asf/hbase-thirdparty
> 2. See the '[DISCUSS] More Shading' thread, http://apache-hbase.679495.n3.
> nabble.com/DISCUSS-More-Shading-td4083025.html
>
> * Doc on how to make use of hbase-thirdparty is coming for the refguide.
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message