hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: [DISCUSS] More Shading
Date Fri, 30 Jun 2017 22:09:17 GMT
I just started a VOTE on hbase-thirdparty and the first RC made from it.
Thanks,
St.Ack

On Tue, Jun 27, 2017 at 3:02 PM, Stack <stack@duboce.net> wrote:

> Bit of an update.
>
> I'd suggest we go ahead w/ the hbase-thirdparty project [2]. It took a
> while but in its current form -- a few poms that package a few jars [1]--
> it at least enables the below:
>
> + Allows us to skip checking in protobuf generated files (25MB!); they can
> be generated inline w/ the build because the hackery patching protobuf has
> been moved out to hbase-thirdparty. There is a patch up on HBASE-17056.
> + Update our guava from 12.0 to 22.0 w/o clashing w/ the guava of others.
> There is a patch at HBASE-17908. It is taking a bit of wrangling getting it
> to land because I pared back transitive includes from hadoop and it takes a
> while to work through the failures.
>
> Other benefits are the protobuf-util lib is on the classpath now -- its in
> hbase-thirdparty relocated; depends on pb and guava -- so we have facility
> to goat "HBASE-18106 Redo ProcedureInfo and LockInfo" and shading netty is
> almost done so we can do with netty as we wilt independent of hadoop and
> downstreamers (the hard part -- relocation of the .so -- should be done).
>
> Let me figure how to run a vote for a couple of poms.....
>
> St.Ack
>
> 1. https://repository.apache.org/content/groups/snapshots/
> org/apache/hbase/thirdparty/ (see hbase-shaded-thirdparty and
> hbase-shaded-protobuf)
> 2. https://git-wip-us.apache.org/repos/asf/hbase-thirdparty
>
>
> On Tue, Jun 20, 2017 at 11:04 AM, Josh Elser <josh.elser@gmail.com> wrote:
>
>> On 6/20/17 1:28 AM, Stack wrote:
>>
>>> On Thu, Apr 13, 2017 at 4:46 PM, Josh Elser<elserj@apache.org>  wrote:
>>>
>>> ...
>>>>
>>>> I think pushing this part forward with some code is the next logical
>>>> step.
>>>> Seems to be consensus about taking our known internal dependencies and
>>>> performing this shade magic.
>>>>
>>>>
>>>> I opened HBASE-18240 "Add hbase-auxillary, a project with hbase utility
>>> including an hbase-shaded-thirdparty module with guava, netty, etc."
>>>
>>> It has a tarball attached that bundles the outline of an hbase-auxillary
>>> project (groupId:org.apache.hbase.auxillary). This project is intended
>>> to
>>> be standalone, in its own repository, publishing its own artifacts under
>>> the aegis of this project's PMC.
>>>
>>> It includes the first instance of an auxillary utility, a module named
>>> hbase-thirdparty-shaded (artifactId:hbase-thirdparty-shaded). Herein
>>> we'll
>>> pull down 3rd party libs and republish at an offset; e.g.
>>> com.google.common.* from guava will be at
>>> org.apache.hbase.thirdparty.shaded.com.google.common.*. Currently it
>>> builds
>>> a jar that includes a relocated guava 22.0.
>>>
>>> I then messed around making hbase-common use it (You have to build the
>>> hbase-auxillary into your local repo). I put up a patch on the issue.
>>> Mostly its mass find-and-replace w/ some clean up of transitive includes
>>> of
>>> guava from hadoop-common and some small fixup of methods renamed between
>>> guava 12.0 and 22.0.
>>>
>>> Unless objection, I was going to press on. Sean offered to help set up
>>> new
>>> repo. We can always undo and delete it if this project fails.
>>>
>>> When done, the hope is we are on a modern version of guava and our netty
>>> and protobuf 3 will be be relocated, 'hidden' from downstream (and won't
>>> clash w/ upstream). I hope to also purge the pre-build we have in our
>>> modules that do protobuf moving this hackery out and under
>>> hbase-thirdparty-shaded.
>>>
>>> St.Ack
>>>
>>
>> Kudos on the JFDI approach :). I think having something concrete to show
>> is the best way to judge success of it.
>>
>> Will keep an eye on HBASE-18240.
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message