Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: <CADcMMgHO0fYKU4g83gT7yyiPGvgAKXjqJbw8MFictM=o9mVq3w@mail.gmail.com>
References: <CADcMMgFsODj_bkR5fgfKULf8HinYR_qvbUt2_8RO40anDJpL4Q@mail.gmail.com>
 <35A15B2D-07EF-445A-9CA8-11BF47C72871@gmail.com> <CAHkeaHUhvp7izJQRBNs10P1B+pgXCp4oRXJL3eLEUqk_hbO-Wg@mail.gmail.com>
 <CADcMMgFX06Y_VNQQN=qtyuANORQfjN51OTLnw0-we1yTVve3OQ@mail.gmail.com>
 <CAAT7MkpiuPqYHY8ziFxqAuCzr3X_DgDERPVOowcw3kYLCGivJw@mail.gmail.com>
 <CADcMMgF5P+M7xfC-QU2Em4T-U2aF-+AJnvd6Wd5J4pzJZBNFHw@mail.gmail.com>
 <58EE462D.1070206@apache.org> <CADcMMgG=Ney8fe9Oy6cYpZEbJUb3ubeWcfketQSPxKcLCH+qjg@mail.gmail.com>
 <58F00DD3.6070105@apache.org> <CADcMMgGyY-Win05tvB97hJcX1eGJSbLSV4-uFYR63yvXWtgk3w@mail.gmail.com>
 <3d02162c-6f38-6f26-67f2-ab44f58993c5@gmail.com> <CADcMMgHO0fYKU4g83gT7yyiPGvgAKXjqJbw8MFictM=o9mVq3w@mail.gmail.com>
From: Stack <stack@duboce.net>
Date: Fri, 30 Jun 2017 15:09:17 -0700
Message-ID: <CADcMMgF3WUiz+fObsXQgGAPxTi968w861eRtGqcCKmY9HcnD-g@mail.gmail.com>
Subject: Re: [DISCUSS] More Shading
To: HBase Dev List <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary="94eb2c186ba425c4c3055334ab97"
archived-at: Fri, 30 Jun 2017 22:09:23 -0000

--94eb2c186ba425c4c3055334ab97
Content-Type: text/plain; charset="UTF-8"

I just started a VOTE on hbase-thirdparty and the first RC made from it.
Thanks,
St.Ack

On Tue, Jun 27, 2017 at 3:02 PM, Stack <stack@duboce.net> wrote:

> Bit of an update.
>
> I'd suggest we go ahead w/ the hbase-thirdparty project [2]. It took a
> while but in its current form -- a few poms that package a few jars [1]--
> it at least enables the below:
>
> + Allows us to skip checking in protobuf generated files (25MB!); they can
> be generated inline w/ the build because the hackery patching protobuf has
> been moved out to hbase-thirdparty. There is a patch up on HBASE-17056.
> + Update our guava from 12.0 to 22.0 w/o clashing w/ the guava of others.
> There is a patch at HBASE-17908. It is taking a bit of wrangling getting it
> to land because I pared back transitive includes from hadoop and it takes a
> while to work through the failures.
>
> Other benefits are the protobuf-util lib is on the classpath now -- its in
> hbase-thirdparty relocated; depends on pb and guava -- so we have facility
> to goat "HBASE-18106 Redo ProcedureInfo and LockInfo" and shading netty is
> almost done so we can do with netty as we wilt independent of hadoop and
> downstreamers (the hard part -- relocation of the .so -- should be done).
>
> Let me figure how to run a vote for a couple of poms.....
>
> St.Ack
>
> 1. https://repository.apache.org/content/groups/snapshots/
> org/apache/hbase/thirdparty/ (see hbase-shaded-thirdparty and
> hbase-shaded-protobuf)
> 2. https://git-wip-us.apache.org/repos/asf/hbase-thirdparty
>
>
> On Tue, Jun 20, 2017 at 11:04 AM, Josh Elser <josh.elser@gmail.com> wrote:
>
>> On 6/20/17 1:28 AM, Stack wrote:
>>
>>> On Thu, Apr 13, 2017 at 4:46 PM, Josh Elser<elserj@apache.org>  wrote:
>>>
>>> ...
>>>>
>>>> I think pushing this part forward with some code is the next logical
>>>> step.
>>>> Seems to be consensus about taking our known internal dependencies and
>>>> performing this shade magic.
>>>>
>>>>
>>>> I opened HBASE-18240 "Add hbase-auxillary, a project with hbase utility
>>> including an hbase-shaded-thirdparty module with guava, netty, etc."
>>>
>>> It has a tarball attached that bundles the outline of an hbase-auxillary
>>> project (groupId:org.apache.hbase.auxillary). This project is intended
>>> to
>>> be standalone, in its own repository, publishing its own artifacts under
>>> the aegis of this project's PMC.
>>>
>>> It includes the first instance of an auxillary utility, a module named
>>> hbase-thirdparty-shaded (artifactId:hbase-thirdparty-shaded). Herein
>>> we'll
>>> pull down 3rd party libs and republish at an offset; e.g.
>>> com.google.common.* from guava will be at
>>> org.apache.hbase.thirdparty.shaded.com.google.common.*. Currently it
>>> builds
>>> a jar that includes a relocated guava 22.0.
>>>
>>> I then messed around making hbase-common use it (You have to build the
>>> hbase-auxillary into your local repo). I put up a patch on the issue.
>>> Mostly its mass find-and-replace w/ some clean up of transitive includes
>>> of
>>> guava from hadoop-common and some small fixup of methods renamed between
>>> guava 12.0 and 22.0.
>>>
>>> Unless objection, I was going to press on. Sean offered to help set up
>>> new
>>> repo. We can always undo and delete it if this project fails.
>>>
>>> When done, the hope is we are on a modern version of guava and our netty
>>> and protobuf 3 will be be relocated, 'hidden' from downstream (and won't
>>> clash w/ upstream). I hope to also purge the pre-build we have in our
>>> modules that do protobuf moving this hackery out and under
>>> hbase-thirdparty-shaded.
>>>
>>> St.Ack
>>>
>>
>> Kudos on the JFDI approach :). I think having something concrete to show
>> is the best way to judge success of it.
>>
>> Will keep an eye on HBASE-18240.
>>
>>
>

--94eb2c186ba425c4c3055334ab97--