Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CADcMMgFUADeSP4PFUmfpn4UzQUEQKr+FpSdzGqXBMi6+zomADQ@mail.gmail.com>
References: <CAHkeaHXBR1ThioFaBhLtWq10x-zyGevMhtoMrF5jx0bOmW6Abg@mail.gmail.com>
 <CADcMMgFUADeSP4PFUmfpn4UzQUEQKr+FpSdzGqXBMi6+zomADQ@mail.gmail.com>
From: Sean Busbey <busbey@apache.org>
Date: Thu, 2 Nov 2017 15:54:15 -0500
Message-ID: <CAN5cbe4BHmmC6rL5GBEruzr=MSpCY5W5-wsPZ7Ns0nbFBkWPNQ@mail.gmail.com>
Subject: Re: Problem with double shading in hbase-thirdparty and hbase-shaded-client?
To: dev <dev@hbase.apache.org>
Content-Type: text/plain; charset="UTF-8"
archived-at: Thu, 02 Nov 2017 20:54:41 -0000

I think something like o.a.h.hbase.thirdparty might be better. then we
can differentiate between "shaded" for downstream of us vs
"thirdparty" for our internal use.

On Thu, Nov 2, 2017 at 3:52 PM, Stack <stack@duboce.net> wrote:
> Good one Jerry He.
>
> I can try changing the offset in hbase-thirdparty in a new release
> (o.a.hbase.shaded?). There used to be a reason why it had to be
> o.a.h.hbase.shaded but IIRC it went away after all shaded dependencies made
> it out to hbase-thirdparty (including the pb3 that we used to have checked
> into core).
>
> S
>
> On Thu, Nov 2, 2017 at 11:45 AM, Jerry He <jerryjch@gmail.com> wrote:
>
>> Mike from another thread asked about shading jackson into
>> hbase-thirdparty. That reminded me of a question I was thinking
>> recently,
>> in conjunction with HBASE-18800 "[Propaganda] Push shaded hbase-client
>> as gateway to an hbase cluster going forward".
>>
>> We have shaded artifacts in hbase-thirdparty
>> (hbase-shaded-miscellaneous, hbase-shaded-protobuf and
>> hbase-shaded-netty). The purpose is to shade and bundle the jars with
>> versions hbase uses internally.
>>
>> Then in hbase-shaded-client we do shading again for some of the same
>> jars, but they are more non-hbase, transitive dependencies.
>> For example, in hbase-shaded-miscellaneous, we shade guava 22. But
>> hbase-client will bring in guava 11 from hadoop-common. In
>> hbase-shaded-protobuf, we shade protobuf 3. But hadoop brings in
>> protobuf 2.5.
>> We will be doing shading of all these in hbase-shaded-client (and the
>> hbase-shaded-mapreduce), with the same relocation
>> (org.apache.hadoop.hbase.shaded).
>>
>> The end result is shaded guava 22 or guava 11?  Protobuf 3 or protobuf 2.5?
>> I checked hbase-shaded-client and found it had the shaded guava 22 classes.
>> Looks like the shading process will only add one version of the
>> classes with the same name.
>>
>> This will potentially have problem, and potentially make it unusable
>> in unlucky situations.
>>
>> I bring it up here for awareness and discussion. First, is this really
>> a problem?
>> Second, how to fix it?  Maybe a different relocation offset in
>> hbase-thirdparty?
>>
>> Thanks,
>>
>> Jerry
>>