Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
Sender: saint.ack@gmail.com
From: Stack <stack@duboce.net>
Date: Sat, 1 Oct 2016 13:10:09 -0700
Message-ID: <CADcMMgFsODj_bkR5fgfKULf8HinYR_qvbUt2_8RO40anDJpL4Q@mail.gmail.com>
Subject: [DISCUSS] More Shading
To: HBase Dev List <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a1143d2f03b6399053dd34c39
archived-at: Sat, 01 Oct 2016 20:10:15 -0000

--001a1143d2f03b6399053dd34c39
Content-Type: text/plain; charset=UTF-8

HBASE-15638 is about shading protobufs. Lets shade other critical libs too
so we can run with versions of the libraries we favor rather than versions
dictated by dependencies.

For example, our guava is from the stone ages. Guava is a quality library
that we should be making use of throughout our code base. We are afraid to
update it because it will break when we share our CLASSPATH with another
component or a dependency of ours transitively includes a conflicting
version. Worse, there have been incidents where we undo Guava usage because
of CLASSPATH clashes (Running a recent HBase with a recent version of Drill
broke on Guava StopWatch import...).

That we should shade critical, popular, core libs seems self-evident
(though I would be interested if folks have other opinions). What I want to
discuss though is how we go about it.

The HBASE-15638 (protobuf shading) approach has us reference the relocated
artifact explicitly. This makes for an ugly ripple across the codebase as
we declare which protobuf Message is intended; either
com.google.protobuf.Message or
org.apache.hadoop.hbase.shaded.com.google.protobuf.Message. It is a pain
making all the changes but the intent is clear. I was thinking we'd do
similar for guava and whatever else we think fits this category. I'd make a
hbase-3rdparty-shaded or some such module and do all hackery therein.

Where it gets awkward is whether or not we check in the shaded artifact
source code (Over in HBASE-15638, we have checked in the relocated
protobuf3 source code because we are going to patch it, for a while at
least). For the build and runtime to work, we do not need the relocated
source code to be present but not having source code present is a hurdle
for devs who use IDEs (Everyone but Sean and Matteo). Their code will be
flagged w/ red flags saying the relocated artifact is missing/unresolvable.
To 'fix', they need build the shaded module and then in their IDE, drop the
shaded module and add the built shaded module jar to the IDE's build-time
CLASSPATH.

This is awkward. Is this too much to ask of devs, especially those getting
going for the first time? I could do up doc and IDE configs to help but
this would be an added hurdle getting setup.

Sean has suggested a pre-build step where in another repo we'd make hbase
shaded versions of critical libs, 'release' them (votes, etc.) and then
have core depend on these. It be a bunch of work but would make the dev's
life easier.

Interested in any thoughts you lot might have.

Thanks,
St.Ack

--001a1143d2f03b6399053dd34c39--