hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: [DISCUSS] More Shading
Date Wed, 12 Apr 2017 21:56:31 GMT
On Wed, Apr 12, 2017 at 8:22 AM, Josh Elser <elserj@apache.org> wrote:

> This makes me wonder if we could construct source jars just the same as
> we're creating shaded jars. Google has lead me to [2][3], but I've never
> tried either. The latter option seems to be acknowledging that the source
> might not actually compile, but the package names would at least be correct.
> I think this would be a good early experiment which, if it does work out,
> removes the only acknowledged "hole" in the current plan.
Mighty Josh.

As it happens, the 'createSources' you cite is currently in use by our
hbase-protocol-shaded module for the specific purpose of keeping our IDE'rs
happy; we relocate protobuf itself so we can have pb3.2 and pb2.5 on our
CLASSPATH but IDEs don't complain because the src for the relocated pb is
checked-in to our codebase.

Here's more if interested:

All modules that make use of protos by convention require a 'pre-build'
step. Generally, the pre-build is required if you add or mod .proto files.
At pre-build we generate class files from the amended .protos and then
check-in the product. See the README in each of our proto-carrying modules.

In the hbase-protocol-shaded case, we abuse the pre-build notion so we can
house an internal pb version, one that does not agree w/ hadoops nor w/
that used by our CPs describing Endpoint services.  We do as follows (the
order may not be exact in below):

 * Generate class files from protos
 * Shade the built artifact (with the createSource flag set).
 * Unjar the artifact which has generated proto and protobuf src classes in
 * Apply a few patches to protobuf to support offheap work (stuff we need
to push back up to pb)
 * Overlay our current protobuf src w/ the new version in the src tree -- a
big no-no (smile).

You then check it all in....

It has been suggested we undo these hokey pre-build steps and just generate
classes from protobuf inline w/ the main build. A plugin we just figured
makes this possible since it provides the platform-appropriate protoc
(org.xolstice.maven.plugins). We could do the simplification currently for
all but the hbase-protocol-shaded module because of the above jujitsu. One
thought is that if we had a pre-build artifact as is being suggested in
this thread, I could move the patched pb there and purge the pre-build step


> We (The HBase PMC) will have to make releases of this new artifact and vote
>> on them. I think it will be a relatively rare event.
>> I'd be up for doing the first cut if folks are game.
>> St.Ack
>> 1. URL via Sean but for committers to view only:
>> https://reporeq.apache.org/
> [2] https://maven.apache.org/plugins/maven-shade-plugin/shade-
> mojo.html#createSourcesJar
> [3] https://maven.apache.org/plugins/maven-shade-plugin/shade-
> mojo.html#shadeSourcesContent

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message