hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15638) Shade protobuf
Date Fri, 15 Apr 2016 05:10:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242448#comment-15242448

Sean Busbey commented on HBASE-15638:

bq. ... then turning that into a shaded module with relocated protobufs in place is pretty
straightforward and lets the source reference the original names.
Pardon me. Isn't that what this patch is doing?

both of the patches posted have references to the shaded protobuf classes in e.g. hbase-client.
That's the same fall out wether we do a dedicated artifact for the shaded protobuf version
or roll it into hbase-protocol.

if all references to our internal use of protobuf are in the hbase-protocol module, then we
needn't have any references to the relocated packages, because we can have the shade plugin
take care of rewriting them just in that module while including the relocated classes that
we use within the jar.

If we go the route of a module that relocated protobuf, we could make a profile in our project
/ top level pom "uses-relocated-protobuf". For those modules that activate it, we can then
use the shade plugin to rewrite the references from the original protobuf to the shaded one.
It would avoid moving classes into hbase-protocol and let us reference the original classes
in source, but we'd need to move anything that has to deal with HDFS' protobuf out of modules
that activate the profile.

> Shade protobuf
> --------------
>                 Key: HBASE-15638
>                 URL: https://issues.apache.org/jira/browse/HBASE-15638
>             Project: HBase
>          Issue Type: Bug
>          Components: Protobufs
>            Reporter: stack
>         Attachments: 15638v2.patch, as.far.as.server.patch
> Shade protobufs so we can move to a different version without breaking the world. We
want to get up on pb3 because it has unsafe methods that allow us save on copies; it also
has some means of dealing with BBs so we can pass it offheap DBBs. We'll probably want to
change PB3 to open it up some more too so we can stay offheap as we traverse PB. This issue
comes of [~anoop.hbase] and [~ram_krish]'s offheaping of the readpath work.
> This change is mostly straight-forward but there are some tricky bits:
>  # How to interface with HDFS? It wants its ByteStrings. Here in particular in FanOutOneBlockAsyncDFSOutputSaslHelper:
> {code}
>       if (payload != null) {
>         builder.setPayload(ByteString.copyFrom(payload));
>       }
> {code}
>  # [~busbey] also points out that we need to take care of endpoints done as pb. Test
at least.
> Let me raise this one on the dev list too.

This message was sent by Atlassian JIRA

View raw message