hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Helmling (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15638) Shade protobuf
Date Wed, 20 Jul 2016 23:45:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386820#comment-15386820
] 

Gary Helmling commented on HBASE-15638:
---------------------------------------

[~saint.ack@gmail.com], I'm not sure how to make this work with coprocessor endpoints.

For endpoints, the user will have:
# Created a .proto file with the service definition for their endpoint
# Run their own version of protoc to do the code gen for the service
# The generated PB messages for request/response, and, as you note, the RpcController and
RpcCallback in the service method signatures will be in the com.google.protobuf package
# The endpoint jar is loaded in HBase
# The ServerRpcController we pass through for the endpoint invocation will extend our shaded
RpcController

So the only way I see this working is if we force endpoint implementors to run a preprocessor
step to shade the endpoint jar prior to step #4.  Maybe that would work, but it seems pretty
cumbersome.  Or else we somehow avoid shading ServerRpcController.

> Shade protobuf
> --------------
>
>                 Key: HBASE-15638
>                 URL: https://issues.apache.org/jira/browse/HBASE-15638
>             Project: HBase
>          Issue Type: Bug
>          Components: Protobufs
>            Reporter: stack
>            Priority: Critical
>         Attachments: 15638v2.patch, as.far.as.server.patch
>
>
> Shade protobufs so we can move to a different version without breaking the world. We
want to get up on pb3 because it has unsafe methods that allow us save on copies; it also
has some means of dealing with BBs so we can pass it offheap DBBs. We'll probably want to
change PB3 to open it up some more too so we can stay offheap as we traverse PB. This issue
comes of [~anoop.hbase] and [~ram_krish]'s offheaping of the readpath work.
> This change is mostly straight-forward but there are some tricky bits:
>  # How to interface with HDFS? It wants its ByteStrings. Here in particular in FanOutOneBlockAsyncDFSOutputSaslHelper:
> {code}
>       if (payload != null) {
>         builder.setPayload(ByteString.copyFrom(payload));
>       }
> {code}
>  # [~busbey] also points out that we need to take care of endpoints done as pb. Test
at least.
> Let me raise this one on the dev list too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message