hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15638) Shade protobuf
Date Wed, 05 Oct 2016 06:11:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547800#comment-15547800

Anoop Sam John commented on HBASE-15638:

Yes.  For Admin public APIs what we did is we deprecated old PB object methods APIs in branch-1
and removed in trunk.  Ya the deprecation happened in the mid of 1.x line only.  The argument
to relax the compatibility guarantee was that Admin is marked evolving.
public interface Admin extends Abortable, Closeable 

Am not sure for CP how we can do now.. Same way of deprecation model as done for Admin or
deprecate in 2.0 and remove in 3.0?

Ping [~enis]

> Shade protobuf
> --------------
>                 Key: HBASE-15638
>                 URL: https://issues.apache.org/jira/browse/HBASE-15638
>             Project: HBase
>          Issue Type: Bug
>          Components: Protobufs
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>             Fix For: 2.0.0
>         Attachments: 15638v2.patch, HBASE-15638.master.001.patch, HBASE-15638.master.002.patch,
HBASE-15638.master.003 (1).patch, HBASE-15638.master.003 (1).patch, HBASE-15638.master.003
(1).patch, HBASE-15638.master.003.patch, HBASE-15638.master.003.patch, HBASE-15638.master.004.patch,
HBASE-15638.master.005.patch, HBASE-15638.master.006.patch, HBASE-15638.master.007.patch,
HBASE-15638.master.007.patch, HBASE-15638.master.008.patch, HBASE-15638.master.009.patch,
> We need to change our protobuf. Currently it is pb2.5.0. As is, protobufs expect all
buffers to be on-heap byte arrays. It does not have facility for dealing in ByteBuffers and
off-heap ByteBuffers in particular. This fact frustrates the off-heaping-of-the-write-path
project as marshalling/unmarshalling of protobufs involves a copy on-heap first.
> So, we need to patch our protobuf so it supports off-heap ByteBuffers. To ensure we pick
up the patched protobuf always, we need to relocate/shade our protobuf and adjust all protobuf
references accordingly.
> Given as we have protobufs in our public facing API, Coprocessor Endpoints -- which use
protobuf Service to describe new API -- a blind relocation/shading of com.google.protobuf.*
will break our API for CoProcessor EndPoints (CPEP) in particular. For example, in the Table
Interface, to invoke a method on a registered CPEP, we have:
> {code}<T extends com.google.protobuf.Service,R> Map<byte[],R> coprocessorService(
> Class<T> service, byte[] startKey, byte[] endKey,                             
               org.apache.hadoop.hbase.client.coprocessor.Batch.Call<T,R> callable)
> throws com.google.protobuf.ServiceException, Throwable{code}
> This issue is how we intend to shade protobuf for hbase-2.0.0 while preserving our API
as is so CPEPs continue to work on the new hbase.

This message was sent by Atlassian JIRA

View raw message