hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张铎 <palomino...@gmail.com>
Subject Re: DISCUSS: Protobufs?
Date Tue, 02 Feb 2016 06:37:55 GMT
https://groups.google.com/forum/#!topic/protobuf/wAqvtPLBsE8

PB2 and PB3 are wire compatible, and of course, protobuf-java is not
compatible so dependency will be a problem... But I think the shaded client
and server can solve the problem?

Thanks.

2016-02-02 14:27 GMT+08:00 Stack <stack@duboce.net>:

> We are running into a few issues with protobufs.
>
> + PB always copies all data before making a Message. This generates garbage
> unnecessarily.
> + CodedInputStream does not support ByteBuffers in 2.5. In 2.6 it does but
> again, copies the data out of the BB always; this is especially painful
> when the BB is a DBB with its data offheap and intent is to keep data
> offheap.
>
> There are other issues. CIS allocates 4k buffers regardless (See
> HBASE-15177).
> And then there was the HBaseZeroCopyByteString fun and games we had a while
> back.
>
> 3.0 PB adds UnsafeByteStrings so can do zero copy. Thats good. But PB3 is
> incompatible with PB2 (or at least, it looks like PB2 clients can't talk to
> PB3 [1]).
>
> There is javanano protobufs. All is open access, but it too looks different
> to PB2 (i've not tried it).
>
> Protostuff seems really quiet these times [2].
>
> Fork (and shade)?
>
> Thoughts?
>
> St.Ack
>
> 1. https://github.com/google/protobuf/releases
> 2. https://groups.google.com/forum/#!forum/protostuff
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message