hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject DISCUSS: Protobufs?
Date Tue, 02 Feb 2016 06:27:33 GMT
We are running into a few issues with protobufs.

+ PB always copies all data before making a Message. This generates garbage
+ CodedInputStream does not support ByteBuffers in 2.5. In 2.6 it does but
again, copies the data out of the BB always; this is especially painful
when the BB is a DBB with its data offheap and intent is to keep data

There are other issues. CIS allocates 4k buffers regardless (See HBASE-15177).
And then there was the HBaseZeroCopyByteString fun and games we had a while

3.0 PB adds UnsafeByteStrings so can do zero copy. Thats good. But PB3 is
incompatible with PB2 (or at least, it looks like PB2 clients can't talk to
PB3 [1]).

There is javanano protobufs. All is open access, but it too looks different
to PB2 (i've not tried it).

Protostuff seems really quiet these times [2].

Fork (and shade)?



1. https://github.com/google/protobuf/releases
2. https://groups.google.com/forum/#!forum/protostuff

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message