hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject DISCUSS: Protobufs?
Date Tue, 02 Feb 2016 06:27:33 GMT
We are running into a few issues with protobufs.

+ PB always copies all data before making a Message. This generates garbage
unnecessarily.
+ CodedInputStream does not support ByteBuffers in 2.5. In 2.6 it does but
again, copies the data out of the BB always; this is especially painful
when the BB is a DBB with its data offheap and intent is to keep data
offheap.

There are other issues. CIS allocates 4k buffers regardless (See HBASE-15177).
And then there was the HBaseZeroCopyByteString fun and games we had a while
back.

3.0 PB adds UnsafeByteStrings so can do zero copy. Thats good. But PB3 is
incompatible with PB2 (or at least, it looks like PB2 clients can't talk to
PB3 [1]).

There is javanano protobufs. All is open access, but it too looks different
to PB2 (i've not tried it).

Protostuff seems really quiet these times [2].

Fork (and shade)?

Thoughts?

St.Ack

1. https://github.com/google/protobuf/releases
2. https://groups.google.com/forum/#!forum/protostuff

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message