hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7233) Serializing KeyValues when passing them over RPC
Date Thu, 06 Dec 2012 08:08:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511216#comment-13511216
] 

Andrew Purtell commented on HBASE-7233:
---------------------------------------

Perhaps the title of this JIRA should be shortened to simply "Serializing KeyValues".

Using any of protobufs, Avro, or Thrift for marshalling/unmarshalling the KeyValue is unlikely
to be viable, lots of object creation churn, small copies, this will kill performance. However
sending a protobuf encoded prologue to a stream of KVs to a client makes sense.

I like the idea of KeyValue encoder.

I also like the idea of negotiating KeyValue encoder selection at connection setup time.

Beyond RPC, I've been looking at extending KeyValue to add tags as described in HBASE-6222.
What I have is a "transitional approach". No matter what else happens here, if KeyValue could
be a versioned serialization that would be great, we could introduce tags without overloading
existing fields in ugly ways (e.g. writing a negative value length to indicate the presence
of tags). Or, without storing tags physically distinct from their KVs in a separate shadow
column. I have implementations that do both, the latter has some undesirable cost as you might
imagine. Versioning KeyValue is tricky if we must be backwards compatible with existing data,
if migration does not involve a HFile rewrite step. How controversial is this?
                
> Serializing KeyValues when passing them over RPC
> ------------------------------------------------
>
>                 Key: HBASE-7233
>                 URL: https://issues.apache.org/jira/browse/HBASE-7233
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>         Attachments: 7233.txt, 7233-v2.txt
>
>
> Undo KeyValue being a Writable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message