hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-794) Language neutral IPC as a first class component of HBase architecture
Date Tue, 05 May 2009 17:19:30 GMT

    [ https://issues.apache.org/jira/browse/HBASE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706112#action_12706112

Andrew Purtell commented on HBASE-794:

@Bryan: Thanks. If I understand correctly, we'd like to hold on to the byte buffers into which
frames were read from the wire, and build some very lightweight structures which index individual
KeyValues contained within. We'd also want to avoid any unmarshalling overhead as the data
type is plain byte[]. This is like exposing some middle layer of the Thrift stack. (A data
stream.) I think it would also be useful if structured data could be mixed inline. (A control
stream.) So maybe we'd have some convenient structured metadata at the start of a transaction
which can be processed using functions at the top of the Thrift stack, followed by a lot of
instances of a KeyValue type, which we could drop down to slice up the backing byte buffer
for zero copy from there. I don't know how feasible this is, just thinking out loud here.

> Language neutral IPC as a first class component of HBase architecture
> ---------------------------------------------------------------------
>                 Key: HBASE-794
>                 URL: https://issues.apache.org/jira/browse/HBASE-794
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: client, ipc, master, regionserver
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
> This issue considers making a language neutral IPC mechanism and wire format a first
class component of HBase architecture. Clients could talk to the master and regionserver using
this protocol instead of HRPC at their option.
> Options for language neutral IPC include:
> * Thrift: http://incubator.apache.org/thrift/
> * Protocol buffers: http://code.google.com/p/protobuf/
> * XDR: http://en.wikipedia.org/wiki/External_Data_Representation

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message