hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-794) Language neutral IPC as a first class component of HBase architecture
Date Sun, 03 May 2009 07:22:30 GMT

    [ https://issues.apache.org/jira/browse/HBASE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705390#action_12705390

ryan rawson commented on HBASE-794:

As for the API of HBase, there are a number of special cases:

- get()/scanner()/next() return null values all the time, or 0 length arrays (in the case
of next()).
- nearly all the data is simple, byte[]. Big wins by not copying the same bytes around a dozen
- Our internal 0-copy systems are called KeyValue, and allow many many values to share the
same underlying bytes (reading from a hfile) or reducing the number of copies from the RPC
-> IO with HDFS.

The actual implementation of a new RPC is a 0.21 task, so we are still just evaluating things.

One thing we will not do in the future is serializing trees to the RPC and back again.  Our
new API may be mostly arrays of KeyValues.  Since a client has to be smart, there is no problems
with pushing detailed knowledge of how we store columns/values to the client.

I'm still not sure how much gain we are going to get by pooling/reusing objects in Java...
Outsmarting the GC tends to bite you hard.

> Language neutral IPC as a first class component of HBase architecture
> ---------------------------------------------------------------------
>                 Key: HBASE-794
>                 URL: https://issues.apache.org/jira/browse/HBASE-794
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: client, ipc, master, regionserver
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
> This issue considers making a language neutral IPC mechanism and wire format a first
class component of HBase architecture. Clients could talk to the master and regionserver using
this protocol instead of HRPC at their option.
> Options for language neutral IPC include:
> * Thrift: http://incubator.apache.org/thrift/
> * Protocol buffers: http://code.google.com/p/protobuf/
> * XDR: http://en.wikipedia.org/wiki/External_Data_Representation

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message