hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: modular build and pluggable rpc
Date Tue, 31 May 2011 21:27:00 GMT
+1 on Server and Client sub-modules.

Jackson 1.7 supports BSON.  Jersey 1.6 (latest) is including Jackson 1.5.5 but can be swap
out with Jackson 1.7 with maven tricks or wait for Jersey to update to the latest jackson.

Regards,
Eric

On 5/31/11 1:22 PM, "Stack" <stack@duboce.net> wrote:

On Mon, May 30, 2011 at 9:55 PM, Eric Yang <eyang@yahoo-inc.com> wrote:
> Maven modulation could be enhanced to have a structure looks like this:
>
> Super POM
>  +- common
>  +- shell
>  +- master
>  +- region-server
>  +- coprocessor
>
> The software is basically group by processor type (role of the process) and a shared
library.
>

I'd change the list above.  shell should be client and perhaps master
and regionserver should be both inside a single 'server' submodule.
We need to add security in there.  Perhaps we'd have a submodule for
thrift, avro, rest (and perhaps rest war file)?  (Is this too many
submodules  -- I suppose once we are submodularized, adding new ones
is trivial.  Its the initial move to submodules that is painful)


> For RPC, there are several feasible options, avro, thrift and jackson+jersey (REST).
 Avro may seems cumbersome to define the schema in JSON string.  Thrift comes with it's own
rpc server, it is not trivial to add authorization and authentication to secure the rpc transport.
 Jackson+Jersey RPC message is biggest message size compare to Avro and thrift.  All three
frameworks have pros and cons but I think Jackson+jersey have the right balance for rpc framework.
 In most of the use case, pluggable RPC can be narrow down to two main category of use cases:
>
> 1. Freedom of creating most efficient rpc but hard to integrate with everything else
because it's custom made.
> 2. Being able to evolve message passing and versioning.
>
> If we can see beyond first reason, and realize second reason is in part polymorphic serialization.
 This means, Jackson+Jersey is probably the better choice as a RPC framework because Jackson
supports polymorphic serialization, and Jersey builds on HTTP protocol.  It would be easier
to versioning and add security on top of existing standards.  The syntax and feature set seems
more engineering proper to me.
>

I always considered http attactive but much too heavy-weight for hbase
rpc; each request/response would carry a bunch of what are for the
most part extraneous headers.  I suppose we should just measure.
Regards JSON messages, thats interesting but hbase is all about binary
data.  Does jackson/jersey do BSON?

St.Ack


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message