hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-7905) Add passing of optional cell blocks over rpc
Date Fri, 22 Feb 2013 06:36:12 GMT

     [ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-7905:

    Attachment: 7905.txt

Patch is a bunch of generated protobuf code so is larger than it actually is.

I am still fixing tests so this patch is not yet complete but am putting it up for
review looking for feedback.  It depends on hbase-7899 and hbase-7900 going in first.

This adds passing of encoded blocks of cells over rpc after the request header and request
on request and then after the response header and response result on reply.  Cell blocks
are optionally passed though I have the multi call doing it going both ways (There may
be more to do yet).

There is much not to like in this patch:

+ We need to ferry the Cells down from the client and into the rpc layer where they
are built ahead of sending. We do it by taking them on to a object to get them over
the proxy; given our protocols all implement protobuf Service, it tacks them on to
till now 'useless' rpccontroller (it is not a payloadcarryingrpccontroller).  It
is symmetric in that we do the same server side both getting Cells and then replying.
We constrained by protobuf Service in that it takes a single param Message only
and return a single Message response.  We could amend protoc and output methods
that take an extra arg, as garyh has suggested.  We can do this later.
+ This is not the fault of this patch, but it does bring it out in starker
relief, when we get to the server-side, the first thing we do is we convert
protos to pojos; there is awful lot of work being done here.  We should let
the protos go into the server and save a conversion.  This we should do.
+ It is a little awkward trying to keep the pb object together with its
data which is being carried separately out in cell blocks.  Hard to avoid but
ugly nonetheless.

Added a Cell proto.  Have Result proto carry Cells rather than KeyValues

Renamed the Mutate proto to be MutationProto.

In rpc.proto, wrote out how the new wire protocol works.  Amended
object types in accordance.  New connectionheader, requestheader,
responseheader, etc., proto types used by rpc

Moved a ServerCallable out of ProtobufUtil which seemed like a strange
place for it and made it standalone in the client package.  There is
alot going on in this class already setting up the multi calls.

Added FatalConnectionException in ipc.  Used when Connection setup fails

Rejigger of HBaseClient and HBaseServer to implement new rpc spec.
Renamed some methods to suit the spec and more what part of the
communication they are doing.  A Call now has a CellScanner to send
or as its response.  Stripped a bunch of unused code (// call)

The QoS stuff was internal to RegionServer.  I kept it for now
(though we should do what Elliott suggests and just have client
say what its priority is -- thats easier -- and we should do it)
but moved it out to its own class.  Removed the crazy bit where
we would deserialize the request just to figure its method name
so could figure a priority.

Added an IPCUtil in ipc package to do some common operation,
some of which are shared by server and client; e.g. cell block
building and reading.

Added PayloadCarryingRpcController class to carry CellScanners across
the proxy divide.

There were a few places where we were doing caching of class constructors
and methods.  Moved them all out to new RelectionCache class and have
everyone use it.

RemoteWithExtrasException has been padded out to carry stuff needed
by RegionMovedException and DoNotRetryException.

RpcServer was crazy and still is.  Needs cleanup badly.

In a bunch of places using TextFormat.  Its nice in that it prints out
the protobuf; handly debugging... but it prints the whole thing.
Need to come along after and add a version that does a shortened version
but will go with this for now (need to get this toString facility up
in UI too... and into the slowToRespond type log messages....
really helps... its something that had to be hand done
when we were up on hand done Writables)

Added convetions to protobufutil and to ResponseConverter to handle CellScanner.

Adds a MessageCodec, a cell block encoder that passes Cells as pb.
> Add passing of optional cell blocks over rpc
> --------------------------------------------
>                 Key: HBASE-7905
>                 URL: https://issues.apache.org/jira/browse/HBASE-7905
>             Project: HBase
>          Issue Type: Sub-task
>          Components: IPC/RPC
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.96.0
>         Attachments: 7905.txt
> Make it so we can pass Cells/data w/o having to bury it all in protobuf to get it over
the wire.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message