hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jurriaan Mous (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12597) Add RpcClient interface and enable changing of RpcClient implementation
Date Sun, 30 Nov 2014 13:34:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229102#comment-14229102

Jurriaan Mous commented on HBASE-12597:

I indeed mention HCI since it is indeed still the default implementation in branch-1. And
I would like to switch out the RpcClient in whatever the default implementation is so I don't
need to reimplement everything in a new Connection implementation. 
I have been working of the 0.98 branch with my project since I currently use that within my
project. I currently use the HConnection for 3 main reasons: getting the region for a table/row,
getting the cluster ID for authorization and getting the Nonce generator. I have studied what
has changed in master-1 and it would not be difficult to incorporate the changes. I only based
my implementation on a few classes in HBase client, mainly RpcClient and the ResultScanners.

"Can you call your client HBaseClient rather than HbaseClient?"
Ah, I used a different naming scheme but since this is what HBase uses itself as a naming
scheme I will adopt it :) 

"How does it relate to asynchbase if it atll? Its more derivative of native hbase client than
asynchbase is?"
Yes, I didn't like asynchbase was a complete new implementation of the Api. While it looks
like it implemented to be as fast as possible it missed out on a few features that I was using.
To use it I would also need to rebuild lots of code which return Puts etc. And studying it
I saw it was easier to do a new implementation based on the Hbase client itself and the newer
different Netty 4 api. Since everything is based on protobuf services it would be relatively
easy. And I had experience with Netty 4 recently so I saw I could do this pretty quickly.
And I used the Hbase implementation as the basis for connection pooling and more so I wouldn't
reinvent the wheel. 

"I see you pass in our configs. Do they work for your async context. I would not be afraid
introducing new ones if it makes your implementation easier... ours have been around a long
time and have been overloaded and their original intent may have been lost over time. Just
Almost all the connection configs in RpcClient mapped 1-to-1 to Netty socket or channel handler
settings so I reused them. (maxretries, socket-timeout, tcpnodelay etc) And for the scanner
they were also great to have some sensible defaults for things like retries, timeouts and
max result count. I did not have any problem implementing them since the async context didn't
matter to them.

"Thats great you carried over codecs. Maybe in netty context, we can drop these and use netty
channel compression instead? (won't work for sasl I suppose but for everything else)."
Yes that would be a great experiment :) Those kind of experiments are the reason I am advocating
this issue to make it easy to swap out the RpcClient so it can also be tested in current environments
which do not use an async context yet. Although such a change would also need changes to the
server. (I havent studied yet how it is implemented and to do a Netty implementation there
is outside my current ambitions)

"Hmm... you support nonce too? If so, thats great."
Yes I currently read it out of the nonce generator of the passed in HConnection. I see that
needs to change with new Connection interface.

> Add RpcClient interface and enable changing of RpcClient implementation
> -----------------------------------------------------------------------
>                 Key: HBASE-12597
>                 URL: https://issues.apache.org/jira/browse/HBASE-12597
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>            Reporter: Jurriaan Mous
> Currently HConnectionImplementation works with the included RpcClient which is a direct
implementation and not defined by an interface.
> It would be great to be able to swap out the default RpcClient with another implementation
which can also be controlled by the default HConnectionImplementation. 
> Suggested changes:
> - Create a RpcClient interface which defines all the ways HConnectionImplementation interacts
with an RPC client. Like getting a blocking protobuf service interface or closing the client.
> - Define which RpcClient implementation to construct by setting a configuration variable
which defaults to the current RpcClient.
> - Possibly create an abstract RpcClient class to only load all the basic Rpc layer configurations
to be used in an implementation.
> Why? It enables experimentation with RpcClients which could enable new features or could
be more performant than the included client. 
> I created a new RpcClient implementation based on Netty which can also be called asynchronously.
It would be great to also be able to use this RpcClient in all the default ways and tests
to see if there are any issues with it. 
> https://github.com/jurmous/async-hbase-client/
> https://github.com/jurmous/async-hbase-client/blob/master/src/main/java/org/apache/hadoop/hbase/ipc/AsyncRpcClient.java

This message was sent by Atlassian JIRA

View raw message