hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Mor <amit....@hotmail.com>
Subject Protobuf Proxy - your feedback is welcomed !
Date Wed, 05 Aug 2015 07:29:36 GMT




I've been thinking of creating a new Twemproxy/Thrift (https://github.com/twitter/twemproxy)
like proxy for HBase that would be based on using (as the underlying transport protocol) the
native protocol-buffer templates that come with HBase source. 
This could be viewed as a Thrift alternative, also, and CMIIW, I don't see any reason why
the HBase community should support 2 protocol types, at least from 'singularity' version onward,
where protobuf is in heavy use internally/externally.
 
I envision that the proxy would provide the following benefits:Async semantics - client would
not need to implement anything new except wrapping their calls in something like futures and
pass the calls to the proxy, using the same 'commands' as the Java native cmds (that are already
defined as protos). Only (well, mostly...) the proxy would have to deal the concerns associated
with contention, starvation, blocking etc.Batching/Pipelining - the service would be able
to batch queries sent from several clients connected to the (local) proxy service and batch
them The proxy service could act as Facade between the client and HBase (i.e. stabilize a
protocol that withhold API changes in HBase client) Separation of concerns - no need for the
application to 'tame' the HBase driver in terms of IO resources, memory and buffer size, connection
pool size, etc.Naturally - Language agonisticTesting - the proxy could hold a mock connection
to HBase or a mock HBase for easier testing
Your kind feedback on the idea, concept and usefulness is very welcomed
Thanks,
Amit Mor
  		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message