hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Malaska <ted.mala...@cloudera.com>
Subject [DISCUSS] Multi-Cluster HBase Client
Date Mon, 29 Jun 2015 18:48:52 GMT
Hey Dev List,


My name is Ted Malaska, long time lover and user of HBase. I would like to
discuss adding in a multi-cluster client into HBase. Here is the link for
the design doc (
https://github.com/tmalaska/HBase.MCC/blob/master/MultiHBaseClientDesignDoc.docx%20(1).docx)
but I have pulled some parts into this main e-mail to give you a high level
understanding of it's scope.


*Goals*

The proposed solution is a multi-cluster HBase client that relies on the
existing HBase Replication functionality to provide an eventual consistent
solution in cases of primary cluster down time.


https://github.com/tmalaska/HBase.MCC/blob/master/FailoverImage.png


Additional goals are:

   -

   Be able to switch between single HBase clusters to Multi-HBase Client
   with limited or no code changes.  This means using the HConnectionManager,
   Connection, and Table interfaces to hide complexities from the developer
   (Connection and Table are the new classes for HConnection, and
   HTableInterface in HBase version 0.99).
   -

   Offer thresholds to allow developers to decide between degrees of
   strongly consistent and eventually consistent.
   - Support N number of linked HBase Clusters


*Read-Replicas*
Also note this is in alinement with Read-Replicas and can work with that.
This client is multi-cluster where Read-Replicas help us to be multi Region
Server.

*Replication*
You will also see in the document that this works with current replication
and requires no changes to it.

*Only a Client change*
You will also see in the doc this is only a new client. Which means no
extra code for the end developer, only addition configs to set it up.

*Github*
This is a github project that shows that this works at:
https://github.com/tmalaska/HBase.MCC
Note this is only a prototype. When adding it to HBase we will use it as a
starting point but there will be changes.

*Initial Results:*

Red is where our primary cluster has failed and you will see from the
bottom to graphs that our puts, deletes, and gets are not interrupted.
https://github.com/tmalaska/HBase.MCC/blob/master/AveragePutTimeWithMultiRestartsAndShutDowns.png

Thanks
Ted Malaska

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message