hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Themis : implements cross-row/corss-table transaction on HBase.
Date Sun, 13 Jul 2014 05:12:24 GMT
On Tue, Jul 8, 2014 at 12:34 AM, 崔建伟 <cuijianwei@xiaomi.com> wrote:

> Hi everyone, I want to introduce our open-source project Themis which
> implements cross-row/corss-table transaction on HBase.
>
> Themis follows google's percolator algorithm(
> http://research.google.com/pubs/pub36726.html), which provides
> ACID-compliant transaction and snapshot isolation. The cross-row
> transaction is based on HBase's single-row atomic semantics and doesn't use
> a central transaction server, so that supports linear-scalability.
>
> Themis depends on a timestamp server to provides global strictly
> incremental timestamp to define the order of transactions, which will be
> used to resolve the write-write and read-write conflicts. The timestamp
> server is lightweight and could achieve hight throughput(500, 000 + qps),
> and Themis will batch timestamp requests across transactions in one Rpc, so
> that it won't become the bottleneck of the system even when processing
> billions of transactions every day.
>
> Although Themis could be implemented totally in client-side, we adopt
> coprocessor framework of HBase to achieve higher performance. Themis
> includes a client-side library to provides transaction APIs, such as
> themisPut/themisGet/themisScan/themisDelete, and a coprocessor library
> loaded on regionserver. Therefore, Themis could be used without changing
> the code and logic of HBase.
>
> We have been validating the correctness of Themis for a few months by a
> AccountTransfer simulation program, which concurrently does cross-row
> transactions by transferring money among different accounts(each account is
> a row in HBase) and verifies total money of all accounts doesn't change in
> the simulation. We have also run Themis on our production environment.
>
> We test the performance of Themis and get comparable result as percolator.
> The single-column transaction represents the worst performance case for
> Themis compared with HBase, the result is:
> 1) For read, the performance of percolator is 90% of HBase;
> 2) For write, the performance of percolator is 23% of HBase.
> The write performance drops a lot because Themis uses two-phase commit
> protocol to achieve ACID of transaction. For multi-row write, we improve
> the performance by paralleling all writes of pre-write phase. For
> single-row write, we are optimizing two-phase commit protocol to achieve
> better performance and will update the result when it is ready. The details
> of performance result could be found in github.
>
> The repository and introduction of Themis include:
> 1. Themis github: https://github.com/XiaoMi/themis/. The source code,
> performance test result and user guide could be found here.
> 2. Themis jira : https://issues.apache.org/jira/browse/HBASE-10999
> 3. Chronos github: https://github.com/XiaoMi/chronos. Chronos is our
> open-source high-availability, high-performance timestamp server to provide
> global strictly incremental timestamp for Themis.
>
> If you find Themis interesting, please leave us comment in the mail, jira
> or github.
>
> Best
> cuijianwei
>

Excellent.  Nice work lads!
St.Ack

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message