hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Themis : implements cross-row/corss-table transaction on HBase.
Date Mon, 10 Nov 2014 15:39:56 GMT
Jianwei:
I used this command to clone your repo:
git clone https://github.com/XiaoMi/themis/tree/for_hbase_0.98 themis

But I only found 0.94 being referenced in the pom.xml files:

$ find . -name pom.xml -exec grep '0.94.' {} \; -print
    <hbase.version>0.94.21</hbase.version>
./themis-client/pom.xml
    <hbase.version>0.94.21</hbase.version>
./themis-coprocessor/pom.xml
    <hbase.version>0.94.21</hbase.version>
./themis-index/pom.xml

Did I miss something ?

Cheers

On Mon, Nov 10, 2014 at 12:50 AM, 崔建伟 <cuijianwei@xiaomi.com> wrote:

> Hi everyone:
> In last few months, we have updated Themis to achieve better performance
> and include more features:
>
> 1. Improve the single-row write performance from 23%(relative drop
> compared with HBase's put) to 60%(for most test cases). For single-row
> write transaction, we only write lock to MemStore in prewrite-phase, then,
> we erase corresponding lock, write data and commit information to HLog in
> commit-phase. This won't break the correctness of percolator algorithm and
> will help improve the performance a lot for single-row write.
>
> 2. Support HBase 0.98. We create a branch:
> https://github.com/XiaoMi/themis/tree/for_hbase_0.98 to make themis
> support HBase 0.98(Currently, support HBase 0.98.5). All the functions of
> master branch will also be implemented in this branch.
>
> 3. Transaction TTL support and Old Data Clean. Users could set TTL for
> read/write transaction respectively. Then, old data which could not be read
> will be cleaned periodically.
>
> 4. MapReduce Support. We implement a group of classes to support read data
> by themis transaction in Mapper job and write data by themis transaction in
> Reduce job.
>
> For more details, please see the github:
> https://github.com/XiaoMi/themis(or
> https://github.com/XiaoMi/themis/tree/for_hbase_0.98) or jira:
> https://issues.apache.org/jira/browse/HBASE-10999 . If you find Themis
> interesting, please leave us comment in the mail, jira or github.
>
> Best
> cuijianwei
>
> ________________________________________
> From: saint.ack@gmail.com <saint.ack@gmail.com> on behalf of Stack <
> stack@duboce.net>
> Sent: Sunday, July 13, 2014 1:12 PM
> To: HBase Dev List
> Cc: user@hbase.apache.org
> Subject: Re: Themis : implements cross-row/corss-table transaction on
> HBase.
>
> On Tue, Jul 8, 2014 at 12:34 AM, 崔建伟 <cuijianwei@xiaomi.com> wrote:
>
> > Hi everyone, I want to introduce our open-source project Themis which
> > implements cross-row/corss-table transaction on HBase.
> >
> > Themis follows google's percolator algorithm(
> > http://research.google.com/pubs/pub36726.html), which provides
> > ACID-compliant transaction and snapshot isolation. The cross-row
> > transaction is based on HBase's single-row atomic semantics and doesn't
> use
> > a central transaction server, so that supports linear-scalability.
> >
> > Themis depends on a timestamp server to provides global strictly
> > incremental timestamp to define the order of transactions, which will be
> > used to resolve the write-write and read-write conflicts. The timestamp
> > server is lightweight and could achieve hight throughput(500, 000 + qps),
> > and Themis will batch timestamp requests across transactions in one Rpc,
> so
> > that it won't become the bottleneck of the system even when processing
> > billions of transactions every day.
> >
> > Although Themis could be implemented totally in client-side, we adopt
> > coprocessor framework of HBase to achieve higher performance. Themis
> > includes a client-side library to provides transaction APIs, such as
> > themisPut/themisGet/themisScan/themisDelete, and a coprocessor library
> > loaded on regionserver. Therefore, Themis could be used without changing
> > the code and logic of HBase.
> >
> > We have been validating the correctness of Themis for a few months by a
> > AccountTransfer simulation program, which concurrently does cross-row
> > transactions by transferring money among different accounts(each account
> is
> > a row in HBase) and verifies total money of all accounts doesn't change
> in
> > the simulation. We have also run Themis on our production environment.
> >
> > We test the performance of Themis and get comparable result as
> percolator.
> > The single-column transaction represents the worst performance case for
> > Themis compared with HBase, the result is:
> > 1) For read, the performance of percolator is 90% of HBase;
> > 2) For write, the performance of percolator is 23% of HBase.
> > The write performance drops a lot because Themis uses two-phase commit
> > protocol to achieve ACID of transaction. For multi-row write, we improve
> > the performance by paralleling all writes of pre-write phase. For
> > single-row write, we are optimizing two-phase commit protocol to achieve
> > better performance and will update the result when it is ready. The
> details
> > of performance result could be found in github.
> >
> > The repository and introduction of Themis include:
> > 1. Themis github: https://github.com/XiaoMi/themis/. The source code,
> > performance test result and user guide could be found here.
> > 2. Themis jira : https://issues.apache.org/jira/browse/HBASE-10999
> > 3. Chronos github: https://github.com/XiaoMi/chronos. Chronos is our
> > open-source high-availability, high-performance timestamp server to
> provide
> > global strictly incremental timestamp for Themis.
> >
> > If you find Themis interesting, please leave us comment in the mail, jira
> > or github.
> >
> > Best
> > cuijianwei
> >
>
> Excellent.  Nice work lads!
> St.Ack
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message