hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: Omid: Transactional Support for HBase
Date Sat, 05 Nov 2011 04:37:27 GMT
Cool stuff Daniel,

Was looking through the code a bit. Seems like you make a best effort to push as much of
the filtering of KVs of uncommitted transactions to HBase and then do some filtering on the
not a bad approach. (I hope I didn't misunderstand the approach, only looked through the code
1/2 hour or so).

One thing I was wondering: Why bookkeeper? Why not store the WAL itself in HBase? That way
you might not even need a separate server.

Did you see: HBaseSI (http://www.cs.uwaterloo.ca/~c15zhang/HBaseSI.pdf), they also do MVCC
on top of unaltered HBase/schema, although from reading that paper I get the impression that
would not scale to scans touching many rows (which is where your client side filtering comes

-- Lars

----- Original Message -----
From: Daniel Gómez Ferro <danielgf@yahoo-inc.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org>; "user@hbase.apache.org" <user@hbase.apache.org>
Cc: Maysam Yabandeh <maysam@yahoo-inc.com>; Flavio Junqueira <fpj@yahoo-inc.com>;
Benjamin Reed <breed@yahoo-inc.com>; Ivan Kelly <ivank@yahoo-inc.com>
Sent: Friday, November 4, 2011 4:24 AM
Subject: Omid: Transactional Support for HBase

(I apologize for resending but I forgot to add the user list.)

Hi all,

It is my pleasure to announce the open source release of Omid, a project whose goal is to
add lock-free transactional support on top of HBase. The current release includes CrSO, a
client-replicated status oracle that detects the write-write conflicts to provide Snapshot
Isolation. CrSO has the following appealing properties:

1) It does not need any modification into the HBase code nor the table scheme.
2) The overhead on HBase DataNodes is negligible (only after an abort)
3) It scales up to 50,000 write transactions per second (TPS) and a thousand of client connections.

We have setup a github project: https://github.com/dgomezferro/omid

More information is available at the wiki: https://github.com/dgomezferro/omid/wiki

If you are interested, installation and running instructions are available on the README:

Please do not hesitate to contact us in the case of any question.

Best Regards,
Daniel Gómez Ferro

View raw message