hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Noels <stev...@outerthought.org>
Subject HBase RowLog Library
Date Mon, 21 Jun 2010 15:11:46 GMT
Hi all,

as promised, here's our take at a component to build queues and WALs that
run on top of HBase.

Available under Apache license from
http://lilycms.org/lily/roadmap/sketchbook/hbaserowlog.html thanks to the
hard work of my colleagues Evert and Bruno.

Rationale:

HBase provides atomic row-level updates. This is one of the important
reasons why we choose HBase <http://hbase.apache.org/> as the underlying
store for our upcoming Lily <http://www.lilycms.org/> open source content
repository, as consistency is very important for users trusting their data
onto our software. Next, we needed some way to rapidly locate HBase-stored
data without key-based access, so we came up with the idea of building
secondary
indexes on top of
HBase<http://lilycms.org/lily/roadmap/sketchbook/hbaseindexes.html>.
Ultimately, we'll also provide a flexible user-level search powered by
SOLR<http://lilycms.org/lily/prerelease/architecture.html>,
which will require us to maintain a frequently (near-real-time) updated SOLR
index alongside our HBase-backed main data store.

All this required us to have a flexible way to facilitate synchronous
actions across multiple HBase tables, and asynchronous queuing of index
updates for SOLR - in a distributed architecture with no real single point
of failure. We've named this the* HBase RowLog library*, and we're releasing
a first version of this RowLog library today.

Enjoy and let us know what you think of it.

Evert is on a well-deserved holiday chasing Pearl Jam concerts throughout
Europe, so you're best of sending remarks here.
Cheers,

Steven
-- 
Steven Noels
http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message