accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: WAL Log - Real-time updates
Date Wed, 22 Oct 2014 18:07:02 GMT
the way this gets done in HBase, i.e. for the HBase Lily Indexer[1], is to
add a replication consumer that isn't an actual cluster. IMHO, you'd be
better off taking that kind of approach rather than trying to consume the
WALs off of HDFS. I haven't attempted to use our replication interface for
this yet, but in principle it should work.

Note that either of these approaches are going to be very fragile across
Accumulo versions because they aren't interfaces intended for consumption.

[1]: http://ngdata.github.io/hbase-indexer/

On Wed, Oct 22, 2014 at 12:59 PM, D P <pacificobuzz@gmail.com> wrote:

> I am working with Accumulo and looking for the best means of knowing when
> something has been updated/inserted into my Accumulo instance.  For
> instance, every-time data is inserted, how can I know externally?  If the
> write-ahead log file stores this, is it best to just read the HDFS WAL log
> with a storm spout to know when something has been inserted into a table?
>
> I am planning to do some real-time visualization with accumulo, but when
> data is inserted I want to be able to notify my UI.
>
> Thanks!
>



-- 
Sean

Mime
View raw message