accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From D P <pacificob...@gmail.com>
Subject Re: WAL Log - Real-time updates
Date Wed, 22 Oct 2014 18:30:01 GMT
The lily indexer/SEP is really interesting.  Thanks for both of your posts

On Wed, Oct 22, 2014 at 2:07 PM, Sean Busbey <busbey@cloudera.com> wrote:

> the way this gets done in HBase, i.e. for the HBase Lily Indexer[1], is to
> add a replication consumer that isn't an actual cluster. IMHO, you'd be
> better off taking that kind of approach rather than trying to consume the
> WALs off of HDFS. I haven't attempted to use our replication interface for
> this yet, but in principle it should work.
>
> Note that either of these approaches are going to be very fragile across
> Accumulo versions because they aren't interfaces intended for consumption.
>
> [1]: http://ngdata.github.io/hbase-indexer/
>
> On Wed, Oct 22, 2014 at 12:59 PM, D P <pacificobuzz@gmail.com> wrote:
>
>> I am working with Accumulo and looking for the best means of knowing when
>> something has been updated/inserted into my Accumulo instance.  For
>> instance, every-time data is inserted, how can I know externally?  If the
>> write-ahead log file stores this, is it best to just read the HDFS WAL log
>> with a storm spout to know when something has been inserted into a table?
>>
>> I am planning to do some real-time visualization with accumulo, but when
>> data is inserted I want to be able to notify my UI.
>>
>> Thanks!
>>
>
>
>
> --
> Sean
>

Mime
View raw message