accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Parise, Jonathan" <Jonathan.Par...@gd-ms.com>
Subject RE: Watching for Changes with Write Ahead Log?
Date Thu, 01 Oct 2015 17:12:33 GMT
I have a few follow up questions in regard to constraints.

What is the lifecycle of a constraint? What I mean by this is are the constraints somehow
tied to Accumulo’s lifecycle or are they just instantiated each time a mutation occurs and
then disposed?

Also, are there multiple instances of the same constraint class at any time or do all mutation
on a table go through the exact same constraint?

My guess is that  when a mutation comes in a new constraint is made through reflection. Then
check() is called, the violation codes are parsed and the object is disposed/finalized.

The reason I ask is that what I want to do is update my ElasticSearch index each time I see
a mutation on the table. However, I don’t want to have to make a connection, send the data
and then tear down the connection each time. That’s a lot of unnecessary overhead and with
all that overhead happening on every mutation performance could be badly impacted.

Is there some way to cache something like a connection and reuse it between calls to the Constraint’s
check() method? How would such a thing be cleaned up if Accumulo is shut down?


Thanks again,

Jon
From: Parise, Jonathan [mailto:Jonathan.Parise@gd-ms.com]
Sent: Wednesday, September 30, 2015 9:21 AM
To: user@accumulo.apache.org<mailto:user@accumulo.apache.org>
Subject: RE: Watching for Changes with Write Ahead Log?

In this particular case, I need to update some of my application state when changes made by
another system occur.

I would need to do a few things to accomplish my goal.


1)      Be notified or see that a table had changed

2)      Checked that against changes I know my system has made

3)      If my system is not the originator of the change, update internal state to reflect
the change.

Examples of state I may need to update include an ElasticSearch index and also an in memory
cache.

I’m going to read up on constraints again and see if I can use them for this purpose.

Thanks!

Jon



From: Adam Fuchs [mailto:afuchs@apache.org]
Sent: Tuesday, September 29, 2015 5:46 PM
To: user@accumulo.apache.org<mailto:user@accumulo.apache.org>
Subject: Re: Watching for Changes with Write Ahead Log?

Jon,

You might think about putting a constraint on your table. I think the API for constraints
is flexible enough for your purpose, but I'm not exactly sure how you would want to manage
the results / side effects of your observations.

Adam


On Tue, Sep 29, 2015 at 5:41 PM, Parise, Jonathan <Jonathan.Parise@gd-ms.com<mailto:Jonathan.Parise@gd-ms.com>>
wrote:
Hi,

I’m working on a system where generally changes to Accumulo will come through that system.
However, in some cases, another system may change data without my system being aware of it.

What I would like to do is somehow listen for changes to the tables my system cares about.
I know there is a write ahead log that I could potentially listen to for changes, but I don’t
know how to use it. I looked around for some documentation about it, and I don’t see much.
I get the impression that it isn’t really intended for this type of use case.

Does anyone have any suggestions on how to watch a table for changes and then determine if
those changes were made by a different system.

Is there some documentation about how to use the write ahead log?


Thanks,

Jon Parise

Mime
View raw message