fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Meier, Caleb" <Caleb.Me...@parsons.com>
Subject order guarantees on event processing
Date Tue, 05 Jul 2016 19:22:49 GMT

We are currently using Fluo to incrementally update precomputed queries for the RDF triplestore
Rya, which is another incubating Apache project.  We have a strategy in place for updating
our queries as new triples are ingested, and we are now
working on delete support.

I'm wondering if Fluo provides any guarantees on the order in which it processes notifications.
 Are notifications for each observer added to a queue?  For example, if Column C has an observer
O and event e1 is written before event e2, is there a guarantee that O will process e1 before
e2?  The reason I'm curious about these guarantees is that we want to address the case of
one user adding a triple and another user deleting the same triple.  We would like the changes
to percolate through our observers in a manner that is consistent with the order in which
the triples are written/deleted from the triples column.

If there are no guarantees, do you have any suggestions as to how to enforce notification
processing order?  Our initial thought is to require our add/delete observer to only process
the event with the lowest add/delete timestamp.  That is, if O attempts to process e2, it
checks for the event with the lowest timestamp (e1) and ignores e2 until it encounters and
processes e1.  This seems like it could create a huge bottleneck given that it may take a
large number of cycles before it processes e1.  Please let me know if you need any additional
info for added context.

Caleb Meier

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message