jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcel Reutegger <marcel.reuteg...@gmx.net>
Subject Re: Is doc addition / indexing synchronous or asynchronous?
Date Thu, 06 Apr 2006 10:05:09 GMT
Carlos Villegas wrote:
> Marcel Reutegger wrote:
>> Jukka Zitting wrote:
>>
>>> Marcel, just out of curiosity, would it be possible to run the indexer
>>> using asynchronous events or would that cause some complications?
>>
>> yes, there are two issues with this approach:
>>
>> - there would be no guarantee anymore that you will find nodes with a
>> query that were just added. This then depends on when the asynchronous
>> notification updates the index.
> 
> Can't this be delayed until query time. When a query is run, determine
> if indexing is required and do it at that time not when changing the node.

this is actually done using a buffer as described in a previous post in 
this thread.

the only problem with this approach is, that text filters are run when 
nodes are added and not when nodes are indexed.
this should be changed and is tracked as issue JCR-264:
http://issues.apache.org/jira/browse/JCR-264

>> - in case of a jvm crash right after a change, that change might not be
>> reflected in the index.
>>
>> while the first point is just a temporary inconsistency, the latter is
>> severe and cannot be recovered.
>>
> 
> why can't the indexer just have a (persistent) list of dirty nodes and
> do everything in the background or when a query is requested. If the jvm
> crashes, the node is still dirty so it's done next time the system is up.

same here. jackrabbit does something similar already. there is a 
redo.log that acts exactly as the 'persistent list' of dirty nodes you 
mentioned. and yes, there is already a background thread ;) that 
periodically flushes pending changes to the index.

the initial question was whether we could register the whole query 
handler as asynchronous listener. that's not possible because of the 
above stated problems. but it is possible to move even more work that is 
currently done by the current thread to a background thread. solving 
JCR-264 should also help to background indexing work.

regards
  marcel

Mime
View raw message