lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephane Bailliez <>
Subject Re: Index Replication / Clustering
Date Thu, 30 Jun 2005 07:57:17 GMT
Nader Henein wrote:
> Considerations that you may want to think about when sanitizing your 
> clustered indecies:
> 1) Number of documents available vs. number of documents in the 
> persistent store.
> 2) Are all the document up to date (involves comparing the existence and 
> the last date updated of Lucene documents to persistent store)
> 3) Have all the documents that should have been deleted actually been 
> deleted from the index, if you delete the documents from the persistent 
> store this is not trivial, we're using an "is_deleted" flag so we can 
> query the index for deleted documents, if we get any results, then 
> there's a problem.
> Sync issues will happen, welcome to the wonderful work of NDC, not by 
> any fault of programming, but simply because this is the nature of 
> network communication, and computing in demanding environments. Failing 
> to provide for contingency and fail safes will give you some of the most 
> intense headaches, ultimately, you want the system to fix itself (assume 
> failures for day one) I've been monitoring the system from a distance 
> for the past year with little or no interference (4 years in total), 
> there is nothing better than knowing that I could take a tire iron to a 
> few of our machines and that would not affect the site performance, I 
> get to sleep at night.

No worries, I'm already banging my head because there are 2 'different' 
incoming sources of messages (the sources of messages are redundant 
satellites connections on 2 different satellites), one is supposed to be 
a live backup, the 2 are running in parallel and the broker must ideally 
wait for a message to come from the 2 sources.

If the wait goes beyond a certain threshold, trigger an alert, put the 
failed node, ...

That's the easy part :)

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message