lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Smith <psm...@aconex.com>
Subject Re: Index Replication / Clustering
Date Mon, 27 Jun 2005 11:22:16 GMT

On 27/06/2005, at 7:14 PM, Nader Henein wrote:

> I implemented a JMS based solution about a year ago because I  
> thought it would solve my atomicity problem and give me a  
> centralized way of indexing, you'll have to use the pluggable  
> persistence (if you use ActiveMQ) to be able to recover from a  
> failure and you'll also need some way of maintaining which records  
> have and haven't been indexed in a persistent store so you can run  
> sanitization later on, assuming that every time you add a document  
> to one of you clustered indecies everything goes well is a sure way  
> of guaranteeing that you end up with n indecies with no two alike  
> especially if you have a high volume indecies with high rates of  
> change.

The rate of updates is actually quite low in our environment, so we  
are lucky, however we will be monitoring consistency closely.  Each  
node exposes statistics about the last item indexed etc so we can  
tell when things are out of sync.  Should a node consistently be out  
of sync we'll just remove it as a worker from Apache for searching  
purposes (still leaving it ready to be used as a hot spare).

We are fortunate that the volume of updates is relatively small (as  
fast as a human can upload things), so we are in a fortunate position  
in our case. Our guarantees are in the order of minutes rather than  
seconds.

Paul
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message