lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Smith <>
Subject Re: Index Replication / Clustering
Date Mon, 27 Jun 2005 11:18:59 GMT
>> If you use ActiveMQ for JMS, you can take advantage of it's  
>> Composite  Destination feature and have a virtual Queue/Topic that  
>> is actually  several Queues/Topics.  This is what we use to keep a  
>> mirror index  server completely in sync.  The application sends an  
>> update message  to a queue named "queue://index1, queue://index2",  
>> which becomes 2  separate queues for the 2 servers, allowing them  
>> to process the same  message whenever they can get around to it.
> Ah, the composite topic, is indeed a good nice. But out of  
> curiosity...did you put your 2 nodes (consumers) as embedded  
> brokers or is the producer as the main broker ?

Neither in our case, a central broker (albeit we plan to have a  
backup plan with ActiveMQ installed on all servers with the same  
config ready to roll, using zeroconf for discovery).

>> We then place Apache in front of these 2 mirrored Index/Search  
>> nodes  so the application can use web-services to query the search  
>> node  without actually being aware that there is 2 of them behind  
>> the  scenes, leaving Apache to do the load-balancing and fail-over  
>> as the  index/search nodes come up/down without the main  
>> application knowing  anything about it.
> Ideally, the 2 nodes have the same state when running.

Ideally, reality is different.  We're going to be monitoring  
consistency of the pair closely, and should the fall out of sync with  
regularity, then one of them is just going to be the hot-spare for  
failure purposes only.
> What happens when a node fails and that you put it back online and  
> that it needs to catch up with all missing messages in its queue ?
> Is it considered 'offline' until it catches up ? If yes how do you  
> do it ? If no, I guess you don't mind that a search request may not  
> give the same result depending on the node it is load-balanced,  
> correct ?

In this case we will manually mark the node via Apache worker configs  
to be be disabled until it has caught up.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message