lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nader Henein <...@bayt.net>
Subject Re: Lucene in clustered environment (Tomcat)
Date Tue, 07 Jun 2005 13:47:17 GMT
I realize I've already asked you this question, but do you need 100% 
real time, because you could run batch them every 2 minutes, and 
concerning Parallel search, unless you really need it, it's overkill in 
this case, a communal index will serve you well and will be much easier 
to maintain. You have to way requirement vs. complexity/ debug time.

Nader Henein

Ben wrote:

>>When you say your cluster is on a single machine, do you mean that you have multiple
webservers on the same machine all of which search a single Lucene index?
>>    
>>
>
>Yes, this is my case.
>
>  
>
>>Do you use Lucene as your persistent store or do you have a DB back there?
>>    
>>
>
>I use Lucene to search for data stored in a PostgreSQL server.
>
>  
>
>>what is your current update/delete strategy because real time inserts from the webservers
directly to the index will not work because you can't have multiple writers.
>>    
>>
>
>I have to do this in real time, what are the available solutions? My
>application has the ability to do batch update/delete to a Lucene
>index but I would like to do this in real time.
>
>One solution I am thinking is to have each cluster has it own index
>and use parallel search. This makes my application even more complex.
>
>  
>
>>I strongly recommend Quartz, it's rock solid and really versatile.
>>    
>>
>
>I am using Quartz, it is really great and supports cluster.
>
>Thanks,
>Ben
>
>
>On 6/7/05, Nader Henein <nsh@bayt.net> wrote:
>  
>
>>When you say your cluster is on a single machine, do you mean that you
>>have multiple webservers on the same machine all of which search a
>>single Lucene index? Because if that's the case, your solution is
>>simple, as long as you persist to a single DB and then designate one of
>>your servers (or even another server) to update/delete the index. Do you
>>use Lucene as your persistent store or do you have a DB back there? and
>>what is your current update/delete strategy because real time inserts
>>from the webservers directly to the index will not work because you
>>can't have multiple writers. Updating a dirty flag on rows that need to
>>be indexed/deleted, or using a table for this task and then batching
>>your updates would be ideal, and if you're using server specific
>>scheduling, I strongly recommend Quartz, it's rock solid and really
>>versatile.
>>
>>My two cents.
>>
>>Nader Henein
>>
>>
>>Ben wrote:
>>
>>    
>>
>>>My cluster is on a single machine and I am using FS index.
>>>
>>>I have already integrated Lucene into my web application for use in a
>>>non-clustered environment. I don't know what I need to do to make it
>>>work in a clustered environment.
>>>
>>>Thanks,
>>>Ben
>>>
>>>On 6/7/05, Nader Henein <nsh@bayt.net> wrote:
>>>
>>>
>>>      
>>>
>>>>IMHO, Issues that you need to consider
>>>>
>>>>   * Atomicity of updates and deletes if you are using multiple indexes
>>>>     on multiple machines (the case if your cluster is over a wide network)
>>>>   * Scheduled indecies to core data comparison and sanitization
>>>>     (intensive)
>>>>
>>>>This all depends on what the volume of change is on your index and
>>>>whether you'll be using a Memory resident index or an FS index.
>>>>
>>>>This should start the ball rolling, we've been using Lucene successfully
>>>>on a distributed cluster for a while now, and as long as you're aware of
>>>>some basic NDS limitations/constraints you should be fine.
>>>>
>>>>Hope this helps
>>>>
>>>>Nader Henein
>>>>
>>>>Ben wrote:
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>Hi
>>>>>
>>>>>I would like to use Lucene in a clustered environment, what are the
>>>>>things that I should consider and do?
>>>>>
>>>>>I would like to use the same ordinary index storage for all the nodes
>>>>>in the the cluster, possible?
>>>>>
>>>>>Thanks,
>>>>>Ben
>>>>>
>>>>>---------------------------------------------------------------------
>>>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>--
>>>>
>>>>Nader S. Henein
>>>>Senior Applications Architect
>>>>
>>>>Bayt.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>      
>>>
>>--
>>
>>Nader S. Henein
>>Senior Applications Architect
>>
>>Bayt.com
>>
>>
>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>    
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
>
>
>
>  
>

-- 

Nader S. Henein
Senior Applications Architect

Bayt.com





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message