lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ruslan Sivak <rsi...@istandfor.com>
Subject Re: Distributed Indexes
Date Sun, 10 Feb 2008 17:55:34 GMT
So nobody's run into anything like this before?  The need to share the 
index between many copies of the app possibly running on multiple servers?

Russ

Ruslan Sivak wrote:
> The app does other things then search the index.  I'm basically using 
> ColdFusion for the website and have four instances running on two 
> servers for load balancing.  Each app does the searches, and the 
> search times are small, the index is small, but it takes a long time 
> to fully create the index (several minutes), and I would like the 
> index to always be up to date (which is why i replicate the changes).
> I basically cache the index for several minutes in a RamDirectory, 
> which works quite well for performance.  If I could store the index in 
> a SQL Table or something, I can have a single place where the index 
> lives and atomic updates.
> Is there a SQL Backend for the index, or should I just take the 
> RamDirectory, serialize it and store it as a BLOB?
>
> Russ
>
> Erick Erickson wrote:
>> With an index that small, I wonder why you bother with so many copies?
>> What kind of load are you hitting it with and how complex are the 
>> queries?
>>
>> Because unless you have *very* high query rate, I'd look at why my 
>> queries
>> were
>> taking so long before complexifying things this way.
>>
>> Best
>> Erick
>>
>> On Feb 7, 2008 4:52 PM, Ruslan Sivak <rsivak@istandfor.com> wrote:
>>
>>  
>>> My index is only 4mb.  Is there a SQL backend for Lucene?
>>>
>>> Russ
>>>
>>> Michael McCandless wrote:
>>>    
>>>> If you're able to tell Windows FRS which specific files to copy, then
>>>> SnapshotDeletionPolicy (in 2.3) should work for this.
>>>>
>>>> It basically protects a consistent snapshot of your index, ensuring
>>>> those files will not be deleted, while not blocking further updates to
>>>> the index.
>>>>
>>>> Mike
>>>>
>>>> Ruslan Sivak wrote:
>>>>
>>>>      
>>>>> I'm wondering if this is a problem that lucene users have already
>>>>> tackled.  I have four copies of the application using a lucene
>>>>> index.  They are located on two physical servers with two copies on
>>>>> each server accessing two copies of the lucene index.  I use Windows
>>>>> FRS (File Replication Service) to replicate the index between the two
>>>>> servers.
>>>>> Things work well most of the time, but sometimes, I believe under
>>>>> load, the index doesn't get a chance to propagate before another
>>>>> write takes place and it gets corrupted.
>>>>> What would you recommend I use to keep the index in sync between the
>>>>> four copies of the app?
>>>>>
>>>>> Russ
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>         
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>       
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>     
>>
>>   
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message