lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Replicating Lucene Index with out SOLR
Date Thu, 28 Aug 2008 05:06:03 GMT
You don't need to copy the whole index every time if you do incremental indexing/updates and
don't optimize the index before copying.  If you use rsync for copying the index, only the
new/modified files be copied.  This is what Solr replication scripts do, too.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: rahul_k123 <vishnudeepak@gmail.com>
> To: general@lucene.apache.org
> Sent: Wednesday, August 27, 2008 11:36:07 PM
> Subject: Re: Replicating Lucene Index with out SOLR
> 
> 
> Currently we index every certain amount of time on A.
> 
> -copy the index
>      Copying the whole index everytime ? 
> 
> Currently i am investigating how i can make use of SOLR replication scripts
> to achive this.
> 
> 
> Is there anyone who did this with out SOLR before?
> 
> 
> Thanks
> 
> 
> 
> Otis Gospodnetic wrote:
> > 
> > Hi,
> > 
> > You may want to ask on the java-user list (more subscribers), which I'm
> > CC-ing, so we can continue discussion there.
> > I think you will have to implement your own logic that runs on A and does
> > something like this:
> > 
> > - stop adding new docs
> > - call commit on the IndexWriter
> > 
> > - copy the index
> > - resume indexing
> > 
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> > 
> > 
> > 
> > ----- Original Message ----
> >> From: rahul_k123 
> >> To: general@lucene.apache.org
> >> Sent: Thursday, August 28, 2008 1:34:41 AM
> >> Subject: Replicating Lucene Index with out SOLR
> >> 
> >> 
> >> I have the following requirement
> >> 
> >> Right now we have multiple indexes  serving our web application. Our
> >> indexes
> >> are around 30 GB size.
> >> 
> >> We want to replicate the index data so that we can use them to distribute
> >> the search load.
> >> 
> >> This is what we need ideally.
> >> 
> >> A – (supports writes and reads)
> >> 
> >> A1 –Replicated Index (Supports reads)  . We want to synchronize this
> >> every 5
> >> mins.
> >> 
> >> 
> >> 
> >> Any help is appreciated.   We are not using SOLR
> >> 
> >> I also interested in knowing what will be the best way so that I can
> >> scale
> >> my application adding more boxes for search if our load increases.
> >> 
> >> Thanks.  
> >> 
> >> -- 
> >> View this message in context: 
> >> 
> http://www.nabble.com/Replicating-Lucene-Index-with-out-SOLR-tp19191752p19191752.html
> >> Sent from the Lucene - General mailing list archive at Nabble.com.
> > 
> > 
> > 
> 
> -- 
> View this message in context: 
> http://www.nabble.com/Replicating-Lucene-Index-with-out-SOLR-tp19191752p19193670.html
> Sent from the Lucene - General mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message