lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Multiple writers
Date Wed, 29 Oct 2003 22:00:54 GMT

--- Dror Matalon <dror@zapatec.com> wrote:
> On Wed, Oct 29, 2003 at 07:56:53AM -0500, Scott Ganyo wrote:
> > Offhand, I would say that using 2 directories and merging them is 
> > exactly what you waht.  It really shouldn't be all that complicated
> and 
> > Lucene should handle the synchronization for you...
> 
> Will it do that if we build the indexes in two different JVMs, and
> let
> one of them handle the merge. I realize that Lucene is thread safe,
> and
> handles the locking, but I thought that that's only true if you do
> all the work in a single JVM.

Lucene uses file-based locks, so running application(s) that use the
same index in multiple JVMs, should be the same as running it in a
single JVM.

Otis


> > Scott
> > 
> > Dror Matalon wrote:
> > 
> > >Hi folks,
> > >
> > >We're in the process of adding search to our online RSS
> aggregator. You
> > >can see it in action at www.fastbuzz.com.
> > >
> > >Currently we have more than five million items in the systems and
> it's
> > >growing at the rate of more than 100,00 a day.  So we need to take
> into
> > >account is that the index is constantly growing.
> > >
> > >One of the things we want to build into the system is the ability
> to
> > >rebuild the index on the fly while still inserting the items that
> are
> > >coming in. 
> > >
> > >We've looked at having things go into different directories and
> then
> > >merge them, but it seems complicated and we'd need to worry about
> race
> > >conditions and locking issues.
> > >
> > >Anyone's done this before? Any suggestions?
> > >
> > >
> > >Regards,
> > >
> > >Dror
> > >
> > > 
> > >
> > 
> > -- 
> > ...there is nothing more difficult to execute, nor more dubious of
> success, 
> > nor more dangerous to administer than to introduce a new order to
> things; 
> > for he who introduces it has all those who profit from the old
> order as his 
> > enemies; and he has only lukewarm allies in all those who might
> profit from 
> > the new. - Niccolo Machiavelli
> > 
> > 
> > 
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> > 
> 
> -- 
> Dror Matalon
> Zapatec Inc 
> 1700 MLK Way
> Berkeley, CA 94709
> http://www.zapatec.com
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 


__________________________________
Do you Yahoo!?
Exclusive Video Premiere - Britney Spears
http://launch.yahoo.com/promos/britneyspears/

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message