lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Search scalability
Date Thu, 11 Nov 2004 14:08:44 GMT
If you load it explicitly, then all 800 MB will make it into RAM.
It's easy to try, the API for this is super simple.

Otis

--- yahootintin-lucene@yahoo.com wrote:

> Does it take 800MB of RAM to load that index into a
> RAMDirectory?  Or are only some of the files loaded into RAM?
> 
> --- Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> 
> > Hello,
> > 
> > 100 parallel searches going against a single index on a single
> > disk
> > means a lot of disk seeks all happening at once.  One simple
> > way of
> > working around this is to load your FSDirectory into
> > RAMDirectory. 
> > This should be faster (could you report your
> > observations/comparisons?).  You can also try using ramfs if
> > you are
> > using Linux.
> > 
> > Otis
> > 
> > --- Ravi <ravi@htinc.com> wrote:
> > 
> > >  We have one large index for a document repository of
> > 800,000
> > > documents.
> > > The size of the index is 800MB. When we do searches against
> > the
> > > index,
> > > it takes 300-500ms for a single search. We wanted to test
> > the
> > > scalability and tried 100 parallel searches against the
> > index with
> > > the
> > > same query and the average response time was 13 seconds. We
> > used a
> > > simple IndexSearcher. Same searcher object was shared by all
> > the
> > > searches. I'm sure people have success in configuring lucene
> > for
> > > better
> > > scalability. Can somebody share their approach?
> > > 
> > > Thanks 
> > > Ravi. 
> > > 
> > >
> >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> > > 
> > > 
> > 
> > 
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> > 
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message