lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi" <r...@htinc.com>
Subject RE: Search scalability
Date Thu, 11 Nov 2004 14:24:46 GMT
Thanks a lot. I'll use RAMDirectory and post my results.  

-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com] 
Sent: Thursday, November 11, 2004 9:09 AM
To: Lucene Users List
Subject: Re: Search scalability

If you load it explicitly, then all 800 MB will make it into RAM.
It's easy to try, the API for this is super simple.

Otis

--- yahootintin-lucene@yahoo.com wrote:

> Does it take 800MB of RAM to load that index into a RAMDirectory?  Or 
> are only some of the files loaded into RAM?
> 
> --- Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> 
> > Hello,
> > 
> > 100 parallel searches going against a single index on a single disk 
> > means a lot of disk seeks all happening at once.  One simple way of 
> > working around this is to load your FSDirectory into RAMDirectory.
> > This should be faster (could you report your 
> > observations/comparisons?).  You can also try using ramfs if you are

> > using Linux.
> > 
> > Otis
> > 
> > --- Ravi <ravi@htinc.com> wrote:
> > 
> > >  We have one large index for a document repository of
> > 800,000
> > > documents.
> > > The size of the index is 800MB. When we do searches against
> > the
> > > index,
> > > it takes 300-500ms for a single search. We wanted to test
> > the
> > > scalability and tried 100 parallel searches against the
> > index with
> > > the
> > > same query and the average response time was 13 seconds. We
> > used a
> > > simple IndexSearcher. Same searcher object was shared by all
> > the
> > > searches. I'm sure people have success in configuring lucene
> > for
> > > better
> > > scalability. Can somebody share their approach?
> > > 
> > > Thanks
> > > Ravi. 
> > > 
> > >
> >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> > > 
> > > 
> > 
> > 
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> > 
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message