lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Smith <ssm...@mainstreamdata.com>
Subject RE: Lucene slow performance
Date Fri, 15 Mar 2013 22:58:38 GMT
Here's the code for the writer:

        IndexWriterConfig iwc = new IndexWriterConfig(Constants.LUCENE_VERSION, _analyzer);
        LogByteSizeMergePolicy lbsm = new LogByteSizeMergePolicy();
        lbsm.setUseCompoundFile(true);
        iwc.setMergePolicy(lbsm);
        Directory fsDir = FSDirectory.open(new File(_IndexDirectory));
        writer = new IndexWriter(fsDir, iwc);

I don't use NRT.  I share an IndexReader among the web sessions.  However, if the index has
been updated since the last time I used it, I will go get another reader.  For this particular
server, the documents are getting updated very frequently.  It would not be "strange" (from
the customer's perspective) if a document received today was updated 10-20 times before the
end of the day and we probably get 2k-3k documents per day.

-----Original Message-----
From: Simon Willnauer [mailto:simon.willnauer@gmail.com] 
Sent: Friday, March 15, 2013 4:45 PM
To: java-user@lucene.apache.org
Subject: Re: Lucene slow performance

Can you tell us a little more about how you use lucene, how do you index, do you use NRT or
do you open an IndexReader for every request, do you maybe us a custom merge policy or somthing
like this, any special IndexWriter settings?

On Fri, Mar 15, 2013 at 11:15 PM, Scott Smith <ssmith@mainstreamdata.com> wrote:
> We have a system that is using lucene and the searches are very slow.  The number of
documents is fairly small (less than 30,000) and each document is typically only 2 to 10 kilo-characters.
 Yet, searches are taking 15-16 seconds.
>
> One of the things I noticed was that the index directory has several thousand (3000+)
.cfs files.  We do optimize the index once per day.  This is a system that probably gets several
thousand document deletes and additions per day (spread out across the day).
>
> Any thoughts.  We didn't really notice this until we went to 4.x.

what do you mean what didn't you notice, the slowness or the CFS files?

simon
>
> Scott
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Mime
View raw message