lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From subwayne <labrassband...@googlemail.com>
Subject Re: IndexSearch very slow after reopening the index
Date Thu, 14 Oct 2010 10:03:49 GMT

Hi Ian,

thank you for your quick response. I am running Lucene on Ubuntu 10.04, 64
bit. I switched from MMapDirectory to NIOFSDirectory without any significant
changes in performance. The Lucene version running is 3.0.2. I followed your
advice and opened the IndexSearcher after I added all documents to the
index. Now, the search performance is slow, too.

I am indexing a vast amount of the Open Directory Project; in particular
60,000 categories with about 1,400,000 documents. Each document is
represented by the short description made available bei ODP. Finally, the
index on hard disk is rather small---408 MBytes.

Regarding the NumericRangeQuery: It's true that I am only interested in an
exact match. However I found on some forum, that one can use the
NumericRangeQuery for exact matches as well without any performance loss.
Nevertheless, I am willing to learn how such an exact match can be don more
efficiently.

While searching with Lucene, I pose about 100 queries like in my first post
for each category.

Last but not least, I installed VisualVM as an Eclipse Plugin. But I am not
familiar with it and need to play with it a little bit.


Ian Lea wrote:
> 
> Do the fast searches that you get while the app is running use the
> searcher you create before you add all the docs to the index?  Surely
> that won't see the added docs.
> 
> There are general tips on speeding up searches at
> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed.  There are
> some gotchas with MMapDirectory depending on your OS and whether you
> are 32 or 64 bit - see the javadocs.  What are you running? What
> happens when you use a standard disk based directory rather than MMap?
>  How many docs are you adding?  How big is the index?  What version of
> lucene are you using?
> 
> Your NumericRangeQuery doesn't look much like a range but I doubt
> that's the problem.
> 
> Finally, you could run a profiler to see where the time is being spent.
> 
> 
> --
> Ian.
> 
-- 
View this message in context: http://lucene.472066.n3.nabble.com/IndexSearch-very-slow-after-reopening-the-index-tp1699711p1699956.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message