lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: IndexSearch very slow after reopening the index
Date Thu, 14 Oct 2010 10:29:14 GMT
OK, so it looks like we're down to a more general "why is searching
slow" question.

The number of docs is not very large by lucene standards.

Work through http://wiki.apache.org/lucene-java/ImproveSearchingSpeed.

If that still doesn't help, pick a slow query and post again with:

. the output of query.toString()
. the number of docs matched
. how long it took first time on new searcher, and second and third
and ... tenth
. ummm... anything else you think might be relevant.

Good luck.


--
Ian.

On Thu, Oct 14, 2010 at 11:03 AM, subwayne
<labrassbandito@googlemail.com> wrote:
>
> Hi Ian,
>
> thank you for your quick response. I am running Lucene on Ubuntu 10.04, 64
> bit. I switched from MMapDirectory to NIOFSDirectory without any significant
> changes in performance. The Lucene version running is 3.0.2. I followed your
> advice and opened the IndexSearcher after I added all documents to the
> index. Now, the search performance is slow, too.
>
> I am indexing a vast amount of the Open Directory Project; in particular
> 60,000 categories with about 1,400,000 documents. Each document is
> represented by the short description made available bei ODP. Finally, the
> index on hard disk is rather small---408 MBytes.
>
> Regarding the NumericRangeQuery: It's true that I am only interested in an
> exact match. However I found on some forum, that one can use the
> NumericRangeQuery for exact matches as well without any performance loss.
> Nevertheless, I am willing to learn how such an exact match can be don more
> efficiently.
>
> While searching with Lucene, I pose about 100 queries like in my first post
> for each category.
>
> Last but not least, I installed VisualVM as an Eclipse Plugin. But I am not
> familiar with it and need to play with it a little bit.
>
>
> Ian Lea wrote:
>>
>> Do the fast searches that you get while the app is running use the
>> searcher you create before you add all the docs to the index?  Surely
>> that won't see the added docs.
>>
>> There are general tips on speeding up searches at
>> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed.  There are
>> some gotchas with MMapDirectory depending on your OS and whether you
>> are 32 or 64 bit - see the javadocs.  What are you running? What
>> happens when you use a standard disk based directory rather than MMap?
>>  How many docs are you adding?  How big is the index?  What version of
>> lucene are you using?
>>
>> Your NumericRangeQuery doesn't look much like a range but I doubt
>> that's the problem.
>>
>> Finally, you could run a profiler to see where the time is being spent.
>>
>>
>> --
>> Ian.
>>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/IndexSearch-very-slow-after-reopening-the-index-tp1699711p1699956.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message