tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: [OT] Re: Lucene and DB speeds
Date Thu, 01 Mar 2007 16:10:50 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sriram,

>> When I query for large datasets, I see a delay between the search and
>> the results. For e.g., if I were to query a database containing about
>> 2 GB of data, with a 400+ MB Licene index, then I get search results
>> after anywhere between 5 seconds to 35 seconds.

400MB is not really that big of an index. A friend of mine runs a Lucene
index at the US Library of Congress that is several GB and they search
it /very/ quickly. Of course, they have some monstrous hardware, too.

>> Any tips on how I could test for where the search and retrieval is
>> taking time ?

Tim Funk wrote:
> A common way is to add timers around the commands and then using a
> logging library to output those times (under info, debug, or trace)

Lucens is a pretty opaque API, unfortunately. I'm guessing that if it's
taking up to 35 seconds to get a response from the index, there's not
much more benchmarking that could be done.

Then again, Sriram, you should check to make sure that the entire 35
seconds is taken up by Lucene calls. IS it possible that your Lucene
search is only part of the time? For instance, it is common practice to
search a Lucene index and then use those results to query a database. Is
it possible that other parts of your transaction are dominating the
wall-clock time?

> You'll probably get more help from the Lucene user list.

Absolutely. You've hit the Apache Tomcat mailing list, which is a web
application server. You didn't even mention that you were talking about
a web application ;)

Good luck,
- -chris

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFF5vsJ9CaO5/Lv0PARAmokAKCIpy0haQOVp7+I8Av3TFCb3R8kIACePEo2
WSWEuq6C77JhFdNa3pIrkho=
=GRQ2
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message