lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kaktu Chakarabati <jimmoe...@gmail.com>
Subject Severe QTime issues to do with updates
Date Sun, 09 Aug 2009 21:30:45 GMT
Hey,
I've recently noticed that there is a very large spike in the QTime for
nodes serving queries, immediately after snappulling and snapinstalling.
The numbers i'm seeing there are obviously some kind of
lock-contention/concurrency issue, as I've monitored iostat/sar and its not
a disk IO
issue ( all the index is still mostly in the os caches, as is also
noticeable in the snappuller.log which runs very fast - the incremental
update to the
lucene index is minimal).
I'm using a strong machine (16GB 2xQuadCore, CentOS5) for this and from all
monitoring the CPU/Disk IO seems minimal through-out the day.
The update runs every 10 minutes, takes about 200seconds to complete (with
my current autoWarm settings)., and the index size
is about 20million documents (~6GB). queries/warmup involve mostly some
pre-fixed set of filters and facets.

I'm using a nightly build of solr from few months back ( nightly exported -
yonik - 2009-01-11 08:05:52 )  which should already have
the benefits of readonlyindexreader, NIOFS, UnInvertedIndex and
ConcurrentLRUCache (for filterCache).
I've read a related thread a guy called oleg posted (
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200901.mbox/%3C339777.57289.qm@web50307.mail.re2.yahoo.com%3E)
, but did not see the thread concluded with any definite conclusion.

Please advise!

Best regards,
-Chak

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message