lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anshum <ansh...@gmail.com>
Subject Re: slow search threads during a disk copy
Date Mon, 23 Aug 2010 07:47:04 GMT
Seems like a case of I/O issues. You may be reading content off the index
while performing searches while the I/O for copy is also happening.

--
Anshum Gupta
http://ai-cafe.blogspot.com


On Mon, Aug 23, 2010 at 1:12 PM, <gaganb@graffiti.net> wrote:

>
> Hi all,
>
>
> We're observing search threads slowing down during directory copies
> performed during updates to the index. The thread dump shows search threads
> blocked on a FSDirectory$FSIndexInput$Descriptor instance:
>
>
>
> "Worker Thread - 12" daemon prio=10 tid=0x082b2400 nid=0x4654 waiting for
> monitor entry [0x988ed000..0x988edf30]
>   java.lang.Thread.State: BLOCKED (on object monitor)
>    at
> org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:542)
>    - waiting to lock <0xc79c9900> (a
> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor)
>    at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
>    at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
>    at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76)
>    at
> org.apache.lucene.index.SegmentTermDocs.read(SegmentTermDocs.java:133)
>    at
> org.apache.lucene.index.MultiSegmentReader$MultiTermDocs.read(MultiSegmentReader.java:573)
>    at org.apache.lucene.search.TermScorer.next(TermScorer.java:106)
>    at
> org.apache.lucene.search.ConjunctionScorer.init(ConjunctionScorer.java:80)
>    at
> org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:48)
>    at
> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:319)
>    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
>    at org.apache.lucene.search.Searcher.search(Searcher.java:118)
>
>    ...
>
>
> The directory copy we do is a 'cp -pr <src-dir>/* <dest-dir>' command prior
> to applying changes (addDocument calls) on the current "available" segment.
> This takes >7 mins to copy a directory of size 1.4G. During this time
> window, the searches are slow and the above thread stacks are observed.
>
>
> Could there be any system level limits we're hitting?
>
>
> Our test environment is:
> lucene-2.3.2
> 4x2.6 GHz, 16G memory
> Red Hat 3.4.6-9
>
>
> Thanks and regards,
>
>
> - Gagan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message