lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gag...@graffiti.net
Subject slow search threads during a disk copy
Date Mon, 23 Aug 2010 07:42:12 GMT

Hi all,


We're observing search threads slowing down during directory copies performed during updates
to the index. The thread dump shows search threads blocked on a FSDirectory$FSIndexInput$Descriptor
instance:



"Worker Thread - 12" daemon prio=10 tid=0x082b2400 nid=0x4654 waiting for monitor entry [0x988ed000..0x988edf30]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:542)
    - waiting to lock <0xc79c9900> (a org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor)
    at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
    at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
    at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76)
    at org.apache.lucene.index.SegmentTermDocs.read(SegmentTermDocs.java:133)
    at org.apache.lucene.index.MultiSegmentReader$MultiTermDocs.read(MultiSegmentReader.java:573)
    at org.apache.lucene.search.TermScorer.next(TermScorer.java:106)
    at org.apache.lucene.search.ConjunctionScorer.init(ConjunctionScorer.java:80)
    at org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:48)
    at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:319)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
    at org.apache.lucene.search.Searcher.search(Searcher.java:118)

    ...


The directory copy we do is a 'cp -pr <src-dir>/* <dest-dir>' command prior to
applying changes (addDocument calls) on the current "available" segment. This takes >7
mins to copy a directory of size 1.4G. During this time window, the searches are slow and
the above thread stacks are observed.


Could there be any system level limits we're hitting?  


Our test environment is:
lucene-2.3.2
4x2.6 GHz, 16G memory
Red Hat 3.4.6-9


Thanks and regards,


- Gagan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message