lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin A. Burton" <bur...@newsmonster.org>
Subject Lucene HARD lock during indexing... BAD bug!?
Date Tue, 24 Feb 2004 20:08:12 GMT
I've recently noticed this major problem with our indexer.

It only seems to happen running on 1.3 final and not on 1.3 RC2.

Essentially we have about 80 threads doing indexing... they all add to a 
RAMDirectory which after 10k documents does a commit to disk.

This appears to be a problem with FSDirectory .. which was not modified 
in 1.3 final (same as in RC2)

If anyone has any suggestions they would be appreciated!

Here's the full stacktrace ...

"ksa-task-thread-32" prio=1 tid=0x89d04140 nid=0x4666 runnable 
[ba7ff000..ba7ff8d0]
        at java.io.RandomAccessFile.readBytes(Native Method)
        at java.io.RandomAccessFile.read(RandomAccessFile.java:307)
        at 
org.apache.lucene.store.FSInputStream.readInternal(FSDirectory.java:433)
        - locked <0x46df74a0> (a 
org.apache.lucene.store.FSInputStream$Descriptor)
        at org.apache.lucene.store.InputStream.refill(InputStream.java:196)
        at org.apache.lucene.store.InputStream.readByte(InputStream.java:81)
        at 
org.apache.lucene.store.InputStream.readVLong(InputStream.java:141)
        at 
org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:117)
        at 
org.apache.lucene.index.TermInfosReader.readIndex(TermInfosReader.java:110)
        at 
org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:82)
        at 
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:141)
        at 
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:120)
        at 
org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:118)
        at org.apache.lucene.store.Lock$With.run(Lock.java:148)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
        - locked <0x4ccc3790> (a org.apache.lucene.store.FSDirectory)
        at 
ksa.index.AdvancedIndexWriter.doDeleteByResource(AdvancedIndexWriter.java:264)
        at 
ksa.index.AdvancedIndexWriter.commit(AdvancedIndexWriter.java:172)
        - locked <0x497cfde0> (a java.lang.Object)
        at 
ksa.index.IndexManager.getAdvancedIndexWriter(IndexManager.java:282)
        - locked <0x497e0a50> (a java.lang.Object)
        at 
ksa.robot.FeedTaskParserListener.onItemEnd(FeedTaskParserListener.java:315)
        at 
org.apache.commons.feedparser.RSSFeedParser.doParseItem(RSSFeedParser.java:260)
        at 
org.apache.commons.feedparser.RSSFeedParser.parse(RSSFeedParser.java:126)
        at 
org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:146)
        at 
org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:113)
        at ksa.robot.FeedTask.doUpdate(FeedTask.java:375)
        at ksa.robot.FeedTask.doUpdate(FeedTask.java:331)
        at ksa.robot.FeedTask.run(FeedTask.java:147)
        at ksa.robot.TaskThread.doProcessTask(TaskThread.java:213)
        at ksa.robot.TaskThread.run(TaskThread.java:78)

-- 

Please reply using PGP:

    http://peerfear.org/pubkey.asc    

    NewsMonster - http://www.newsmonster.org/
    
Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
  IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster


Mime
View raw message