lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: OOM during commit - Lucene 3.6
Date Mon, 30 Sep 2013 20:30:45 GMT
The JVM is crashing right?  (Not an OOMException)?

This must be a bug in Jrockit.  If you use a different JVM do you
still see the crash?

Mike McCandless

http://blog.mikemccandless.com


On Mon, Sep 30, 2013 at 2:39 PM, Saurabh Sethi
<Saurabh_Sethi@symantec.com> wrote:
> Hi,
>
> I have a method that deletes entries from Lucene and calls IndexWriter.commit() in the
end. This method is called multiple times and each time the batch size varies from 5-20 items.
>
> After some deletions, commit() causes JVM to crash. I am using Lucene 3.6 and JDK is
jrockit-jdk-1.6.0_22-R28.1.1-4.0.1-x64.
>
> Following is the partial JVM dump:
>
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030) Roots Dump Helper:
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              thread=0x00000000681A2530
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)                 method=org/apache/lucene/index/IndexWriter.prepareCommit(Ljava/util/Map;)V
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              sp=0x000000006AC4D5D0
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              ip=0x000000002088CC8A
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              ref=0x000000006AC4D5A0
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              *ref=0x00000003D914CD81
(interior pointer to 0x00000003D914CD80)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              stackIter=0x000000005F7DFD00
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              refIter=0x000000005F7DFD58
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              storage=rbp
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030) Roots Processed Thread: "Email
Batch Committer 2" id=251 idx=0x21c tid=21388 lastJavaFrame=0x000000006AC4D2C0
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at jrockit/vm/Allocator.getNewTla(JJ)V(Native
Method)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at jrockit/vm/Allocator.allocObjectOrArray(Allocator.java:354)[optimized]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at java/util/HashMap.<init>(HashMap.java:253)[inlined]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at java/util/HashSet.<init>(HashSet.java:86)[optimized]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/SegmentInfos.files(SegmentInfos.java:836)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexFileDeleter.incRef(IndexFileDeleter.java:480)[inlined]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexFileDeleter.checkpoint(IndexFileDeleter.java:453)[optimized]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter$ReaderPool.commit(IndexWriter.java:648)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.prepareCommit(IndexWriter.java:3383)[optimized]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.commitInternal(IndexWriter.java:3491)[optimized]
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.commit(IndexWriter.java:3473)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.commit(IndexWriter.java:3457)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/TextIndexWriter.commit(TextIndexWriter.java:348)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/LocalTextIndexImpl.flush(LocalTextIndexImpl.java:483)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/LocalTextIndexImpl.commitDeletes(LocalTextIndexImpl.java:369)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/LocalTextIndexImpl.deleteByIDs(LocalTextIndexImpl.java:280)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/TextHelper.removeEsaDocIDs(TextHelper.java:298)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/indexer/EmailBatchCommitter.deleteDerivedFTDocs(EmailBatchCommitter.java:2004)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
> 2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/indexer/EmailBatchCommitter.commitPartitionDocuments(EmailBatchCommitter.java:1291)[optimized]
>
> Would appreciate any help on this.
>
> Thanks,
> Saurabh

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message