lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shivender Devarakonda (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2361) OutOfMemoryException while Indexing
Date Sat, 03 Apr 2010 16:25:27 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853154#action_12853154
] 

Shivender Devarakonda commented on LUCENE-2361:
-----------------------------------------------

Thanks for helping us on this.

We create a Daemon Thread that does the index updating functionality.

1-  Create a IndexDir like this:
 fDirectoryName = new File(tracesDir, relativeIndexDir);
        fIndexDir = FSDirectory.open(fDirectoryName, NoLockFactory.getNoLockFactory());
        
        fIndexDir.setReadChunkSize(8192);



2- The Index inserter thread does the following functionality, The new writer is invoked for
a batch size of 100.

  		IndexWriter writer = null;
                
                try
                {
                    writer = new IndexWriter(fIndexDir, this.getAnalyzer(), !this.indexExists(),
IndexWriter.MaxFieldLength.LIMITED);
                    writer.setMaxBufferedDocs(batchSize);
                    writer.addDocument(doc);

                    // while we're open and there's more in the queue, add more
                    numInserted += addMore(writer, luceneQueue, batchSize);
                }
                finally
                {
                    Utils.close(writer);
                    insertionTime = MasterClock.currentTimeMillis() - startTime;
                }

We do not set any RAMBufferSize in this case.

Please let me know if you need anything on this.

> OutOfMemoryException while Indexing
> -----------------------------------
>
>                 Key: LUCENE-2361
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2361
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.9.1
>         Environment: Windows
>            Reporter: Shivender Devarakonda
>
> Hi,
> We use lucene 2.9.1 version.
> We see the following OutOfMemory error in our environment, I think This is happening
at a significant high load. Have you observed this anytime? Please let me know your thoughts
on this.
> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: PermGen
space
> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> Caused by: java.lang.OutOfMemoryError: PermGen space
> 	at java.lang.String.$$YJP$$intern(Native Method)
> 	at java.lang.String.intern(Unknown Source)
> 	at org.apache.lucene.util.SimpleStringInterner.intern(SimpleStringInterner.java:74)
> 	at org.apache.lucene.util.StringHelper.intern(StringHelper.java:36)
> 	at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:356)
> 	at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
> 	at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:116)
> 	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:638)
> 	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:608)
> 	at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:686)
> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4979)
> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614)
> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message