lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Core dumped
Date Thu, 27 May 2010 17:01:55 GMT
What have you set various indexwriter properties to? Particularly
things like merge factor, max buffered docs and ram buffer size.

The first thing I'd look at is MergeFactor. From the JavaDocs:
Determines how often segment indices are merged by addDocument(). With
smaller values, less RAM is used while indexing, and searches on unoptimized
indices are faster, but indexing speed is slower. With larger values, more
RAM is used during indexing, and while searches on unoptimized indices are
slower, indexing is faster. Thus larger values (> 10) are best for batch
index creation, and smaller values (< 10) for indices that are interactively
maintained.


HTH
Erick

On Thu, May 27, 2010 at 11:52 AM, Saurabh Agarwal <srbh.grwl@gmail.com>wrote:

> Hi,
> when I am running Lucene on a 512 MB system.
> I am getting the following error
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>        at
>
> org.apache.lucene.index.DocumentsWriter$ByteBlockAllocator.getByteBlock(DocumentsWriter.java:1206)
> and sometimes
>  An unexpected error has been detected by Java Runtime Environment:
> # Problematic frame:
> # J  java.nio.ByteBuffer.arrayOffset()I
>
>
> if I decrease my corpus size it indexes it perfectly!!!
>
> Can someone tell me how to index a larger corpus even if it takes more
> time!!!
>
> Thanks
>
> Saurabh Agarwal
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message