lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Agarwal <srbh.g...@gmail.com>
Subject Re: Core dumped
Date Thu, 27 May 2010 17:52:00 GMT
I will check it out!!
Saurabh Agarwal


On Thu, May 27, 2010 at 11:13 PM, Erick Erickson <erickerickson@gmail.com>wrote:

> The larger your RAMbufferSize, the more memory you consume FWIW.
>
> OK, then, does it always OOM on the same document? Are you trying to index
> any particularly large documents?
>
> Erick
>
> On Thu, May 27, 2010 at 1:28 PM, Saurabh Agarwal <srbh.grwl@gmail.com
> >wrote:
>
> > RAMBufferSize id 50 Mb, i tried with 200 too
> > the index is unoptimized
> > MergeFactor is Default 10 and I have not changed it
> >
> > MaxBuffered Docs is also default
> > Saurabh Agarwal
> >
> >
> > On Thu, May 27, 2010 at 10:31 PM, Erick Erickson <
> erickerickson@gmail.com
> > >wrote:
> >
> > > What have you set various indexwriter properties to? Particularly
> > > things like merge factor, max buffered docs and ram buffer size.
> > >
> > > The first thing I'd look at is MergeFactor. From the JavaDocs:
> > > Determines how often segment indices are merged by addDocument(). With
> > > smaller values, less RAM is used while indexing, and searches on
> > > unoptimized
> > > indices are faster, but indexing speed is slower. With larger values,
> > more
> > > RAM is used during indexing, and while searches on unoptimized indices
> > are
> > > slower, indexing is faster. Thus larger values (> 10) are best for
> batch
> > > index creation, and smaller values (< 10) for indices that are
> > > interactively
> > > maintained.
> > >
> > >
> > > HTH
> > > Erick
> > >
> > > On Thu, May 27, 2010 at 11:52 AM, Saurabh Agarwal <srbh.grwl@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > > when I am running Lucene on a 512 MB system.
> > > > I am getting the following error
> > > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> space
> > > >        at
> > > >
> > > >
> > >
> >
> org.apache.lucene.index.DocumentsWriter$ByteBlockAllocator.getByteBlock(DocumentsWriter.java:1206)
> > > > and sometimes
> > > >  An unexpected error has been detected by Java Runtime Environment:
> > > > # Problematic frame:
> > > > # J  java.nio.ByteBuffer.arrayOffset()I
> > > >
> > > >
> > > > if I decrease my corpus size it indexes it perfectly!!!
> > > >
> > > > Can someone tell me how to index a larger corpus even if it takes
> more
> > > > time!!!
> > > >
> > > > Thanks
> > > >
> > > > Saurabh Agarwal
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message