lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Fang...@emc.com>
Subject RE: exception to open a large index Insufficient system resources exist
Date Wed, 02 Sep 2009 02:49:22 GMT
32 bit JVM, 1.3G allocated heap size, Lucene 2.4.1.

In my option, this exception should not be caused by out of memory or out of system file handle
because different exception should be thrown for these two cases.

Any hint?

Thanks,
Fang, Li

-----Original Message-----
From: Uwe Schindler [mailto:uwe@thetaphi.de] 
Sent: Tuesday, September 01, 2009 6:14 PM
To: java-user@lucene.apache.org
Subject: RE: exception to open a large index Insufficient system resources exist

Which Lucene version, 64 bit JVM?

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Fang_Li@emc.com [mailto:Fang_Li@emc.com]
> Sent: Tuesday, September 01, 2009 12:04 PM
> To: java-user@lucene.apache.org
> Subject: RE: exception to open a large index Insufficient system resources
> exist
> 
> We are running on Windows 2003 Enterprise Edition with NTFS file system on
> a local disc. JDK version is 1.5.0.12.
> 
> The problem was discussed before and there is no clear solution confirmed.
> 
> Thanks.
> 
> -----Original Message-----
> From: Danil ŢORIN [mailto:torindan@gmail.com]
> Sent: Tuesday, September 01, 2009 5:56 PM
> To: java-user@lucene.apache.org
> Subject: Re: exception to open a large index Insufficient system resources
> exist
> 
> There should be no problem with large segments.
> Please describe OS, FileSystem and JDK you are running on.
> 
> There might be some problems with file >2Gb on Win32/FAT, or in some
> ancient Linuxes.
> 
> On Tue, Sep 1, 2009 at 12:37, <Fang_Li@emc.com> wrote:
> > I met a problem to open an index bigger than 8GB and the following
> > exception was thrown. There is a segment which is bigger than 4GB
> > already. After searching internet, it is said that not using compound
> > index may solve the problem.
> >
> > The same exception was thrown when merging with another index happens.
> > If the problem is caused by the big segment, we can adjust Lucene
> > parameters to control the segment size.
> > Anyone knows what's the cause for this exception? As reproducing the
> > problem will take a long time, your idea will be save us a lot of
> > effort.
> >
> > java.io.IOException: Insufficient system resources exist to complete the
> > requested service
> >        at java.io.RandomAccessFile.readBytes(Native Method)
> >        at java.io.RandomAccessFile.read(RandomAccessFile.java:315)
> >        at
> > org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> > y.java:596)
> >        at
> > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.jav
> > a:157)
> >        at
> > org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.j
> > ava:38)
> >        at
> > org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
> >        at
> > org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.jav
> > a:73)
> >        at
> > org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> >        at
> > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> >        at
> > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:228)
> >        at
> > org.apache.lucene.index.MultiSegmentReader.<init>(MultiSegmentReader.jav
> > a:55)
> >        at
> > org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexRead
> > er.java:104)
> >        at
> > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.j
> > ava:653)
> >        at
> > org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.j
> > ava:115)
> >        at
> > org.apache.lucene.index.IndexReader.open(IndexReader.java:316)
> >        at
> > org.apache.lucene.index.IndexReader.open(IndexReader.java:227)
> >
> > Thanks,
> > -Li
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message