lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Fang...@emc.com>
Subject RE: exception to open a large index Insufficient system resources exist
Date Tue, 01 Sep 2009 10:03:32 GMT
We are running on Windows 2003 Enterprise Edition with NTFS file system on a local disc. JDK
version is 1.5.0.12.

The problem was discussed before and there is no clear solution confirmed.

Thanks.

-----Original Message-----
From: Danil ŢORIN [mailto:torindan@gmail.com] 
Sent: Tuesday, September 01, 2009 5:56 PM
To: java-user@lucene.apache.org
Subject: Re: exception to open a large index Insufficient system resources exist

There should be no problem with large segments.
Please describe OS, FileSystem and JDK you are running on.

There might be some problems with file >2Gb on Win32/FAT, or in some
ancient Linuxes.

On Tue, Sep 1, 2009 at 12:37, <Fang_Li@emc.com> wrote:
> I met a problem to open an index bigger than 8GB and the following
> exception was thrown. There is a segment which is bigger than 4GB
> already. After searching internet, it is said that not using compound
> index may solve the problem.
>
> The same exception was thrown when merging with another index happens.
> If the problem is caused by the big segment, we can adjust Lucene
> parameters to control the segment size.
> Anyone knows what's the cause for this exception? As reproducing the
> problem will take a long time, your idea will be save us a lot of
> effort.
>
> java.io.IOException: Insufficient system resources exist to complete the
> requested service
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(RandomAccessFile.java:315)
>        at
> org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> y.java:596)
>        at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.jav
> a:157)
>        at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.j
> ava:38)
>        at
> org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
>        at
> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.jav
> a:73)
>        at
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
>        at
> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
>        at
> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:228)
>        at
> org.apache.lucene.index.MultiSegmentReader.<init>(MultiSegmentReader.jav
> a:55)
>        at
> org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexRead
> er.java:104)
>        at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.j
> ava:653)
>        at
> org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.j
> ava:115)
>        at
> org.apache.lucene.index.IndexReader.open(IndexReader.java:316)
>        at
> org.apache.lucene.index.IndexReader.open(IndexReader.java:227)
>
> Thanks,
> -Li
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message