lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Insuffient resources during optimize merge
Date Fri, 03 Jul 2009 10:51:27 GMT
Hi


Google throws up some hits for this error relating to obscure problems
on Windows with UNC paths and shared rather than local storage.  If
that's not relevant it might help to know what OS you are on, and
versions of lucene and java.

As a workaround you could perhaps try not using the compound file format.



--
Ian.



On Fri, Jul 3, 2009 at 11:12 AM, Carl Austin<Carl.austin@detica.com> wrote:
> Hi,
>
> I have an issue when optimizing a large index (40GB). Basically the
> optimize falls over with an IOException stating there are not enough
> system resources, within a RandomAccessFile.readBytes call followed back
> to the CompoundFileWriter close method. I assume this is when creating a
> new CFS file from the separate other files.
> Unfortunately there is no hint at which system resource, and I have no
> access to the machine this fails on, otherwise I would run some testing
> to see what resource usage is going on.
> The process running this, has 1.3GB of memory allowance.
>
> If anyone has any insights they would be much appreciated. Stack trace
> included now:
>
> Exception in thread "Lucene Merge Thread #0"
> org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException:
> Insufficient system resources exist to complete the requested service
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(Co
> ncurrentMergeScheduler.java:309)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:286)
> Caused by: java.io.IOException: Insufficient system resources exist to
> complete the requested service
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(Unknown Source)
>        at
> org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> y.java:596)
>        at
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.
> java:136)
>        at
> org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j
> ava:228)
>        at
> org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java
> :184)
>        at
> org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j
> ava:204)
>        at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260)
>        at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge
> Scheduler.java:205)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:260)
> Exception in thread "Thread-12957887" java.lang.RuntimeException:
> java.io.IOException: background merge hit exception: _cx:c3072736
> _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791 _dn:c4045 _do:c3643
> _dp:c3197 into _dq [optimize]
>        at a.b.c.d.e.f.ImportThread$ImportRunner.run(Unknown Source)
>        at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: background merge hit exception:
> _cx:c3072736 _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791
> _dn:c4045 _do:c3643 _dp:c3197 into _dq [optimize]
>        at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2258)
>        at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2203)
>        at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2183)
>        ... 2 more
> Caused by: java.io.IOException: Insufficient system resources exist to
> complete the requested service
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(Unknown Source)
>        at
> org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> y.java:596)
>        at
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.
> java:136)
>        at
> org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j
> ava:228)
>        at
> org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java
> :184)
>        at
> org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j
> ava:204)
>        at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260)
>        at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge
> Scheduler.java:205)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:260)
>
> Thanks
>
> Carl Austin

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message