lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manish Joshi <mjo...@zedo.com>
Subject Re: Insuffient resources during optimize merge
Date Fri, 03 Jul 2009 10:33:59 GMT
I assume it must be memory related issue.Not sure just a guess.
-Manish

On Fri, Jul 3, 2009 at 3:42 PM, Carl Austin <Carl.austin@detica.com> wrote:

> Hi,
>
> I have an issue when optimizing a large index (40GB). Basically the
> optimize falls over with an IOException stating there are not enough
> system resources, within a RandomAccessFile.readBytes call followed back
> to the CompoundFileWriter close method. I assume this is when creating a
> new CFS file from the separate other files.
> Unfortunately there is no hint at which system resource, and I have no
> access to the machine this fails on, otherwise I would run some testing
> to see what resource usage is going on.
> The process running this, has 1.3GB of memory allowance.
>
> If anyone has any insights they would be much appreciated. Stack trace
> included now:
>
> Exception in thread "Lucene Merge Thread #0"
> org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException:
> Insufficient system resources exist to complete the requested service
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(Co
> ncurrentMergeScheduler.java:309)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:286)
> Caused by: java.io.IOException: Insufficient system resources exist to
> complete the requested service
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(Unknown Source)
>        at
> org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> y.java:596)
>        at
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.
> java:136)
>        at
> org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j
> ava:228)
>        at
> org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java
> :184)
>        at
> org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j
> ava:204)
>        at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260)
>        at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge
> Scheduler.java:205)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:260)
> Exception in thread "Thread-12957887" java.lang.RuntimeException:
> java.io.IOException: background merge hit exception: _cx:c3072736
> _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791 _dn:c4045 _do:c3643
> _dp:c3197 into _dq [optimize]
>        at a.b.c.d.e.f.ImportThread$ImportRunner.run(Unknown Source)
>        at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: background merge hit exception:
> _cx:c3072736 _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791
> _dn:c4045 _do:c3643 _dp:c3197 into _dq [optimize]
>        at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2258)
>        at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2203)
>        at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2183)
>        ... 2 more
> Caused by: java.io.IOException: Insufficient system resources exist to
> complete the requested service
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(Unknown Source)
>        at
> org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> y.java:596)
>        at
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.
> java:136)
>        at
> org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j
> ava:228)
>        at
> org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java
> :184)
>        at
> org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j
> ava:204)
>        at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260)
>        at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge
> Scheduler.java:205)
>        at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:260)
>
> Thanks
>
> Carl Austin
>
>
>
>
>
> This message should be regarded as confidential. If you have received this
> email in error please notify the sender and destroy it immediately.
> Statements of intent shall only become binding when confirmed in hard copy
> by an authorised signatory.  The contents of this email may relate to
> dealings with other companies within the Detica Group Limited group of
> companies.
>
> Detica Group Limited is registered in England under No: 03328242.
>
> Registered offices: Warwick House, PO Box 87, Farnborough Aerospace Centre,
> Farnborough, Hampshire, GU14 6YU, England.
>
>


-- 
Manish Joshi
Adserving Team
Zedo India

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message