lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Sitsky <s...@nuix.com>
Subject Re: java.io.IOException: background merge hit exception: Caused by: java.io.FileNotFoundException: _1vw.fdt
Date Wed, 28 Jul 2010 05:04:36 GMT
Hi Mike,

Definitely no OOME (or other critical errors), and if there were, we
would have terminated the program straight away.

Cheers,
David

On 28 July 2010 01:44, Michael McCandless <lucene@mikemccandless.com> wrote:
> Were there any exceptions during indexing, before the
> TerminateProcess() call?  EG OOME?
>
> Mike
>
> On Tue, Jul 27, 2010 at 8:43 AM, Michael McCandless
> <lucene@mikemccandless.com> wrote:
>> On Mon, Jul 26, 2010 at 7:10 PM, David Sitsky <sits@nuix.com> wrote:
>>> During processing.. there might be a number of reasons why we need to
>>> shutdown the indexing process, but perhaps what is unusual is we call
>>> the win32 API TerminateProcess() call rather than System.exit(), for
>>> slightly obscure reasons.  When calling exit(), this still calls a
>>> large body of code (for example dll shutdown hooks) which in some
>>> situations, we found could "hang" the exiting process, which was a
>>> problem for us.
>>>
>>> In a sense, this should be no different to killing the process under
>>> Windows using the task manager, or kill -9 on unix systems.
>>
>> OK.  Lucene should be fine after a kill -9 or a TerminateProcess,
>> assuming Windows really does act like Unix and any "committed" IO
>> operations done by the process prior to the kill are in fact committed
>> to the filesystem.
>>
>>> At no time did the machine itself crash, and the disk involved (I'm
>>> told) was a local RAID filesystem.  I am guessing the disk has write
>>> caching enabled, but given the machine didn't crash, this shouldn't
>>> matter.
>>
>> Right, the write caching shouldn't matter since the machine didn't go down.
>>
>>> Something else that is slightly unusual is we explicitly call commit()
>>> at certain times to flush indexing work to disk.
>>
>> OK that's fine.
>>
>>> Its interesting in both instances, CheckIndex said there was 1 broken
>>> segment containing 1 document.
>>
>> Yeah that is curious.... I'll try to mull.
>>
>> Any chance you could run with an infoStream set on IndexWriter?  Then
>> if this happens again I can pour over that...
>>
>> Mike
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>



-- 
Cheers,
David

Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia    Ph: +61 2 9280 0699
Web: http://www.nuix.com                            Fax: +61 2 9212 6902

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message