Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: error (idunn.apache.osuosl.org: domain ix.netcom.com from
 209.86.89.70 cause and error)
DomainKey-Status: good
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=dk20050327; d=ix.netcom.com;
  b=t2cwBn9ENCbX0CJR1JiVwT8pR03cuU9iP957O2rcuu1T9QawDzoDeo8l4+D3JejN;
  h=Received:Mime-Version:In-Reply-To:References:Content-Type:Message-Id:Content-Transfer-Encoding:From:Subject:Date:To:X-Mailer:X-ELNK-Trace:X-Originating-IP;
Mime-Version: 1.0 (Apple Message framework v752.2)
In-Reply-To: <200609111914.00062.paul.elschot@xs4all.nl>
References: <4504ACA4.6090309@manawiz.com>
 <c68e39170609110636i22373c01ud487b8653d50a4e9@mail.gmail.com>
 <200609111914.00062.paul.elschot@xs4all.nl>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <71382EBC-0231-4D1D-8291-D4389CE9462E@ix.netcom.com>
Content-Transfer-Encoding: 7bit
From: robert engels <rengels@ix.netcom.com>
Subject: Re: After kill -9 index was corrupt
Date: Mon, 11 Sep 2006 12:34:22 -0500
To: java-dev@lucene.apache.org

A kill -9 should not affect the OS's writing of dirty buffers  
(including directory modifications). If this were the case, massive  
system corruption would almost always occur every time a kill -9 was  
used with any program.

The only thing a kill -9 affects is user level buffering. The OS  
always maintains a consistent view of directory modifications and or  
file modification that were requesting by programs.

This entire discussion is pointless.

If the hardware is performing lazy writes, then corruption might be  
caused during a hard-reset (e.g. power failure, hardware failure (cpu  
lockup), or device driver failure in some VERY RARE cases). Even a  
kernel panic should allow the physical devices to flush their dirty  
buffers (as the controller can in many cases detect this).

The only way to prevent this is if the OS exposes a system call to  
synchronously force the hardware to flush any buffers. The Java fsync 
() could use the OS call to peform this operation, but most likely  
would not since the performance penalty would be significant, and  
those users requiring this level of reliability would be better  
served by using UPSs and fault-tolerant systems (at every level -  
CPU, disk, etc.).


On Sep 11, 2006, at 12:13 PM, Paul Elschot wrote:

> On Monday 11 September 2006 15:36, Yonik Seeley wrote:
>> On 9/10/06, Chuck Williams <chuck@manawiz.com> wrote:
>>> Could a kill -9 prevent data from reaching disk for files that were
>>> previously closed?
>>
>> No.  After a close() the OS should have all the data... the process
>> may be killed but the OS will eventually flush all the buffers, etc.
>> File creation is pretty much always synchronous so I have no idea how
>> your problem could have happened (missing segment files).  IO  
>> error or
>> something else temporarily filling up the disk?
>
> "Pretty much always" is logically equivalent to "not always" :) ,  
> see also
> below.
>
>> If you have a power loss or crash, then that *can* cause data loss.
>> There may be mount options to make more file operations synchronous,
>> or you could maybe write your own Directory implementation to make
>> things more synchronous.
>
> New segments will have to be flushed/fsynced before the segments
> file. This could be hidden in a Directory in case a Directory hides  
> Lucene's
> segments file.
>
> Regards,
> Paul Elschot
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org