Return-Path: Delivered-To: apmail-lucene-java-dev-archive@www.apache.org Received: (qmail 17885 invoked from network); 11 Sep 2006 13:57:52 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 11 Sep 2006 13:57:52 -0000 Received: (qmail 33516 invoked by uid 500); 11 Sep 2006 13:57:50 -0000 Delivered-To: apmail-lucene-java-dev-archive@lucene.apache.org Received: (qmail 33472 invoked by uid 500); 11 Sep 2006 13:57:50 -0000 Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-dev@lucene.apache.org Delivered-To: mailing list java-dev@lucene.apache.org Received: (qmail 33461 invoked by uid 99); 11 Sep 2006 13:57:50 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Sep 2006 06:57:50 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: local policy) Received: from [66.111.4.26] (HELO out2.smtp.messagingengine.com) (66.111.4.26) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Sep 2006 06:57:49 -0700 Received: from frontend3.internal (frontend3.internal [10.202.2.152]) by frontend1.messagingengine.com (Postfix) with ESMTP id EF061DA5CC4 for ; Mon, 11 Sep 2006 09:57:26 -0400 (EDT) Received: from heartbeat2.internal ([10.202.2.161]) by frontend3.internal (MEProxy); Mon, 11 Sep 2006 09:57:28 -0400 X-Sasl-enc: FS3YAk0Dt7D3Ajw/WJbOHApprhnv0QuwTxDVyAUDJhIG 1157983047 Received: from [10.17.4.90] (pool-71-162-112-159.bstnma.fios.verizon.net [71.162.112.159]) by mail.messagingengine.com (Postfix) with ESMTP id AC8F31CD3 for ; Mon, 11 Sep 2006 09:57:27 -0400 (EDT) Message-ID: <45056B37.1090505@mikemccandless.com> Date: Mon, 11 Sep 2006 09:57:11 -0400 From: Michael McCandless User-Agent: Thunderbird 1.5.0.5 (Windows/20060719) MIME-Version: 1.0 To: java-dev@lucene.apache.org Subject: Re: After kill -9 index was corrupt References: <4504ACA4.6090309@manawiz.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Yonik Seeley wrote: > On 9/10/06, Chuck Williams wrote: >> Could a kill -9 prevent data from reaching disk for files that were >> previously closed? > > No. After a close() the OS should have all the data... the process > may be killed but the OS will eventually flush all the buffers, etc. > File creation is pretty much always synchronous so I have no idea how > your problem could have happened (missing segment files). IO error or > something else temporarily filling up the disk? > > If you have a power loss or crash, then that *can* cause data loss. > There may be mount options to make more file operations synchronous, > or you could maybe write your own Directory implementation to make > things more synchronous. Agreed ... it's hard to explain how this could have occurred without the OS / machine actually going down abruptly. Is this all on a single machine, local filesystem? Are you really sure your underlying IO system is "healthy", no silent file system corruption going on or anything? However, I do think it would be a good idea to [optionally] add a sync() call on committing the segments file to still be robust to OS / machine crashing... it would slow down performance of indexing but hopefully not by too much since the segments file is small. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org For additional commands, e-mail: java-dev-help@lucene.apache.org