Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of markrmiller@gmail.com
 designates 64.233.170.186 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=f6sowpnTLY920TkJ0aHgezpfxtSzyGs/jSJ3O5REtTOIGthX95InEVAEN+G8nlmjpT
         QD8D+Y2HFa4bkHmMoyC1qSvrVai9ZPAf/s8Fi8f+J/7/pcfOKzNBSmQuCGxyvFYSoLIE
         uPg/SowP4Aca8NXiYNviT3bBAG4vSqcWrz76I=
Message-ID: <49244922.3000605@gmail.com>
Date: Wed, 19 Nov 2008 12:13:06 -0500
From: Mark Miller <markrmiller@gmail.com>
User-Agent: Thunderbird 2.0.0.17 (X11/20080925)
MIME-Version: 1.0
To: java-dev@lucene.apache.org
Subject: Re: Option to  fsync files
References: <49242377.3050204@gmail.com>
 <8C44F3C4-4294-42F2-B3C1-2DF8C11C3838@mikemccandless.com>
 <49243F48.5090406@gmail.com>
 <2E7EC816-349A-415D-8EB3-F57F192C73D6@mikemccandless.com>
 <49244557.2010905@gmail.com>
 <53532008-DE80-430C-8ADE-47519C44A564@ix.netcom.com>
In-Reply-To: <53532008-DE80-430C-8ADE-47519C44A564@ix.netcom.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Okay, I'll admit I am trusting some else testing this. One of the java 
database implementations (hsql or something?) talks about it and tests 
it it to be the case also with derby and other transaction systems. 
Corruption didn't appear that hard for him to lure out at all. Also, if 
you google slashdot your harddrive lies to you, there is more testing of 
it. I have also seen bits or pieces about it elsewhere.

I choose to believe myself, but I will admit I was 100% wrong about 
santa clause, so take it for what its worth. I havn't tested it at all.

robert engels wrote:
> I would really like to see some PROOF of these drives "lying".
>
> If that were the case, no database system would ever be reliable on 
> these drives !  And data corruption would be happening all over the 
> place !
>
> On Nov 19, 2008, at 10:56 AM, Mark Miller wrote:
>
>> Michael McCandless wrote:
>>>
>>> Mark Miller wrote:
>>>
>>>> It is dangerous, but probably not any more dangerous than using a 
>>>> consumer hard drive that lies to sync (don't know the numbers, but 
>>>> I have to assume some/many are doing this with Lucene - in which 
>>>> case you pay perf for a false sense of security<g>).
>>>
>>> Well, if the consumer drive is in fact lying, then sync should be 
>>> wicked fast ;)  So you get a false sense of security without paying 
>>> anything!
>> That occurred to me as well, but they must do some work :) , or the 
>> lie would be to just return...maybe it is. I mean if it doesn't 
>> guarantee, whats the point...oh yeah, to fool benchmarks. Still 
>> boggles my mind.
>>
>> I wasn't being very serious though. Like I said, I'm not suggesting 
>> we allow it to be turned off, I was just looking for it because I 
>> wanted to debug...your option was just as good for my case.
>>>
>>>> Not a real suggestion at this point though. Just thinking about 
>>>> some of the reports I have seen of much slower indexing with 2.4 
>>>> (the latest being to the solr list today). Can't imagine why 
>>>> someone would see such a drastic change (I imagine you could 
>>>> imagine a lot better), other than maybe the sync is hobbling their 
>>>> specific situation (in which case i'd guess its not lying if it 
>>>> where going to be so slow though <g> Or its AIX or something <g>). 
>>>> Would be cool to be able to flip it off and test. Sounds like thats 
>>>> simple enough already  though. I could whip up an off for solr 
>>>> testing easy enough.
>>>
>>> Yeah let's dig down and get to the root cause here -- if turning off 
>>> sync in fact fixes the slowdown we should understand why sync is 
>>> being called so often.
>> Yeah, the root cause is likely elsewhere I'm sure...it would have to 
>> be a pretty broken sync to add so much time (im fairly sure hes using 
>> defaults so he not going to have a billion index files to sync or 
>> something). Just captured my attention as a possibility and I wanted 
>> to be able to test without. Most 2.3 - 2.4 changes don't seem like 
>> they would slow things down. Could be a problem with Solr as well in 
>> this particular case though.
>>>
>>> Mike
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org