lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: Option to fsync files
Date Wed, 19 Nov 2008 16:56:55 GMT
Michael McCandless wrote:
> Mark Miller wrote:
>> It is dangerous, but probably not any more dangerous than using a 
>> consumer hard drive that lies to sync (don't know the numbers, but I 
>> have to assume some/many are doing this with Lucene - in which case 
>> you pay perf for a false sense of security<g>).
> Well, if the consumer drive is in fact lying, then sync should be 
> wicked fast ;)  So you get a false sense of security without paying 
> anything!
That occurred to me as well, but they must do some work :) , or the lie 
would be to just return...maybe it is. I mean if it doesn't guarantee, 
whats the point...oh yeah, to fool benchmarks. Still boggles my mind.

I wasn't being very serious though. Like I said, I'm not suggesting we 
allow it to be turned off, I was just looking for it because I wanted to 
debug...your option was just as good for my case.
>> Not a real suggestion at this point though. Just thinking about some 
>> of the reports I have seen of much slower indexing with 2.4 (the 
>> latest being to the solr list today). Can't imagine why someone would 
>> see such a drastic change (I imagine you could imagine a lot better), 
>> other than maybe the sync is hobbling their specific situation (in 
>> which case i'd guess its not lying if it where going to be so slow 
>> though <g> Or its AIX or something <g>). Would be cool to be able to

>> flip it off and test. Sounds like thats simple enough already  
>> though. I could whip up an off for solr testing easy enough.
> Yeah let's dig down and get to the root cause here -- if turning off 
> sync in fact fixes the slowdown we should understand why sync is being 
> called so often.
Yeah, the root cause is likely elsewhere I'm would have to be 
a pretty broken sync to add so much time (im fairly sure hes using 
defaults so he not going to have a billion index files to sync or 
something). Just captured my attention as a possibility and I wanted to 
be able to test without. Most 2.3 - 2.4 changes don't seem like they 
would slow things down. Could be a problem with Solr as well in this 
particular case though.
> Mike
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message