lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexWriter commit
Date Mon, 15 Sep 2008 20:03:06 GMT

It's only if power is lost *after* the call to IndexWriter.commit()  
has successfully returned, that the guarantee holds.

commit() does not return until all newly written and referenced files  
in the index have been successfully fsync'd (and the OS does not  
return from fsync until all bytes are on stable storage).

If you lose power while commit() is still running, then the only  
guarantee is that the index will be consistent as of the last time you  
successfully ran commit (or, IndexWriter.close, which also calls  
commit).

In any event, as of 2.4, a power loss (or OS crash) should not corrupt  
your index (unlike pre-2.4), but, you will lose all changes since the  
last successful commit.

Mike

Dipen wrote:

> hi michael,
>  this is rather hard for me to understand, if a system loses power
> (electricity), how can it be ensured that fsync() call will happen  
> at all,
> this commit function relies on fsync() but what if OS doesnt have  
> time or
> power in this case to actually call fsync() and synchronize. I read  
> about
> fsync() long time bacck and had this same question.
>
> Is there anything like ultimate transaction preservation procedure  
> as a
> state of no power is almost unpedictable and unplannable. This is  
> rather
> silly question I know, but just coz you mentioned and I happened to  
> read, I
> wouldn loose this opportunity to ask this :)
>
> thanks
>
> On Tue, Sep 16, 2008 at 12:18 AM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>>
>> There is no difference, unless your computer/OS crashes or loses  
>> power
>> shortly after you had call the method.
>>
>> In that case, there's a big difference: commit() guarantees your  
>> index will
>> be intact (assuming the storage system holding your index was not  
>> damaged)
>> but with flush(), which doesn't sync() the index files it writes,  
>> it's
>> entirely possible that the index will become corrupt because some  
>> files were
>> still in the OS's write cache when it crashed.
>>
>> But the guarantee only holds if the underlying storage system is  
>> "honest"
>> about fsync(), ie, it truly flushes all written bytes for that file  
>> to disk
>> before returning.
>>
>> Mike
>>
>>
>> Cam Bazz wrote:
>>
>> Hello,
>>>
>>> What is the difference between flush in <2.4 and commit?
>>>
>>> Also I have been looking over docs, and they mention commit(long)  
>>> but
>>> there is no commit(long) method but only commit()
>>>
>>> Best.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message