lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <reng...@ix.netcom.com>
Subject Re: [jira] Commented: (LUCENE-1044) Behavior on hard power shutdown
Date Mon, 12 Nov 2007 20:23:59 GMT
I would be wary of the additional complexity of doing this.

It would be my vote to making 'sync' an option, and if set, all files  
are sync'd before close.

With proper hardware setup, this should be a minimal performance  
penalty.

What about writing a marker at the end of each file? I am not sure it  
is guarenteed but the segments is syncd, and the segment files have  
the correct marker, then the segment file is ok. Otherwise the "bad"  
segments/versions can be removed (on start up).

On Nov 12, 2007, at 2:06 PM, Michael McCandless wrote:

>
> I'll look into this approach.
>
> We must also sync/close the file before we can open it for reading, eg
> for creating compound file or if a merge kicks off.
>
> Though if we are willing to not commit a new segments_N after saving a
> segment and before creating its compound found then we don't need to
> sync the segment files in that case.
>
> I think I would put all this logic (to manage background sync'ing)
> under FSDirectory.
>
> Mike
>
> "Yonik Seeley" <yonik@apache.org> wrote:
>> On Nov 12, 2007 1:41 PM, robert engels <rengels@ix.netcom.com> wrote:
>>> Would it not be simpler to pure Java...
>>>
>>> Add the descriptor that needs to be sync'd (and closed) to a Queue.
>>> Start a Thread to sync/close descriptors.
>>>
>>> In commit(), wait for all sync threads to terminate using join().
>>
>> This would also need to be hooked in with file deletion (since a file
>> could be created and deleted before commit()).
>>
>> -Yonik
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message