db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Van Couvering <David.Vancouver...@Sun.COM>
Subject Re: [jira] Created: (DERBY-218) Add Relaxed Durability option
Date Wed, 13 Apr 2005 00:27:25 GMT
I also struggled with this idea of databases that risk corruption, but 
it seems to be a common feature people like to use.  It's available in 
MySQL and oft used in fast and loose web applications.  I think it's 
also an option in Pointbase, Postgres, and Firebird (with Firebird if I 
remember right it's even the default configuration).

Usually what you do is delay the log writes and run them in the 
background.  As I understand it, you don't get data corruption, but you 
may lose a few transactions.  Many people are quite happy with this 
tradeoff.

David

Mike Matrigali wrote:

> I would also like to see the in memory implementation contributed, but
> I think that is a different discussion.
> 
>  From responses to Dan's original post on building a system with the
> sync options disabled it seemed like there was enough response that
> those options should be made available.  I admit I am worried because
> this system can no longer guarantee recoverability.  It would be
> interesting to know how people would use such a configuration. Obviously 
> for a database that need not last longer than a connection
> this option would work, and in memory would probably work better.
> For databases that last longer than a connection, what risk of
> database corruption is an acceptable trade off to better performance?
> This is a new idea for me, as all databases I have worked up for now
> have not provided a less than durable option.
> 
> I believe the current durability options being discussed are:
> 1) sync of the log file at each commit
> 2) sync of the log file before data page is forced to disk
> 3) sync of page allocation when file is grown
> 4) sync of data writes during checkpoint
> 
> The simple change is just to allow each sync to be disabled, maybe
> allowing control over all 4.
> 
> An interesting discussion is if there are changes that could be made
> to make it less likely to corrupt the whole database on a JVM or machine
> crash when the above syncs have been disabled.
> 
> I would have to think carefully and maybe some tests need to be written, 
> but I believe the only reasone for #3 is to insure that during redo
> crash recovery we don't run out of space.  We actually already can run 
> out of space for growing the log during undo crash recovery, but in that
> case we just halt the boot and print that we need more space - maybe
> something similar could be done for page allocation.  Note that the
> change is still allocating the pages it just is not syncing before 
> inserting onto the pages.
> 
> #1 is probably the biggest performance win, in the case of very short
> update transactions.  Unfortunately the jdbc standard which derby 
> implements defaults to autocommit=true - so often derby intial 
> performance results for new users looks bad when compared to other 
> databases which default to not syncing at commit time.
> 
> Part of this change should be to document the possible recovery failures 
> which can result by not syncing.
> 
> /mikem
> 
> 
> Suresh Thalamati wrote:
> 
>> Sunitha Kambhampati (JIRA) wrote:
>>
>>> Add Relaxed Durability option ------------------------------
>>>
>>>         Key: DERBY-218
>>>         URL: http://issues.apache.org/jira/browse/DERBY-218
>>>     Project: Derby
>>>        Type: Improvement
>>>  Components: Store     Versions: 10.1.0.0    Environment: all
>>>    Reporter: Sunitha Kambhampati
>>>     Fix For: 10.1.0.0
>>>
>>>
>>> Dan Debrunner posted a fix to allow for relaxed durability changes in 
>>> http://article.gmane.org/gmane.comp.apache.db.derby.user/681/match=relaxed+durability

>>>
>>>
>>> 1) Need to add this option in Derby maybe as some property
>>>
>>> 2) Also from discussions on the list, Mike suggested that - that the 
>>> logging system be changed to somehow
>>> record that the database has operated in this manner, so that if
>>> the database goes corrupt we don't waste effort trying to figure out
>>> what when wrong.  Probably need some way to mark the log records, the
>>> log control file and write a message to the user error log file.
>>>
>>>
>>>
>>>  
>>>
>> How about  adding  in-memory support for Derby instead of  trying to 
>> confuse the users with relaxed durabilty option.
>> In this mode ,  will the database be  recoverable to some consistent 
>> state always with  possible loss of  some transaction data in
>> case of a crash ?
>>   -suresh
>>
>>
>>
>>
> 

Mime
View raw message