db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Thalamati <suresh.thalam...@gmail.com>
Subject Re: [jira] Created: (DERBY-218) Add Relaxed Durability option
Date Tue, 12 Apr 2005 23:38:55 GMT

I believe providing  mode which can cause non-bootable database to gain 
performace  is a bad  idea.
Most of the users like me does not read manuals in detail  and will find 
hard way that the database is hosed :-)
Say for example  I want to use Derby to store system monitoring 
information with relaxed durabilty mode.
It   would be ok to lose some data , but it will not be acceptable if 
the database can not be booted.

I think durabilty option#2 should not be available,  the  transaction 
log should be synced when a data page is forced to disk to
avoid non-recoverable datbases for sure. These kind of  log syncs can  
be reduced by just making the page cache bigger,
and also there should be checkpoint atleast once in a while may be for a 
100MB(just a randon number)
 or on a  boot otherwise the  disk space used by the transaction log 
will never be released.

Thanks
-suresh

Mike Matrigali wrote:

> I would also like to see the in memory implementation contributed, but
> I think that is a different discussion.
>
> From responses to Dan's original post on building a system with the
> sync options disabled it seemed like there was enough response that
> those options should be made available.  I admit I am worried because
> this system can no longer guarantee recoverability.  It would be
> interesting to know how people would use such a configuration. 
> Obviously for a database that need not last longer than a connection
> this option would work, and in memory would probably work better.
> For databases that last longer than a connection, what risk of
> database corruption is an acceptable trade off to better performance?
> This is a new idea for me, as all databases I have worked up for now
> have not provided a less than durable option.
>
> I believe the current durability options being discussed are:
> 1) sync of the log file at each commit
> 2) sync of the log file before data page is forced to disk
> 3) sync of page allocation when file is grown
> 4) sync of data writes during checkpoint
>
> The simple change is just to allow each sync to be disabled, maybe
> allowing control over all 4.
>
> An interesting discussion is if there are changes that could be made
> to make it less likely to corrupt the whole database on a JVM or machine
> crash when the above syncs have been disabled.
>
> I would have to think carefully and maybe some tests need to be 
> written, but I believe the only reasone for #3 is to insure that 
> during redo
> crash recovery we don't run out of space.  We actually already can run 
> out of space for growing the log during undo crash recovery, but in that
> case we just halt the boot and print that we need more space - maybe
> something similar could be done for page allocation.  Note that the
> change is still allocating the pages it just is not syncing before 
> inserting onto the pages.
>
> #1 is probably the biggest performance win, in the case of very short
> update transactions.  Unfortunately the jdbc standard which derby 
> implements defaults to autocommit=true - so often derby intial 
> performance results for new users looks bad when compared to other 
> databases which default to not syncing at commit time.
>
> Part of this change should be to document the possible recovery 
> failures which can result by not syncing.
>
> /mikem
>
>
> Suresh Thalamati wrote:
>
>> Sunitha Kambhampati (JIRA) wrote:
>>
>>> Add Relaxed Durability option ------------------------------
>>>
>>>         Key: DERBY-218
>>>         URL: http://issues.apache.org/jira/browse/DERBY-218
>>>     Project: Derby
>>>        Type: Improvement
>>>  Components: Store     Versions: 10.1.0.0    Environment: all
>>>    Reporter: Sunitha Kambhampati
>>>     Fix For: 10.1.0.0
>>>
>>>
>>> Dan Debrunner posted a fix to allow for relaxed durability changes 
>>> in 
>>> http://article.gmane.org/gmane.comp.apache.db.derby.user/681/match=relaxed+durability

>>>
>>>
>>> 1) Need to add this option in Derby maybe as some property
>>>
>>> 2) Also from discussions on the list, Mike suggested that - that the 
>>> logging system be changed to somehow
>>> record that the database has operated in this manner, so that if
>>> the database goes corrupt we don't waste effort trying to figure out
>>> what when wrong.  Probably need some way to mark the log records, the
>>> log control file and write a message to the user error log file.
>>>
>>>
>>>
>>>  
>>>
>> How about  adding  in-memory support for Derby instead of  trying to 
>> confuse the users with relaxed durabilty option.
>> In this mode ,  will the database be  recoverable to some consistent 
>> state always with  possible loss of  some transaction data in
>> case of a crash ?
>>   -suresh
>>
>>
>>
>>
>
>



Mime
View raw message