db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Thalamati <suresh.thalam...@gmail.com>
Subject Re: [PATCH] increasinng the possible log file id's from 2^22 -1 to 2^33 -1 (derby-101)
Date Fri, 20 May 2005 19:17:45 GMT
By looking at the code , it is kind of tricky to handle upgrade, because 
users can crash in the middle of recovery
and reboot again N times , which can generate new log for uncommitted 
transaction and also all the constant that manipulates LSN
are static now  and are used by static routines in 
org.apache.derby.impl.store.raw.log.LogCounter.java, like 
:makeLogInstantAsLong().

To handle upgrade:
1)   all static methods  in LogCounter.java has to be replaced by  
instance methods , so that
    in the same JVM, one  can run a  database that is in softupgrade 
mode and one that is new/upgraded..
2)  modify the code to update the version number on disk(log control 
file )  to only when  the recovery is complete;
      currently it is updated in the boot code itself.
3)  perform a checkpoint after recovery on upgrade, so that new log file 
limits can be uses, As it will make
    sure that no old log instant format will be in use.

Instead of  doing all of the above , I can reduce the limit to 2^31 -1 
instead of 2^33-1 ;
this way I would not need to change LogCounter.FILE_NUMBER_SHIFT  from 
32 (old value) to 30,
which causes all upgrade issues.

with 2^31 -1 it will take approximately 68 years (2147483647/(24 * 60 * 
60 * 365))
 to hit the log file number limit  if a new log file  is generated every 
second., instead of 272 years
with 2^33-1.

If no one has objections , I would prefer to increase the log file 
number limit to 2^31 -1 and
avoid major upgrade issues. If any one strongly believes  it should be 
2^33 -1 ,  I will try
to do the necessary upgrade changes.

Any comments/suggestions ?

Thanks
-suresht




Suresh Thalamati wrote:

> I just realized , this patch actaully break's hard upgrade also, if  
> log needs to be replayed.
> Becuase I extract the log file number and log file postion from the 
> instant that is on disk ,
> differently after my change.
>
> If  any one else on the list  is testing  upgrade is hitting this 
> problem, please let me know.
> I can post a patch to backout the portion that cause upgrade issues.
>
>
> Thanks
> -suresht
>
>
> Mike Matrigali wrote:
>
>> I am looking at committing this.  The changes look good to me, but I
>> believe there are upgrade issues to handle.
>>
>> For a hard upgrade either new or old databases are fine.
>> For a soft upgrade I think there is a problem if the db generates enough
>> log files to start using the new bits, and then the software is reverted
>> to before the fix.
>>
>> Seems like using the bits needs to be somehow only enabled for hard
>> upgrade.  It would be best if it was controlled just by hard upgrade,
>> but if that is not possible then just doing it for databases created
>> since this version would also work - but still leave problems with old
>> hard upgraded databases.
>>
>> I have reviewed the code and am running tests.  I plan on committing
>> this part of the fix and let you address upgrade issues with a follow on
>> patch.
>>
>> Suresh Thalamati wrote:
>>
>>  
>>
>
>
>



Mime
View raw message