db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Thalamati <suresh.thalam...@gmail.com>
Subject Re: [PATCH] increasinng the possible log file id's from 2^22 -1 to 2^33 -1 (derby-101)
Date Sun, 22 May 2005 18:43:27 GMT

Attached patch increases the maximum possible  log file number to  2^31 
-1  and  keeps the  old limit of 2^22-1 on soft upgrade.
Earlier patch for this problem increased it to 2^33 -1 , but that fix 
created  upgrade problems because of  the  additional bits
that are used from  the file position field,  so the limit is being 
reduced to  2^31-1.

Thanks
-suresht


Mike Matrigali wrote:

> This seems like a reasonable 1st step, that would solve current
> upgrade issues.  I suggest you submit this patch, close the current
> jira, and submit a new future improvement jira noting what would
> have to be done to eek out the last few bits (or documenting it
> somewhere appropriate in the code is also fine - just rather not see
> you have learned lost).  Given that this
> soft upgrade stuff is new I would rather be safe this time.
>
> /mikem
>
> Suresh Thalamati wrote:
>
>> By looking at the code , it is kind of tricky to handle upgrade, 
>> because users can crash in the middle of recovery
>> and reboot again N times , which can generate new log for uncommitted 
>> transaction and also all the constant that manipulates LSN
>> are static now  and are used by static routines in 
>> org.apache.derby.impl.store.raw.log.LogCounter.java, like 
>> :makeLogInstantAsLong().
>>
>> To handle upgrade:
>> 1)   all static methods  in LogCounter.java has to be replaced by  
>> instance methods , so that
>>    in the same JVM, one  can run a  database that is in softupgrade 
>> mode and one that is new/upgraded..
>> 2)  modify the code to update the version number on disk(log control 
>> file )  to only when  the recovery is complete;
>>      currently it is updated in the boot code itself.
>> 3)  perform a checkpoint after recovery on upgrade, so that new log 
>> file limits can be uses, As it will make
>>    sure that no old log instant format will be in use.
>>
>> Instead of  doing all of the above , I can reduce the limit to 2^31 
>> -1 instead of 2^33-1 ;
>> this way I would not need to change LogCounter.FILE_NUMBER_SHIFT  
>> from 32 (old value) to 30,
>> which causes all upgrade issues.
>>
>> with 2^31 -1 it will take approximately 68 years (2147483647/(24 * 60 
>> * 60 * 365))
>> to hit the log file number limit  if a new log file  is generated 
>> every second., instead of 272 years
>> with 2^33-1.
>>
>> If no one has objections , I would prefer to increase the log file 
>> number limit to 2^31 -1 and
>> avoid major upgrade issues. If any one strongly believes  it should 
>> be 2^33 -1 ,  I will try
>> to do the necessary upgrade changes.
>>
>> Any comments/suggestions ?
>>
>> Thanks
>> -suresht
>>
>>
>>
>>
>> Suresh Thalamati wrote:
>>
>>> I just realized , this patch actaully break's hard upgrade also, if  
>>> log needs to be replayed.
>>> Becuase I extract the log file number and log file postion from the 
>>> instant that is on disk ,
>>> differently after my change.
>>>
>>> If  any one else on the list  is testing  upgrade is hitting this 
>>> problem, please let me know.
>>> I can post a patch to backout the portion that cause upgrade issues.
>>>
>>>
>>> Thanks
>>> -suresht
>>>
>>>
>>> Mike Matrigali wrote:
>>>
>>>> I am looking at committing this.  The changes look good to me, but I
>>>> believe there are upgrade issues to handle.
>>>>
>>>> For a hard upgrade either new or old databases are fine.
>>>> For a soft upgrade I think there is a problem if the db generates 
>>>> enough
>>>> log files to start using the new bits, and then the software is 
>>>> reverted
>>>> to before the fix.
>>>>
>>>> Seems like using the bits needs to be somehow only enabled for hard
>>>> upgrade.  It would be best if it was controlled just by hard upgrade,
>>>> but if that is not possible then just doing it for databases created
>>>> since this version would also work - but still leave problems with old
>>>> hard upgraded databases.
>>>>
>>>> I have reviewed the code and am running tests.  I plan on committing
>>>> this part of the fix and let you address upgrade issues with a 
>>>> follow on
>>>> patch.
>>>>
>>>> Suresh Thalamati wrote:
>>>>
>>>>  
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>


Mime
View raw message