db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: [PATCH] increasinng the possible log file id's from 2^22 -1 to 2^33 -1 (derby-101)
Date Fri, 20 May 2005 23:31:28 GMT
This seems like a reasonable 1st step, that would solve current
upgrade issues.  I suggest you submit this patch, close the current
jira, and submit a new future improvement jira noting what would
have to be done to eek out the last few bits (or documenting it
somewhere appropriate in the code is also fine - just rather not see
you have learned lost).  Given that this
soft upgrade stuff is new I would rather be safe this time.


Suresh Thalamati wrote:
> By looking at the code , it is kind of tricky to handle upgrade, because 
> users can crash in the middle of recovery
> and reboot again N times , which can generate new log for uncommitted 
> transaction and also all the constant that manipulates LSN
> are static now  and are used by static routines in 
> org.apache.derby.impl.store.raw.log.LogCounter.java, like 
> :makeLogInstantAsLong().
> To handle upgrade:
> 1)   all static methods  in LogCounter.java has to be replaced by  
> instance methods , so that
>    in the same JVM, one  can run a  database that is in softupgrade mode 
> and one that is new/upgraded..
> 2)  modify the code to update the version number on disk(log control 
> file )  to only when  the recovery is complete;
>      currently it is updated in the boot code itself.
> 3)  perform a checkpoint after recovery on upgrade, so that new log file 
> limits can be uses, As it will make
>    sure that no old log instant format will be in use.
> Instead of  doing all of the above , I can reduce the limit to 2^31 -1 
> instead of 2^33-1 ;
> this way I would not need to change LogCounter.FILE_NUMBER_SHIFT  from 
> 32 (old value) to 30,
> which causes all upgrade issues.
> with 2^31 -1 it will take approximately 68 years (2147483647/(24 * 60 * 
> 60 * 365))
> to hit the log file number limit  if a new log file  is generated every 
> second., instead of 272 years
> with 2^33-1.
> If no one has objections , I would prefer to increase the log file 
> number limit to 2^31 -1 and
> avoid major upgrade issues. If any one strongly believes  it should be 
> 2^33 -1 ,  I will try
> to do the necessary upgrade changes.
> Any comments/suggestions ?
> Thanks
> -suresht
> Suresh Thalamati wrote:
>> I just realized , this patch actaully break's hard upgrade also, if  
>> log needs to be replayed.
>> Becuase I extract the log file number and log file postion from the 
>> instant that is on disk ,
>> differently after my change.
>> If  any one else on the list  is testing  upgrade is hitting this 
>> problem, please let me know.
>> I can post a patch to backout the portion that cause upgrade issues.
>> Thanks
>> -suresht
>> Mike Matrigali wrote:
>>> I am looking at committing this.  The changes look good to me, but I
>>> believe there are upgrade issues to handle.
>>> For a hard upgrade either new or old databases are fine.
>>> For a soft upgrade I think there is a problem if the db generates enough
>>> log files to start using the new bits, and then the software is reverted
>>> to before the fix.
>>> Seems like using the bits needs to be somehow only enabled for hard
>>> upgrade.  It would be best if it was controlled just by hard upgrade,
>>> but if that is not possible then just doing it for databases created
>>> since this version would also work - but still leave problems with old
>>> hard upgraded databases.
>>> I have reviewed the code and am running tests.  I plan on committing
>>> this part of the fix and let you address upgrade issues with a follow on
>>> patch.
>>> Suresh Thalamati wrote:

View raw message