Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Derby Development" <derby-dev@db.apache.org>
Received-SPF: neutral (hermes.apache.org: local policy)
Message-ID: <423896FC.6070200@debrunners.com>
Date: Wed, 16 Mar 2005 12:28:44 -0800
From: Daniel John Debrunner <djd@debrunners.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
 rv:1.7.3) Gecko/20040910
MIME-Version: 1.0
To: derby-dev <derby-dev@db.apache.org>
Subject: System catalog upgrade mechanics
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I explained hard/soft upgrade a long time ago here

version scheme
http://mail-archives.eu.apache.org/mod_mbox/db-derby-dev/200408.mbox/%3c412E278B.5070109@debrunners.com%3e

upgrade
http://mail-archives.eu.apache.org/mod_mbox/db-derby-dev/200408.mbox/%3c412E31EB.4090302@debrunners.com%3e

Here is some more on the mechanics of the upgrade for the system
catalogs. I will add this into code comments in the relevant classes. I
working on a submission that adds more comments into this area and sets
the system catalog version number correctly for 10.1. This will then
allow people to code the upgrade mechanics for new features.

System catalog upgrade issues - there are various kinds of changes that
affect the system catalogs with a new release. A release may not contain
all of these types, but will typically include at least one.

1) Change structure of system tables, examples are
  - add new system table
  - add new column to existing system table
  - add index to existing system table

2) Modify contents of system table(s), examples are
  - fix incorect information, e.g. inccorrect nullability of column
  - Change a value for a set of rows to support new functionality (rare,
usually the code uses the old values)

3) New version of engine will write information that will not be
understood by a previous release, e.g.

  - new table type value in systables
  - new alias descriptor in sysaliases


When running in soft upgrade mode the upgrade code (in DD_Version) needs to:

a) check the current soft upgrade level (property
derby.softDataDictionaryVersion) and determine which "safe changes" can
be made.
b) apply "safe changes" from category 2) in a single transaction. An
example would be fixing incorrect information.
c) set derby.softDataDictionaryVersion to represent the current engine
version to indicate the changes in B) have been applied.
Otherwise in soft upgrade no other changes are made to the system catalogs

In full upgrade mode, initial database connection with upgrade=true
these steps are made:

A) check the current hard upgrade level (property DataDictionaryVersion)
and determine which changes are required.

B) Make any changes from categories 1) and 2) in a single transaction.

C) set DataDictionaryVersion to represent the current engine version
thus indicating the upgrade on the system catalogs has been completed.
Thus new features can execute code that would result in changes in
category 3

Code that will make changes in category 3) or requires other changes
made by hard upgrade is required to check the version of the system
catalogs using the DataDictionary.checkVersion method, or the
corresponding utility method in the parser. This check will throw an
exception if the system catalog has not been hard upgraded to the
current level.

As examples, with the current development going on for Derby I know of
these items that require upgrade thought:

- Java signatures in method definitions, writes information into the
system catalog in its current structure, but the signature would not be
understood by 10.0. Would need to perform a checkVersion(10.1) if a
signature is present. (10.1 is not the actual constant, just to give an
idea).

- Log record checksum to detect out of order writes. If in soft upgrade
mode, need to ensure checksum records are not written. Though this is a
store upgrade issue, not a system catalog one.

- Synonym support, like java signatures, most likely would be
implemented by new information in system catalogs that would not be
understood by 10.0, thus a checkVersion(10.1) would be required.


Dan.