Mike, thanks for going through the wiki page. Especially, the store section since I am not too familiar with store code.
Following are my responses
1)Mike:Obtain collation id(type) from DataValueDescriptor(DVD) through DVD.getCollateId method.
The way I had collation in mind, collation type would be stored only in DataTypeDescriptor(DTD) and DVD would only store Collator object in DVD, ie the DVD will be unaware of how the Collator object was created using a specific collation type. The DVD for non-default-collation subclass of SQLChar is CollatorSQLChar and currently, it's constructor looks as follows
public CollatorSQLChar(String val, RuleBasedCollator collatorForCharacterDatatypes)
ie, no collation type info is passed to the CollatorSQLChar. May be to support the store requirement, we should add another parameter to the constructor above, called collationId. This collationId can later be retrieved by store using
DVF.getCollateId. Any comments from anyone?
2)Mike:Need to somehow test a store btree recovery to make sure this works in the redo recovery case.
I will add this test case requirement under Testing section on the wiki page.
3)Mike:Version number of store level column metadata
I will replace 1st item under Store section with what you have proposed. Just to be clear, here is what I think Store will be doing. In order to support soft-upgrade mode, Derby 10.3 will have to recognize and support the
pre-10.3 store level column metadata which will not have collation info in it. In addition, Derby 10.3 will have the new store level column metadata which includes collation info for all 10.3 databases (ie upgraded pre-10.3
database, newly created 10.3 database with default collation and newly created 10.3 database with territory based collation).
I will also add test soft-upgrade and hard-upgrade mode under Testing section of the wiki page.
4)Mike:I believe there are more than one of these Monitor interfaces that will have to be moved in DVF.
That is very possible. I didn't look through the store code in detail to see what other apis should be moved to DVF from Monitor interface.
5)Mike:impact on sorter
I will add a line item for sorter and will also add test requirement for sorter that will fit in memory and the sorter that will spill over to the disk.
Please let me know if I missed anything.
On 4/2/07, Mike Matrigali <email@example.com> wrote:
I have included some comments based on th following wiki page:
> 6)Store needs a way to determine the collation type for a given DVD.
This collation type will then be saved in the column metadata. Provide
the api on DVD to return the correct collation type.
Just an addition here. The way I expect this info to get down to store
is that the template passed in when creating a conglomerate will have
DVD's that have the correct collation id associated with them.
This should be true for both btree and heap conglomerate creates, you
should update iapi interface doc, but I don't think the actual args
to the interface change.
I assume the api is something like int getCollateId() on a DVD.
> 7)BasicDatabase needs to set the Locale value in the DVF after DVF
has booted. Probably with an api like
DVF.setLocale(Locale). DVF will
use this Locale object to construct the correct Collator in case user
has requested territory based collation through jdbc url at database
Need to somehow test a store btree recovery to make sure this works in the
redo recovery case.
> Store changes
> 1)Store column level metadata for collate in Store. Store keeps a
version number that describes the structure of column level metadata.
pre-10.3 databases which get upgraded to 10.3 and for new
10.3 databases with default collatoin(UCS_BASIC), the structure of
column level metadata will remain same as 10.2 structure of column level
metadata, ie they will not include collate information in their store
metadata. A new version would be used in Store for structure of column
level metadata if the newly created 10.3 database has asked for
territory based collation. In other words, information about collate
will be kept in Store column level metadata only if we are working with
a 10.3 newly created database with territory based collation. This
approach will make sure that we do not have to do an on-disk store
metadata upgrade when upgrading a pre-10.3 database to 10.3 version.
This is not exactly what I expected. I think there should be a single
10.3 metadata format that includes collation information (whether it is
default or not). The actual format of the data can optmize as
appropriate for non-default values if we think that is worth it. I
think the right thing to do is do
only manage either 10.3 version store metadata or pre-10.3 - and not base it
on whether there is a non-default collate or not. The actual written format
of the 10.3 metadata may optimize the case where there is no non-default
collate - but it is still a 10.3 version.
So the world breaks down into:
pre-10.3 db's, which includes dbs that are running against 10.3 but only
o These db's never get the
10.3 metadata format
hard upgrade 10.3 db's
o Code will read both pre and post 10.3 metadata. All new
always write the 10.3 version metadata. The code has to read both
old and new formats.
new 10.3 db's
o code will only write 10.3 metadata.
So what I was expecting was:
1)Store column level metadata for collate in Store. Store keeps a
version number that describes the structure of column level metadata.
For existing pre-10.3 databases which get soft upgraded to 10.3, the
structure of column level metadata will remain same as 10.2 structure of
column level metadata, ie they will not include collate information in
their store metadata.
For any conglomerate created in a 10.3 new database or a 10.3 hard
upgraded database a new version would be used in Store to include
information about the collation for each column's metadata stored.
> 2)Currently, store uses Monitor to create DVD template rows. The
logic of creating DVDs using formatids should be factored out from
Monitor into DataValueFactory. Talking in terms of code,
should call DVF.classFromIdentifier rather
I believe there are more than one of these Monitor interfaces that will
have to be moved in DVF.
> 3)This item is related to item 2. With Derby
10.3, collation type
will be the additional metadata in store for each column. When store
will call DVF to create DVD template row, it will pass the formatids and
the collation types. DVF will need to be able to assoicate the correct
Collator with the DVD for Char datatypes depending on the collation
type. And in order to find the correct Collator, DVF needs to know the
locale of the database. This locale information will be set on DVF using
a new method on DVF called void setLocale(Locale). This call will be
made by BasicDatabase after DVF has finished booting and before store
I think you should add an item to track the work in the sorter. Maybe
tracking this as part of the aggregate items? I know there are some
template stuff specific to the sorter. You should at least make sure to
test both a sort that is in memory and a sort that is too big to fit
into memory that includes special collation stuff.