db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mamta Satoor" <msat...@gmail.com>
Subject Re: some comments on collation wiki page
Date Tue, 03 Apr 2007 07:25:03 GMT
Mike, thanks for going through the wiki page. Especially, the store section
since I am not too familiar with store code.

Following are my responses
1)Mike:Obtain collation id(type) from DataValueDescriptor(DVD) through
DVD.getCollateId method.
The way I had collation in mind, collation type would be stored only in
DataTypeDescriptor(DTD) and DVD would only store Collator object in DVD, ie
the DVD will be unaware of how the Collator object was created using a
specific collation type. The DVD for non-default-collation subclass of
SQLChar is CollatorSQLChar and currently, it's constructor looks as follows
     public CollatorSQLChar(String val, RuleBasedCollator
collatorForCharacterDatatypes)
ie, no collation type info is passed to the CollatorSQLChar. May be to
support the store requirement, we should add another parameter to the
constructor above, called collationId. This collationId can later be
retrieved by store using DVF.getCollateId. Any comments from anyone?

2)Mike:Need to somehow test a store btree recovery to make sure this works
in the redo recovery case.
I will add this test case requirement under Testing section on the wiki
page.

3)Mike:Version number of store level column metadata
I will replace 1st item under Store section with what you have proposed.
Just to be clear, here is what I think Store will be doing. In order to
support soft-upgrade mode, Derby 10.3 will have to recognize and support the
pre-10.3 store level column metadata which will not have collation info in
it. In addition, Derby 10.3 will have the new store level column metadata
which includes collation info for all 10.3 databases (ie upgraded
pre-10.3database, newly created
10.3 database with default collation and newly created 10.3 database with
territory based collation).
I will also add test soft-upgrade and hard-upgrade mode under Testing
section of the wiki page.

4)Mike:I believe there are more than one of these Monitor interfaces that
will have to be moved in DVF.
That is very possible. I didn't look through the store code in detail to see
what other apis should be moved to DVF from Monitor interface.

5)Mike:impact on sorter
I will add a line item for sorter and will also add test requirement for
sorter that will fit in memory and the sorter that will spill over to the
disk.

Please let me know if I missed anything.
Mamta

On 4/2/07, Mike Matrigali <mikem_app@sbcglobal.net> wrote:
>
> I have included some comments based on th following wiki page:
>
> > 6)Store needs a way to determine the collation type for a given DVD.
> This collation type will then be saved in the column metadata. Provide
> the api on DVD to return the correct collation type.
>     Just an addition here.  The way I expect this info to get down to
> store
>     is that the template passed in when creating a conglomerate will have
>     DVD's that have the correct collation id associated with them.
>     This should be true for both btree and heap conglomerate creates, you
>     should update iapi interface doc, but I don't think the actual args
>     to the interface change.
>
>     I assume the api is something like int getCollateId() on a DVD.
>
> >
> > 7)BasicDatabase needs to set the Locale value in the DVF after DVF
> has booted. Probably with an api like DVF.setLocale(Locale). DVF will
> use this Locale object to construct the correct Collator in case user
> has requested territory based collation through jdbc url at database
> create time.
> >
> Need to somehow test a store btree recovery to make sure this works in the
> redo recovery case.
>
> > Store changes
> >
> > 1)Store column level metadata for collate in Store. Store keeps a
> version number that describes the structure of column level metadata.
> For existing pre-10.3 databases which get upgraded to 10.3 and for new
> 10.3 databases with default collatoin(UCS_BASIC), the structure of
> column level metadata will remain same as 10.2 structure of column level
> metadata, ie they will not include collate information in their store
> metadata. A new version would be used in Store for structure of column
> level metadata if the newly created 10.3 database has asked for
> territory based collation. In other words, information about collate
> will be kept in Store column level metadata only if we are working with
> a 10.3 newly created database with territory based collation. This
> approach will make sure that we do not have to do an on-disk store
> metadata upgrade when upgrading a pre-10.3 database to 10.3 version.
> >
> This is not exactly what I expected.  I think there should be a single
> 10.3 metadata format that includes collation information (whether it is
> default or not).  The actual format of the data can optmize as
> appropriate for non-default values if we think that is worth it.  I
> think the right thing to do is do
> only manage either 10.3 version store metadata or pre-10.3 - and not base
> it
> on whether there is a non-default collate or not.  The actual written
> format
> of the 10.3 metadata may optimize the case where there is no non-default
> collate - but it is still a 10.3 version.
>
> So the world breaks down into:
> pre-10.3 db's, which includes dbs that are running against 10.3 but only
>     soft upgrade.
>     o These db's never get the 10.3 metadata format
>
> hard upgrade 10.3 db's
>     o Code will read both pre and post 10.3 metadata.  All new
> conglomerates
>       always write the 10.3 version metadata.  The code has to read both
>       old and new formats.
>
> new 10.3 db's
>     o code will only write 10.3 metadata.
>
> So what I was expecting was:
>
> 1)Store column level metadata for collate in Store. Store keeps a
> version number that describes the structure of column level metadata.
> For existing pre-10.3 databases which get soft upgraded to 10.3, the
> structure of column level metadata will remain same as 10.2 structure of
> column level metadata, ie they will not include collate information in
> their store metadata.
> For any conglomerate created in a 10.3 new database or a 10.3 hard
> upgraded database a new version would be used in Store to include
> information about the collation for each column's metadata stored.
>
>
>
> > 2)Currently, store uses Monitor to create DVD template rows. The
> logic of creating DVDs using formatids should be factored out from
> Monitor into DataValueFactory. Talking in terms of code,
> RowUtil.newClassInfoTemplate should call DVF.classFromIdentifier rather
> than Monitor.classFromIdentifier.
>     I believe there are more than one of these Monitor interfaces that
> will
>     have to be moved in DVF.
> >
> > 3)This item is related to item 2. With Derby 10.3, collation type
> will be the additional metadata in store for each column. When store
> will call DVF to create DVD template row, it will pass the formatids and
> the collation types. DVF will need to be able to assoicate the correct
> Collator with the DVD for Char datatypes depending on the collation
> type. And in order to find the correct Collator, DVF needs to know the
> locale of the database. This locale information will be set on DVF using
> a new method on DVF called void setLocale(Locale). This call will be
> made by BasicDatabase after DVF has finished booting and before store
> starts booting.
> >
>
> I think you should add an item to track the work in the sorter.  Maybe
> you are
> tracking this as part of the aggregate items?  I know there are some
> template stuff specific to the sorter.  You should at least make sure to
> test both a sort that is in memory and a sort that is too big to fit
> into memory that includes special collation stuff.
>
>
>

Mime
View raw message