cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Yadav <bhais...@apache.org>
Subject Re: Merge 4.1-new-db-schema.sql to db/db/schema-40to410.sql
Date Tue, 19 Feb 2013 10:46:12 GMT
On Mon, Feb 18, 2013 at 2:30 PM, Koushik Das <koushik.das@citrix.com> wrote:
> Rohit,
>
> Currently I see that there are 2 sql files related to upgrade from version A to B, for
e.g. schema-40to410.sql and schema-40to410-cleanup.sql.
> 1. What all should go in the normal one and what all in the -cleanup one? For e.g. should
deleting table metadata go in cleanup script, should deleting table data go in cleanup script?

In the schema one you should keep rules like add columns, add tables,
copying data (like copying to uuid column from id) etc. In the cleanup
script you should do things like removing columns or data. Think in
this way, when multiple mgmt servers are running and not all of them
are on same version, we want the first step of database upgrade to be
backward compatible to the old one which means you're not removing any
column, after you've upgraded all mgmt server you are free to cleanup
and remove any column, copy any data from those columns to maybe new
ones.

> 2. What is the order in which the scripts will be executed? Upgrade followed by corresponding
cleanup for each version? OR is it like call all upgrades for all versions followed by all
cleanup scripts?

At present, all create schemas are called first and cleanup is done
only at the end. We'll have the same way for 4.1 but starting 4.2,
once we will have the DatabaseCreator tool, it would much easier and
sysadmins could decide how they want to upgrade. Alex can comment on
it.

> 3. If the ordering is based on former case (in 2 above) then it is better to combine
them into one single sql script. If it is the latter then I would like to understand the reason
behind that?

Read explanation in 1. In case of clustered mgmt servers, if an old
mgmt server is running they will die if you remove any column of some
table it can access. While one may argue why not bring down the whole
cloud and then do upgrades, which is fine (and will be possible with
dbcreator/4.2) but not possible as only mgmt server upgrades the db
now.

Regards.

>
> Thanks,
> Koushik
>
>> -----Original Message-----
>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com] On Behalf
>> Of Rohit Yadav
>> Sent: Saturday, February 16, 2013 5:51 PM
>> To: cloudstack-dev@incubator.apache.org; Edison Su
>> Cc: Sheng Yang; Alex Huang; Kishan Kavala; Min Chen
>> Subject: Re: Merge 4.1-new-db-schema.sql to db/db/schema-40to410.sql
>>
>> I've applied my changes on master, to gain visibility of those changes and so
>> community can grok I started a new thread of db changes for 4.1 to support
>> rolling db upgrades.
>>
>> Regards.
>>
>> On Sat, Feb 16, 2013 at 1:58 PM, Rohit Yadav <bhaisaab@apache.org> wrote:
>> > On Fri, Feb 15, 2013 at 11:26 PM, Edison Su <Edison.su@citrix.com> wrote:
>> >>
>> >>
>> >>> -----Original Message-----
>> >>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com] On
>> >>> Behalf Of Rohit Yadav
>> >>> Sent: Friday, February 15, 2013 2:58 AM
>> >>> To: Edison Su; cloudstack-dev@incubator.apache.org
>> >>> Cc: Sheng Yang; Alex Huang; Kishan Kavala; Min Chen
>> >>> Subject: Merge 4.1-new-db-schema.sql to db/db/schema-40to410.sql
>> >>>
>> >>> Edison, you'd created a 4.1-new-db-schema sql, may I merge
>> >>> 4.1-new-db- schema.sql  to db/db/schema-40to410.sql? We'll also need
>> >>> a db/db/schema- 40to410-cleanup.sql and upgrading class.
>> >>
>> >> Yes, it should be merged into schema-40to410.sql.
>> >
>> > Hi Edison, there was one merge issue while moving the extra create
>> > rules from create-schema to schema-40to410.sql. In table
>> > cloud.template_s3_ref;
>> >
>> > diff --git a/setup/db/db/schema-40to410.sql
>> > b/setup/db/db/schema-40to410.sql index 9a9441d..3fb1e32 100644
>> > --- a/setup/db/db/schema-40to410.sql
>> > +++ b/setup/db/db/schema-40to410.sql
>> > @@ -171,7 +171,7 @@ CREATE TABLE `cloud`.`template_s3_ref` (
>> >    CONSTRAINT `fk_template_s3_ref__template_id` FOREIGN KEY
>> > `fk_template_s3_ref__template_id` (`template_id`) REFERENCES
>> > `vm_template` (`id`),
>> > -  INDEX `i_template_s3_ref__swift_id`(`s3_id`),
>> > +  INDEX `i_template_s3_ref__s3_id`(`s3_id`),
>> >    INDEX `i_template_s3_ref__template_id`(`template_id`)
>> >  ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
>> >
>> > I chose the change in create-schema, not not in schema-40to410.sql.
>> > Please check if it's okay, or fix as needed. Thanks.
>> >
>> >
>> >
>> >>
>> >>>
>> >>> There is some difference between the 4.0 create-schema.sql and that
>> >>> in
>> >>> 4.1: http://people.apache.org/~bhaisaab/diff-create-schema-40-41.sql
>> >>>
>> >>> Alex suggested in last discussion that we should not change the
>> >>> schema file and have upgrade paths and cleanup paths. This would
>> >>> help us do rolling upgrades. But Alex, there are a lot of difference
>> >>> between the
>> >>> 4.0 schema and 4.1 schema sqls (about 4k lines of diff):
>> >>> http://people.apache.org/~bhaisaab/diff4041sqls.sql If we don't want
>> >>> developers to change anything in the schema files, how do you
>> >>> propose we fix it now for 4.1? Move all these changes to 40to41 sql?
>> >>>
>> >>> Sheng (ipv6), Kishan (regions), Vijay (autoscale) I see some create
>> >>> schema changes, should we move them to db/db/schema-40to410.sql?
>> >>>
>> >>> Min, the create-schema-view did not exist for 4.0, should we keep it
>> >>> as it as, our move it as upgrade path?
>> >>>
>> >>> For 4.1, we should do following changes so DatabaseCreator could be
>> >>> refactored and used starting 4.2/master:
>> >>> - Change version to 4.0 in create-schema
>> >>> - Make sure there is difference between 4.0 and 4.1 create-schema
>> >>> sqls
>> >>> - Move new additions to db/db/schema-40to410.sql, define a cleanup
>> >>> path/sql and upgrading class from 40-41
>> >>> - Fix cloudstack-setup-databases, for 4.1 we won't use
>> >>> databasecreator, stick with the present script (won't risk
>> >>> regressions and whatnot)
>> >>>
>> >>> Regards.

Mime
View raw message