cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <daan.hoogl...@gmail.com>
Subject Re: [DISCUSS] Checking in code that will break others' environments
Date Thu, 06 Mar 2014 13:15:22 GMT
Ah, yes and it is an answer to Mike's initial question indeed. It is
not sufficient for what Miguel describes however.

On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira
<MFerreira@schubergphilis.com> wrote:
> Hi all,
>
> I've been looking at the way DB related code (SQL scripts and Java classes) are updated
and what is the impact of those updates on a live cloudstack DB.
> By the way, my intention is to support a per-commit DB upgrade of a running system.

On Thu, Mar 6, 2014 at 2:02 PM, Donal Lafferty
<donal.lafferty@citrix.com> wrote:
> The thread started with a discussion of problems with out of date databases.  Mike pulled
from master, rebuilt, and found out his testbed's database was out of date and no longer works.
>
> We've tried to find a database solution, but out of databases are caused by more than
schema changes.  The data in the tables and other pieces of CloudStack can change after a
commit, especially towards the end of release when everyone's feature is being merged.
>
> With 4.2, I dealt with this issue by writing a script that rebuilt my testbed by replaying
the API calls I made to set it up in the first place.
>
> Is this a solution you've considered for your dev environment?
>
> DL
>
>> -----Original Message-----
>> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
>> Sent: 06 March 2014 12:50
>> To: dev
>> Cc: Daan Hoogland
>> Subject: Re: [DISCUSS] Checking in code that will break others' environments
>>
>> No, but how do you mean with respect to this thread?
>>
>> On Thu, Mar 6, 2014 at 1:48 PM, Donal Lafferty <donal.lafferty@citrix.com>
>> wrote:
>> > Hi Daan,
>> >
>> > Is there anything stopping you from scripting the configuration of
>> > your CloudStack testbed?  E.g. with Marvin or CloudMonkey
>> >
>> > DL
>> >
>> >> -----Original Message-----
>> >> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
>> >> Sent: 06 March 2014 09:53
>> >> To: dev
>> >> Cc: Daan Hoogland
>> >> Subject: Re: [DISCUSS] Checking in code that will break others'
>> >> environments
>> >>
>> >> I totally agree with the incremental approach. I am a fascist at time
>> >> because i would even want people to add downgrade scripts to any db
>> >> change they make. Having them not adjust their sql is a good first step,
>> though.
>> >>
>> >> On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira
>> >> <MFerreira@schubergphilis.com> wrote:
>> >> > Hi all,
>> >> >
>> >> > I've been looking at the way DB related code (SQL scripts and Java
>> >> > classes)
>> >> are updated and what is the impact of those updates on a live cloudstack
>> DB.
>> >> > By the way, my intention is to support a per-commit DB upgrade of a
>> >> running system.
>> >> >
>> >> > Anyway, I completely agree that sending an email warning people
>> >> > about
>> >> changes that can potentially break running environments is a good
>> >> thing, and can save a lot of time and headaches.
>> >> > I also like the idea of Rajani of using tools to help in the DB
>> >> > migration
>> >> process.
>> >> > However, I would like to put forward another idea: what about
>> >> > making sure
>> >> that the changes to the DB (schema and data) are always incremental?
>> >> >
>> >> > I mean, once a SQL statement (say A) is added to a script it could
>> >> > be kept as
>> >> is, and subsequent modifications cloud be made via new statements
>> >> (say B), instead of adapting A.
>> >> > This would allow people to upgrade their databases to the point
>> >> > they ran
>> >> statement A, and afterwards upgrade it again to the point they ran
>> >> statement B.
>> >> > The same principle could also be applied to the Java classes that
>> >> > do data
>> >> migration, but maybe there it might be a bit more involved.
>> >> >
>> >> > Cheers,
>> >> > Miguel
>> >> >
>> >> > -----Original Message-----
>> >> > From: Koushik Das [mailto:koushik.das@citrix.com]
>> >> > Sent: donderdag 6 maart 2014 8:25
>> >> > To: <dev@cloudstack.apache.org>
>> >> > Subject: Re: [DISCUSS] Checking in code that will break others'
>> >> > environments
>> >> >
>> >> > Before doing a git pull, I generally check the sql schema changes
>> >> > and run
>> >> the delta manually on my existing setup. In most of the cases that
>> >> works for me without having to redeploy the db.
>> >> >
>> >> > -Koushik
>> >> >
>> >> > On 06-Mar-2014, at 11:43 AM, Mike Tutkowski
>> >> <mike.tutkowski@solidfire.com> wrote:
>> >> >
>> >> >> Yeah, I definitely just meant a "heads up" during development if
>> >> >> you are going to change something that will break other people's
>> >> >> environments who update. If these people know in advance, they
may
>> >> >> choose to postpone an update until they are at a better point.
>> >> >>
>> >> >>
>> >> >> On Wed, Mar 5, 2014 at 11:01 PM, Rajani Karuturi
>> >> >> <Rajani.Karuturi@citrix.com
>> >> >>> wrote:
>> >> >>
>> >> >>> Across versions db migration is taken care. I think this is
bound
>> >> >>> to occur while working on a release, if multiple people work
on
>> >> >>> the same branch with different work-in-progress features.
>> >> >>>
>> >> >>> Could we move to flyway or liquibase which can take care of
db
>> >> >>> versioning and migration?
>> >> >>>
>> >> >>>
>> >> >>> ~Rajani
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On 06-Mar-2014, at 2:08 am, Mike Tutkowski
>> >> >>> <mike.tutkowski@solidfire.com>
>> >> >>> wrote:
>> >> >>>
>> >> >>>> Yeah, in this case, I'm not referring to erroneous code
that
>> >> >>>> breaks a person's environment (since hopefully the person
>> >> >>>> wouldn't have knowingly checked in such code), but rather,
say,
>> >> >>>> DB-type changes that improve the system, but break current
>> setups.
>> >> >>>>
>> >> >>>> Just a heads-up e-mail with some easily identifiable tag.
>> >> >>>>
>> >> >>>> Can anyone think of a good tag for this? It's not always
DB
>> >> >>>> related, so
>> >> >>> we
>> >> >>>> might want the tag to be more general.
>> >> >>>>
>> >> >>>>
>> >> >>>> On Wed, Mar 5, 2014 at 1:28 PM, Ian Duffy <ian@ianduffy.ie>
>> wrote:
>> >> >>>>
>> >> >>>>> +1 to this.
>> >> >>>>>
>> >> >>>>> Having the build suddenly break due to a git pull has
been very
>> >> >>> annoying!
>> >> >>>>> I usually end up searching through the commit log and
doing a
>> >> >>>>> resets until I find a commit where it works. Then waiting
>> >> >>>>> awhile until I do a git pull again and hoping the code
was fixed.
>> >> >>>>>
>> >> >>>>> On 5 March 2014 20:19, Mike Tutkowski
>> >> >>>>> <mike.tutkowski@solidfire.com>
>> >> >>>>> wrote:
>> >> >>>>>> Hi,
>> >> >>>>>>
>> >> >>>>>> I encountered a bit of a problem this morning and
thought I
>> >> >>>>>> would bring
>> >> >>>>> it
>> >> >>>>>> up for discussion.
>> >> >>>>>>
>> >> >>>>>> If we already have a policy around this, please
let me know.
>> >> >>>>>>
>> >> >>>>>> So, I fetched the latest and rebased my local 4.4
development
>> >> >>>>>> branch on
>> >> >>>>> top
>> >> >>>>>> of master. This all went just fine.
>> >> >>>>>>
>> >> >>>>>> When I rebuilt and re-started the CS Management
Server, I soon
>> >> >>> realized I
>> >> >>>>>> could no longer log in from the GUI.
>> >> >>>>>>
>> >> >>>>>> As it turns out, the DB schema had been updated
and so my
>> >> >>>>>> database was
>> >> >>>>> out
>> >> >>>>>> of date. The code was querying for fields that
didn't exist in my
>> DB.
>> >> >>>>>>
>> >> >>>>>> As far as I know, the easiest way to get around
this is to
>> >> >>>>>> destroy my current cloud, run the script to re-build
my
>> >> >>>>>> database, then re-create
>> >> >>> my
>> >> >>>>>> cloud, which is somewhat time consuming.
>> >> >>>>>>
>> >> >>>>>> Do we have a process in place currently in which
we ask those
>> >> >>>>>> who make
>> >> >>>>> such
>> >> >>>>>> changes to send out a notification e-mail to dev@
to give
>> >> >>>>>> people a
>> >> >>>>> heads up
>> >> >>>>>> that updating will lead to such issues? On previous
projects,
>> >> >>>>>> we would
>> >> >>>>> send
>> >> >>>>>> out an e-mail and then people could be aware to
only update if
>> >> >>>>>> they
>> >> >>> were
>> >> >>>>>> prepared for such re-work.
>> >> >>>>>>
>> >> >>>>>> To be clear here, I'm not meaning to pick on anyone
in
>> >> >>> particular...this
>> >> >>>>>> has happened several times over the course of my
CloudStack
>> >> >>>>>> development
>> >> >>>>> and
>> >> >>>>>> I expect that I, too, have checked in such code
(without
>> >> >>>>>> sending out a relevant e-mail) that lead people
to have to
>> >> >>>>>> perform such a complete re-build action un-expectedly.
>> >> >>>>>>
>> >> >>>>>> What do people think about this? Maybe we should
just add an
>> >> >>>>>> e-mail tag
>> >> >>>>> or
>> >> >>>>>> something and point people to the relevant commit?
>> >> >>>>>>
>> >> >>>>>> Thanks!
>> >> >>>>>>
>> >> >>>>>> --
>> >> >>>>>> *Mike Tutkowski*
>> >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >> >>>>>> e: mike.tutkowski@solidfire.com
>> >> >>>>>> o: 303.746.7302
>> >> >>>>>> Advancing the way the world uses the
>> >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> >>>>>> *(tm)*
>> >> >>>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> *Mike Tutkowski*
>> >> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >> >>>> e: mike.tutkowski@solidfire.com
>> >> >>>> o: 303.746.7302
>> >> >>>> Advancing the way the world uses the
>> >> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> >>>> *(tm)*
>> >> >>>
>> >> >>>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> *Mike Tutkowski*
>> >> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> >> e: mike.tutkowski@solidfire.com
>> >> >> o: 303.746.7302
>> >> >> Advancing the way the world uses the
>> >> >> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> >> *(tm)*
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Daan
>>
>>
>>
>> --
>> Daan



-- 
Daan

Mime
View raw message