Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (nike.apache.org: domain of donal.lafferty@citrix.com
 designates 185.25.65.24 as permitted sender)
From: Donal Lafferty <donal.lafferty@citrix.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
CC: Daan Hoogland <DHoogland@schubergphilis.com>
Subject: RE: [DISCUSS] Checking in code that will break others' environments
Thread-Topic: [DISCUSS] Checking in code that will break others' environments
Thread-Index: 
 AQHPOLBrNj5QIBXqZEKpJ57teEV5lJrS4LqAgAAC5QCAAJ1FAIAAA0wAgAAUH4CAACangIAAAo4AgABBaAA=
Date: Thu, 6 Mar 2014 12:48:33 +0000
Message-ID: <9ADDE3F979256644BED8F0D244BE51F00FA7FC@AMSPEX01CL02.citrite.net>
References: 
 <CACOnxChoHbXa75ZAD_bdPXRgeJrxospyZJQWLAJDkuKM7GmSVw@mail.gmail.com>
 <CAEJ3w4XWywoxVeP9GPAQdxQrWuY2xgtvh36xxULr2dKQCepOcQ@mail.gmail.com>
 <CACOnxCg5d5rgNqdub78btJOjk8KXxoAaDDs1M+brjOBogyQodw@mail.gmail.com>
 <697ECC0B-FEEB-46E8-84E3-FDA190615A5B@citrix.com>
 <CACOnxCiELULECvQR38QOpepHZ-7g9NhfzXt_35gK+NCy=5snVQ@mail.gmail.com>
 <DCF3A557-D18B-4E53-BA91-3AFD88ADE3FC@citrix.com>
 <02E3616756CDC04687D08374DA63D32C4C083B@SBPOMB401.sbp.lan>
 <CAGQtxvan9JasowCBrwP20LAp=qhODioo1xCZ8X1bkwC7ZtqVmg@mail.gmail.com>
In-Reply-To: 
 <CAGQtxvan9JasowCBrwP20LAp=qhODioo1xCZ8X1bkwC7ZtqVmg@mail.gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi Daan,

Is there anything stopping you from scripting the configuration of your Clo=
udStack testbed?  E.g. with Marvin or CloudMonkey

DL

> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: 06 March 2014 09:53
> To: dev
> Cc: Daan Hoogland
> Subject: Re: [DISCUSS] Checking in code that will break others' environme=
nts
>=20
> I totally agree with the incremental approach. I am a fascist at time bec=
ause i
> would even want people to add downgrade scripts to any db change they
> make. Having them not adjust their sql is a good first step, though.
>=20
> On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira
> <MFerreira@schubergphilis.com> wrote:
> > Hi all,
> >
> > I've been looking at the way DB related code (SQL scripts and Java clas=
ses)
> are updated and what is the impact of those updates on a live cloudstack =
DB.
> > By the way, my intention is to support a per-commit DB upgrade of a
> running system.
> >
> > Anyway, I completely agree that sending an email warning people about
> changes that can potentially break running environments is a good thing, =
and
> can save a lot of time and headaches.
> > I also like the idea of Rajani of using tools to help in the DB migrati=
on
> process.
> > However, I would like to put forward another idea: what about making su=
re
> that the changes to the DB (schema and data) are always incremental?
> >
> > I mean, once a SQL statement (say A) is added to a script it could be k=
ept as
> is, and subsequent modifications cloud be made via new statements (say B)=
,
> instead of adapting A.
> > This would allow people to upgrade their databases to the point they ra=
n
> statement A, and afterwards upgrade it again to the point they ran
> statement B.
> > The same principle could also be applied to the Java classes that do da=
ta
> migration, but maybe there it might be a bit more involved.
> >
> > Cheers,
> > Miguel
> >
> > -----Original Message-----
> > From: Koushik Das [mailto:koushik.das@citrix.com]
> > Sent: donderdag 6 maart 2014 8:25
> > To: <dev@cloudstack.apache.org>
> > Subject: Re: [DISCUSS] Checking in code that will break others'
> > environments
> >
> > Before doing a git pull, I generally check the sql schema changes and r=
un
> the delta manually on my existing setup. In most of the cases that works =
for
> me without having to redeploy the db.
> >
> > -Koushik
> >
> > On 06-Mar-2014, at 11:43 AM, Mike Tutkowski
> <mike.tutkowski@solidfire.com> wrote:
> >
> >> Yeah, I definitely just meant a "heads up" during development if you
> >> are going to change something that will break other people's
> >> environments who update. If these people know in advance, they may
> >> choose to postpone an update until they are at a better point.
> >>
> >>
> >> On Wed, Mar 5, 2014 at 11:01 PM, Rajani Karuturi
> >> <Rajani.Karuturi@citrix.com
> >>> wrote:
> >>
> >>> Across versions db migration is taken care. I think this is bound to
> >>> occur while working on a release, if multiple people work on the
> >>> same branch with different work-in-progress features.
> >>>
> >>> Could we move to flyway or liquibase which can take care of db
> >>> versioning and migration?
> >>>
> >>>
> >>> ~Rajani
> >>>
> >>>
> >>>
> >>> On 06-Mar-2014, at 2:08 am, Mike Tutkowski
> >>> <mike.tutkowski@solidfire.com>
> >>> wrote:
> >>>
> >>>> Yeah, in this case, I'm not referring to erroneous code that breaks
> >>>> a person's environment (since hopefully the person wouldn't have
> >>>> knowingly checked in such code), but rather, say, DB-type changes
> >>>> that improve the system, but break current setups.
> >>>>
> >>>> Just a heads-up e-mail with some easily identifiable tag.
> >>>>
> >>>> Can anyone think of a good tag for this? It's not always DB
> >>>> related, so
> >>> we
> >>>> might want the tag to be more general.
> >>>>
> >>>>
> >>>> On Wed, Mar 5, 2014 at 1:28 PM, Ian Duffy <ian@ianduffy.ie> wrote:
> >>>>
> >>>>> +1 to this.
> >>>>>
> >>>>> Having the build suddenly break due to a git pull has been very
> >>> annoying!
> >>>>> I usually end up searching through the commit log and doing a
> >>>>> resets until I find a commit where it works. Then waiting awhile
> >>>>> until I do a git pull again and hoping the code was fixed.
> >>>>>
> >>>>> On 5 March 2014 20:19, Mike Tutkowski
> >>>>> <mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> I encountered a bit of a problem this morning and thought I would
> >>>>>> bring
> >>>>> it
> >>>>>> up for discussion.
> >>>>>>
> >>>>>> If we already have a policy around this, please let me know.
> >>>>>>
> >>>>>> So, I fetched the latest and rebased my local 4.4 development
> >>>>>> branch on
> >>>>> top
> >>>>>> of master. This all went just fine.
> >>>>>>
> >>>>>> When I rebuilt and re-started the CS Management Server, I soon
> >>> realized I
> >>>>>> could no longer log in from the GUI.
> >>>>>>
> >>>>>> As it turns out, the DB schema had been updated and so my
> >>>>>> database was
> >>>>> out
> >>>>>> of date. The code was querying for fields that didn't exist in my =
DB.
> >>>>>>
> >>>>>> As far as I know, the easiest way to get around this is to
> >>>>>> destroy my current cloud, run the script to re-build my database,
> >>>>>> then re-create
> >>> my
> >>>>>> cloud, which is somewhat time consuming.
> >>>>>>
> >>>>>> Do we have a process in place currently in which we ask those who
> >>>>>> make
> >>>>> such
> >>>>>> changes to send out a notification e-mail to dev@ to give people
> >>>>>> a
> >>>>> heads up
> >>>>>> that updating will lead to such issues? On previous projects, we
> >>>>>> would
> >>>>> send
> >>>>>> out an e-mail and then people could be aware to only update if
> >>>>>> they
> >>> were
> >>>>>> prepared for such re-work.
> >>>>>>
> >>>>>> To be clear here, I'm not meaning to pick on anyone in
> >>> particular...this
> >>>>>> has happened several times over the course of my CloudStack
> >>>>>> development
> >>>>> and
> >>>>>> I expect that I, too, have checked in such code (without sending
> >>>>>> out a relevant e-mail) that lead people to have to perform such a
> >>>>>> complete re-build action un-expectedly.
> >>>>>>
> >>>>>> What do people think about this? Maybe we should just add an
> >>>>>> e-mail tag
> >>>>> or
> >>>>>> something and point people to the relevant commit?
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=3Dplay>
> >>>>>> *(tm)*
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the
> >>>> cloud<http://solidfire.com/solution/overview/?video=3Dplay>
> >>>> *(tm)*
> >>>
> >>>
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkowski@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud<http://solidfire.com/solution/overview/?video=3Dplay>
> >> *(tm)*
> >
>=20
>=20
>=20
> --
> Daan