continuum-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse McConnell" <jesse.mcconn...@gmail.com>
Subject Re: build scheduling issues
Date Wed, 08 Nov 2006 14:57:27 GMT
the issue isn't cycles within a normal m2 build, but when you shove 4
or 5 of them together into one continuum instance and then ask them
all to play together.

For example if you shoved continuum, maven-shared and archiva all into
the same continuum instance and triggered a build with all of them in
the same directed graph then we have had instances of cycles in the
past (not sure about right now, mpir cycle might be refactored out
thanks to joakim).

emm had another example last night when we were talking about it as well.

jesse

On 11/8/06, Brett Porter <brett@apache.org> wrote:
> Sorry, am I missing something? How do we end up with a cycle in
> Continuum? Is there a specific example?
>
> I know it's possible - we had to allow it in the repository (eg,
> dom4j <-> jaxen), but it is certainly undesirable and honestly should
> be rare, especially in m2 built artifacts. Should it produce a build
> warning?
>
> Basically, the treatment there is to just stop following the tree
> when you hit the cycle, rather than changing the way things are
> ordered. So it's really arbitrary which might come first, but isn't
> really a concern there.
>
> Anyway, just wondering.
>
> Cheers,
> Brett
>
> On 08/11/2006, at 11:46 PM, Emmanuel Venisse wrote:
>
> > Yes
> >
> > Jesse McConnell a écrit :
> >> we should add a page that analyzes each schedule for cycles...that
> >> would be a cool little feature
> >> On 11/8/06, Emmanuel Venisse <emmanuel@venisse.net> wrote:
> >>> Yes, we need a global ordering, so projects will be build
> >>> independently of groups, because in some
> >>> case a cycle can be created between groups (not necessary between
> >>> projects).
> >>>
> >>> In case a cycle is detected between projects, continuum can't
> >>> find the build order. In this case, I
> >>> think we need to sort a little project so will reduce build
> >>> errors. So if we have a cycle, we can
> >>> sort projects in a group and build them. In most of case (maven
> >>> projects), we don't have a cycle in
> >>> a group.
> >>>
> >>> Emmanuel
> >>>
> >>> Brett Porter a écrit :
> >>> > I think you want global ordering. Grouping should just be a
> >>> > display/management technique, not anything that changes how
> >>> projects are
> >>> > handled.
> >>> >
> >>> > However, this needs to be reviewed as a whole (which I think
> >>> Emmanuel is
> >>> > doing), such that builds can be triggered when their
> >>> dependencies change
> >>> > which will help with the ordering as it won't be dependant on
> >>> them all
> >>> > being triggered at the same time?
> >>> >
> >>> > - Brett
> >>> >
> >>> > On 08/11/2006, at 9:51 AM, Jesse McConnell wrote:
> >>> >
> >>> >> I was reading through the DefaultContinuum.buildProjects
> >>> ( Schedule id
> >>> >> ) method and after discussing some things with Emmanuel...I
> >>> think we
> >>> >> have a problem here.  When I went through and refactored
> >>> things to
> >>> >> support a more Project Group centric setup with continuum I
> >>> changed
> >>> >> this method a bit.
> >>> >>
> >>> >> Originally, this method would gather up all projects that
> >>> would be
> >>> >> triggered by that schedule, run them all through the project
> >>> sorter
> >>> >> and then build each in sequence.
> >>> >>
> >>> >> When I added the project groups to this mix, I changed things
> >>> to be on
> >>> >> a project group basis, so that on a project group by project
> >>> group
> >>> >> basis it would order the projects and build them.  At the time
I
> >>> >> thought this was the way to go...but maybe not.
> >>> >>
> >>> >> 17:14 <evenisse> we need to take all projects from all groups,
> >>> sort them
> >>> >> 17:15 <evenisse> if we don't have a cycle, it's ok and we
> >>> build all
> >>> >> 17:15 <evenisse> if it isn't ok, we sort project by group
> >>> >>
> >>> >> For example, if we loaded up a Plexus group and a Maven
> >>> group...the
> >>> >> way it currently is (with my change) it would process all
> >>> triggered
> >>> >> builds within one group and then process all triggered builds
> >>> in the
> >>> >> other group.   This would not take into account potential
> >>> dependencies
> >>> >> between the two.
> >>> >>
> >>> >> Does anyone have any thoughts on this?  I am inclined to fix
> >>> it up so
> >>> >> its like it used to be where all projects across all project
> >>> groups
> >>> >> are thrown into the graph....I keep feeling like I am missing
> >>> >> something wrong with this, but I can't pin it down.
> >>> >>
> >>> >> One thing that perhaps Emmanuel could explain a bit more is
> >>> the third
> >>> >> comment there.  In our conversation on this he said that he
> >>> thinks
> >>> >> that the cycles are cropping up all the time, and if thats the
> >>> case
> >>> >> then we are building a lot of unordered builds which would
> >>> account for
> >>> >> some of the strange reports we have been getting.  Are you
> >>> saying that
> >>> >> if we detect the cycle we default back to the way I am doing
> >>> it now?
> >>> >> order within the groups...
> >>> >>
> >>> >> jesse
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --jesse mcconnell
> >>> >> jesse.mcconnell@gmail.com
> >>> >
> >>> >
> >>> >
> >>>
> >>>
>


-- 
jesse mcconnell
jesse.mcconnell@gmail.com
Mime
View raw message