continuum-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marica Tan" <c...@exist.com>
Subject Re: First Part of Distributed Builds
Date Tue, 06 Jan 2009 02:53:11 GMT
On Tue, Jan 6, 2009 at 12:48 AM, Wendy Smoak <wsmoak@gmail.com> wrote:

> On Fri, Dec 12, 2008 at 6:46 AM, Marica Tan <ctan@exist.com> wrote:
>
> > I've completed most of the distributed build manager part and there's
> also
> > nap's transport layer using atlassian's xmlrpc. You can checkout the
> > continuum-distributed-builds branch to test it.
> ...
> > Comments, Suggestions, and/or Violent Reactions are welcome :D
>
> Any further news on this?  I see lots of commits, but not much
> discussion... even having a conversation with yourself will help
> everyone else keep up with changes and possibly get involved. :)
>

Sorry for the lack of updates :-(


>
> One thing I'm concerned about is how I'm going to take a dozen
> individual Continuum instances and get all that data into the new
> master.  Has the model changed?  First priority is all the
> projects/build defs/schedules/etc., and secondly the build results
> which are important for statistics.
>

I only added a new field "distributedBuildEnabled" in the
SystemConfiguration table. Build result is returned to the master once the
build is finished.


So here's how the distributed builds work.

*SLAVE*

- slave or the build agent is a separate web application that builds
projects for the master.
- slave can reside in the same machine as the master.
- slave communicates with the master through XML RPC.
- requirements
  * slave machine must already have maven2, maven1, ant, java, svn, and
other third-party applications needed to build the projects installed
- setup
  *  set SLAVE_HOME
  *  create a continuum-buildagent.xml in SLAVE_HOME/conf/
  *  edit continuum-buildagent.xml

*Sample:

*  <buildOutputDirectory>*/home/slave/buildOutputDirectory*
</buildOutputDirectory>
 <workingDirectory>*/home/slave/workingDirectory*</workingDirectory>
  <continuumServerUrl>*http://localhost:8080/master-xmlrpc*
</continuumServerUrl>
 <installations>
   <installation>
     <name>*Maven 2 Home*</name>
     <type>*maven2*</type>
     <varValue>*/opt/apache-maven-2.0.9*</varValue>
   </installation>
 </installations>

      Slave agents will use the default installations of maven, java and ant
when building projects. Installations in the continuum-buildagent.xml will
only be used   by the master for viewing.

  *  deploy continuum-buildagent-webapp.war
  *  start container


*MASTER*

- master is a continuum instance.
- in the distributed builds mode, the master will still serve all requests
but will delegate all build tasks to the slaves.
- enable/disable distributed builds in the configuration page.


The first thing you have to do is to register the slaves to the master. This
can be done by adding it either in the continuum.xml or using the web UI.
Once you have registered the slaves, the master will start delegating build
tasks in the queue to the slaves. Master will send all information needed by
the slave to build the projects. Once the build is finished, the slave will
send back the build result to the master.


Bugs:
- cancellation of build task


To follow:
- notification
- viewing of slave installations


Question:
1. when there's a connection error in the slave, do we treat it as a
transient error and only log it? ( no build result, set project's state to
previous state )


Will document this later on.


Thanks,
-- 
Marica

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message