continuum-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emmanuel Venisse" <emmanuel.veni...@gmail.com>
Subject Re: Distributed builds
Date Tue, 18 Nov 2008 10:09:59 GMT
On Tue, Nov 18, 2008 at 6:49 AM, Marica Tan <ctan@exist.com> wrote:

> Additional proposal for distributed builds
>
> 1. Need to have a separate builder for distributed and non-distributed
> builds
>    - non-distributed builds: group level update then build project one by
> one.
>    - distributed builds: update and build project one by one.


At the same time, I'd like to see only one checkout directory for projects
with nested modules instead of a duplicated checkout directory that we have
actually.

Why a one by one build for distributed build? Can it be possible to let the
user to choose?


>
> 2. Continuum server is considered as the local build agent. It will be
> included when looking for an available build agent.
>
> 3. Central remote repository
>    - for local build agent
>      a. add remote repository to settings.xml
>    - for remote build agents
>      a. add remote repository to settings.xml


Where? on the continuum machine or on the agent machine?

>
>      b. M1 or M2 projects: deploy artifacts recently installed in the local
> repo to the remote repo.
>      c. ANT or SHELL projects: don't know yet how to deploy artifacts for
> these type of projects. so maybe limit the distributed builds for M1 and/or
> M2 projects only?


With the plugin system we want to add, we'll can create a plugin that will
work on the generated artifacts, a user will can choose a file to manipulate
after the build, for example he will can choose to copy the generated
artifact onto a repository.

>
>
> 4. Remote build agents will not use any database. All information will be
> stored in a configuration file. Continuum server (master agnet) system
> administrator's credentials will be stored in the settings.xml as well.


I hope local build agents won't use it too. Local and remote agents must
have the same code.


>
>
> WDYT?
>
> Thanks,
> Marica
>
>
> On Fri, Oct 24, 2008 at 12:34 PM, Emmanuel Venisse <
> emmanuel.venisse@gmail.com> wrote:
>
> > On Thu, Oct 23, 2008 at 4:23 PM, Brett Porter <brett@apache.org> wrote:
> >
> > >
> > > On 23/10/2008, at 9:38 PM, Emmanuel Venisse wrote:
> > >
> > >
> > >> The process is:
> > >>
> > >> if latest agent version isn't deployed on the agent server
> > >>   copy the agent to the agent server
> > >> end if
> > >> start ssh session
> > >> launch agent with the project context (agent is a simple standalone to
> > >> start
> > >> when needed, it isn't a service)
> > >> stop ssh session
> > >>
> > >> With this mechanism, we get in real time the output of the agent
> instead
> > >> of
> > >> to get the build result at the end. It's important to can to follow
> the
> > >> build output during the build so the build result page is updated
> > >> automatically.
> > >>
> > >
> > > I think maybe this is a good alternative to add in the future, but
> > probably
> > > not the only way. It might have some drawbacks, like:
> > > * long builds and flaky connections would lose the build if the
> > connection
> > > drops
> > > * a nuisance to run on Windows :)
> > >
> > > I think maybe the most robust for the future is JMS like David B had in
> > > place earlier, since you can use the built in features to make sure
> build
> > > results never get lost, and can probably handle some servers going
> down.
> > >
> > > But I also like XMLRPC because it already exists so is a good starting
> > > point - we can get something working and then improve later by slimming
> > down
> > > the builder and improving the communication medium.
> > >
> > > WDYT?
> >
> >
> > Ok
> >
> >
> > >
> > >
> > > Cheers,
> > > Brett
> > >
> > > --
> > > Brett Porter
> > > brett@apache.org
> > > http://blogs.exist.com/bporter/
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message