continuum-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lustig, Marc (Allianz Deutschland, externer Mitarbeiter)" <EXTERN.LUSTIG_M...@allianz.de>
Subject AW: AW: running n parallel Continuum installations
Date Tue, 23 Sep 2008 13:32:22 GMT
thanks, Simon, for your input!
I didn't recognise that all those transient jars which are downloaded as dependencies (and
installed in the local repo) are potentially producing a race-condition, too.

Btw., I think Hudson solves its parallel building approach by distributing build-processes
into nodes, whereas a node is a separate server.

Using Continuum I see two possible solutions in order to allow multiple instances on the same
machine.  Either deploying each instance in a separate servlet-container, and starting this
container with a distinct user. This allows for pretty fine isolation, but consumes more memory-resources,
as each instances needs its own servlet-container.

Or to follow your proposal and start maven with the --settings option.
This sounds straight-forward, but I see two open questions:
1) how to add the --settings option _automatically_ (not for each build-definition)
2) what about the maven-calls that use the Continuum-embedded Maven-distribution? 

regards
Marc




-----Urspr√ľngliche Nachricht-----
Von: Simon Kitching [mailto:skitching@apache.org] 
Gesendet: Dienstag, 23. September 2008 14:48
An: users@continuum.apache.org
Betreff: Re: AW: running n parallel Continuum installations

Hi Marc,

You might get away with this if your apps have absolutely *no* transient 
dependencies in common. But that is unlikely.

As I understand the current maven situation, if you have two projects 
that depend on foo-1.0-snapshot for example, then two builds running 
parallel *using the same repo* may try to download a new version of this 
file concurrently and both try to write simultaneously to the same files 
in the repo. You then have corrupted files in the repo which is 
obviously not good.

Maven supports the "--settings" commandline option to specify an 
alternate settings file. And a settings file can specify where the maven 
repository is kept. So I would recommend that each of your "parallel" 
instances use a different settings.xml file with a separate repository. 
This will use more diskspace, as each repo will have its own copy of the 
dependencies - but no write-race conditions can occur as the repos are 
separate. Note, however, that this means that two parallel instances 
will not see each other's built artifacts; they will only see artifacts 
that have been deployed to a remote maven repository that is listed in 
the <repository> section. In practice, this setup would be just like 
having a continuum installed on multiple physical servers. Or like 
having each continuum instance running as a different user (where again 
there would be a separate repo for each user).

If you could somehow prevent any downloads of new files into the repo 
from succeeding, eg by defining a <mirrorOf> entry that points to an 
always-empty repo then that should also work, as long as two parallel 
builds never try to build+install the same artifact. But of course then 
if someone updates a pom.xml to add a new dependency then you will 
somehow have to load that new dependency manually into your repo. That 
seems like a hassle to manage, but might be ok if you already have 
strict QA rules on adding new dependencies.

Regards,
Simon


Lustig, Marc (Allianz Deutschland, externer Mitarbeiter) schrieb:
> so, if each mvn-project exists in only one of the Continuum-instances, no conflict should
occur, right?  
> Deploying the artifact into Archiva is synchronized anyway, I suppose.
>
>
> -----Urspr√ľngliche Nachricht-----
> Von: Emmanuel Venisse [mailto:emmanuel.venisse@gmail.com] 
> Gesendet: Dienstag, 23. September 2008 13:39
> An: users@continuum.apache.org
> Betreff: Re: running n parallel Continuum installations
>
> The potential issue with a single local repo with a parallel build is if two
> build try to download or install the same artifact in it.
> Emmanuel
>
> On Tue, Sep 23, 2008 at 12:01 PM, Lustig, Marc (Allianz Deutschland,
> externer Mitarbeiter) <EXTERN.LUSTIG_MARC@allianz.de> wrote:
>
>   
>> Our project contains several hundred Maven-projects, which need to get
>> tested and snaphots built using Continuum.
>> My plan is to install a number of Continuum-webapps into a single JBoss
>> AS-container, each one having its dedicated database.
>> Anybody sees any potential issues in such scenario?
>>
>> (I read on the list that there is an issue related to accessing the local
>> mvn-repo, but I don't understand what actually might cause any trouble
>> here?!)
>>
>>     
>
>   


Mime
View raw message