www-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Mason ...@jmason.org>
Subject Re: Move builds off of Hudson master
Date Thu, 09 Jul 2009 12:26:54 GMT
On Tue, Jul 7, 2009 at 07:23, Paul Querna<paul@querna.org> wrote:
> I am mostly curious how adding more build slaves solves these
> reliability problems, since they all seem to stem from builds taking
> excessive amounts of time, freezing, or having whacky OOM issues.

After this mail, I spent a couple of days monitoring "hung" builds.

There were a couple which were indeed frozen/OOMing tests.  These have
been resolved through use of the "build timeout" plugin, which is now
set to time out long-running builds after 2 hours for those projects.
I haven't observed any builds that the build timeout couldn't deal
with, btw.

However, the majority of backlogs were due to contention for the
limited number of executors; particularly the 2 on the main instance.
There are a few projects that perform 1.5-hour Maven deployments from
this.  While these were going on, it was routine to see ~15 other
builds queueing up.

So IMO, yep, we really do need to expand the executor pool.


View raw message