hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Local repo sharing for maven builds
Date Tue, 06 Oct 2015 09:35:26 GMT

> On 5 Oct 2015, at 19:45, Colin McCabe <cmccabe@alumni.cmu.edu> wrote:
> On Mon, Sep 28, 2015 at 12:52 AM, Steve Loughran <stevel@hortonworks.com> wrote:
>> the jenkins machines are shared across multiple projects; cut the executors to 1/node
and then everyone's performance drops, including the time to complete of all jenkins patches,
which is one of the goals.
> Hi Steve,
> Just to be clear, the proposal wasn't to cut the executors to 1 per
> node, but to have multiple Docker containers per node (perhaps 3 or 4)
> and run each executor in an isolated container.  At that point,
> whatever badness Maven does on the .m2 stops being a problem for
> concurrently running jobs.

I'd missed that bit. Yes, something with a containerized ~//m2 repo gets the isolation without
playing with mvn  version fixup

> I guess I don't feel that strongly about this, but the additional
> complexity of the other solutions (like running a "find" command in
> .m2, or changing artifactID) seems like a disadvantage compared to
> just using multiple containers.  And there may be other race
> conditions here that we're not aware of... like a TOCTOU between
> checking for a jar in .m2 and downloading it, for example.  The
> Dockerized solution skips all those potential failure modes and
> complexity.
> cheers,
> Colin

View raw message