impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Behm <alex.b...@cloudera.com>
Subject Re: Jenkins jobs to verify patches
Date Wed, 14 Dec 2016 00:55:21 GMT
The way I see it we have two competing interests that need to be reconciled:
1. We want to encourage contributions by making it as easy as possible to
do private test runs. This speaks in favor of allowing any contributor to
run on jenkins.impala.io.
2. Cloudera pays the bill for the EC2 Jenkins workers, so we do need to
protect Cloudera against abuse.

I am strongly in favor of (1) and I think we can mitigate the damage by
(2). My proposal is as follows:

- We provide an anonymous private build-and-test job for all contributors.
We limit the number of concurrent executors for that job to minimize the
financial damage of abuse.
- We provide a committer-only private build-and-test job that allows
significantly more concurrent executors.

Both jobs accept as input a link to a gerrit code review (could be a
draft). That way we have a secure audit trail in case of abuse.

On Tue, Dec 13, 2016 at 1:51 PM, Jim Apple <jbapple@cloudera.com> wrote:

> Before Impala joined the ASF, code reviews had to go through a
> "verification" step in which a Jenkins jobs downloaded the patch, ran
> all tests and replied back to Jenkins that everything was OK. That
> Jenkins job ran on Cloudera infrastructure and could not be accessed,
> even in a read-only way, by people outside of Cloudera.
>
> To follow the Apache way, I am laboring to replace that with a Jenkins
> server that can be used by any authorized person and read by any
> person. It is at http://jenkins.impala.io:8080. It is able to verify
> patches just like the old Jenkins machine. A few remaining questions:
>
> 1. What should the prerequisites be to turning off the Cloudera-only
> Jenkins verification path?
>
> 2. Who should be able to run jobs on jenkins.impala.io? Some
> possibilities: Committers only, anyone who asks, PMC members only,
> contributors who ask after submitting 5 patches. Higher bars lead to
> less likelihood of abuse, lower ones to easier contributions from
> newbies.
>
> My proposal is this:
>
> 1. We should turn off the CLoudera-only Jenkins verification path
> January 2. jenkins.impala.io is in pretty good shape, and we can make
> further improvements as needed. For instance, it took me five minutes
> just now to cut down on the not-so-interesting debug output from the
> main verification job,
> http://jenkins.impala.io:8080/job/ubuntu-14.04-from-scratch/
>
> 2. Everyone with five patches can request a login.
>
> I'm not married to these ideas, but I wanted to provide a jumping-off
> point for discussion.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message