www-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikhil Khandelwal <nikhi...@microsoft.com>
Subject Re: Self-Adding Jenkins Build Machines
Date Tue, 07 Jun 2016 18:47:49 GMT
There was no further off-list discussion or conclusion here. Cordova project ended up giving
up the idea of running our tests using Apache Jenkins CI.

We setup our own on Azure[1] and using the Github PR builder plugin which does can use github
comments [2] to indicate status and does not require any extra access to Github which is controlled
by Apache.

[1] http://cordova-ci.cloudapp.net:8080/	
[2] https://github.com/apache/cordova-plugin-file-transfer/pull/146#issuecomment-221606884

Thanks,
Nikhil

On 5/20/16, 1:43 AM, "Michael Dürig" <mduerig@apache.org> wrote:


Hi,

Could you please update this thread with what was the outcome of any 
off-list discussion?

I'm asking as we (the Jackrabbit PMC) might have similar requirements 
for our Oak sub-project. In particular we might be interested in self 
added slaves so the hardware requirements for our tests could be met.

Michael


On 8.4.16 11:26 , Nikhil Khandelwal wrote:
> Hi David, Daniel, Andrew, Jfarrell,
> I would like propose we meet online on google hangouts/skype/slack/IRC/hipchat to discuss
the range of solutions presented here. There is a bunch of discussion back and forth, but
we are blocked now and would like to make more progress here. I can send out a poll to find
a suitable time for us to meet next week to move this further.
>
> It's critical for our project to have PR related testing. We support multiple major platforms
include Android, iOS, Windows, Windows phone, OS X and hence, testing matrix and hardware
is complicated - travis CI & appveyor are not sufficient.
>
> We have over 300 PRs open (s.apache.org/cordovaPulls) in over 40 repos that our project
manages. Our ability to automate testing is critical for our long term success and management
of the high volume of issues and community engagement. We have something working with Apache
buildbot here: ci.cordova.io but it only does post-commit test runs. They have proved to be
less effective than pull request and do not make the PR contributor accountable for breaks.
Hence, we want to implement PR based testing now using Jenkins.
>
> Thanks for your time and help - we really appreciate it.
>
> Thanks,
> Nikhil
>
> -----Original Message-----
> From: Sarangan Rajamanickam
> Sent: Tuesday, April 5, 2016 10:46 AM
> To: builds@apache.org; Nikhil Khandelwal <nikhilkh@microsoft.com>
> Cc: jfarrell@apache.org; Daniel Takamori <pono@apache.org>; andrew bayer <abayer@apache.org>
> Subject: RE: Self-Adding Jenkins Build Machines
>
> Hi David,
>
> Please find the responses below:
>
> 1. Who is providing the machines? How many? Who will have access? (physical and logical)
Microsoft will be providing the slaves (One Windows 10 machine & One Mac OS machine) mentioned.
Microsoft Cordova committers will be having access to these machines.
>
> 2. What access will the ASF have?
> ASF will not have access to these machines.
>
> 3. What's the agreement behind providing the machines?
> These machines will be used for running the CI builds for Apache cordova project in a
periodic basis and also for each PR (and the results will be updated in PR)
>
> 4. Please let us know if you have any more questions. Also, I would like to mention the
fact that a similar setup (Master Apache Machine & Slave Microsoft Machines) is in existence
with the buildbot setup.
> Could you please let us know if there is any difference between allowing buildbot slaves
and Jenkins slave?
>
> Regards
> Sarangan Rajamanickam
> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Wednesday, March 30, 2016 9:35 PM
> To: Nikhil Khandelwal <nikhilkh@microsoft.com>
> Cc: builds@apache.org; jfarrell@apache.org; Daniel Takamori <pono@apache.org>;
andrew bayer <abayer@apache.org>
> Subject: Re: Self-Adding Jenkins Build Machines
>
> I apologize for the lag.
>
> So, I guess I'd like to understand a few things.
> Who is providing the machines? How many? Who will have access?
> (physical and logical)
> What access will the ASF have?
> What's the agreement behind providing the machines.
>
> Historically we've resisted a single project plugging in dedicated resources that can
only be used by that specific project.
>
> --David
>
> On Mon, Mar 21, 2016 at 6:00 PM, Nikhil Khandelwal <nikhilkh@microsoft.com> wrote:
>> Hi David, Daniel, Jake,
>> How can we move this discussion forward? We have great ideas on how to improve testability
of our project and manage the high volume of pull requests that we receive. However, determining
the next steps here are critical to move forward.
>>
>> As Dmitry mentioned below, would a meeting be appropriate to discuss the merits of
each approach you suggested below?
>>
>> Thanks,
>> Nikhil
>>
>> -----Original Message-----
>> From: Dmitry Blotsky [mailto:dblotsky@microsoft.com]
>> Sent: Tuesday, March 15, 2016 6:09 PM
>> To: builds@apache.org
>> Cc: David Nalley <david@gnsa.us>; jfarrell@apache.org; Daniel Takamori
>> <pono@apache.org>
>> Subject: RE: Self-Adding Jenkins Build Machines
>>
>> Hi David, Daniel, Jake,
>>
>> Friendly ping re: joining our own machines to the ASF Jenkins, similar to how we
connected them to the ASF Buildbot. We're working on major improvements to Cordova's CI process,
and we'd just like to know if this scenario is at all possible, even if in the future. We
just want to avoid carrying out work only to find out that this won't be possible.
>>
>> We're also willing to help out with any of the work that needs to be done to enable
this process, if it's indeed possible. And if you guys would prefer a higher bandwidth discussion,
we're available on the Infra Hipchat and we can even set up a meeting on Skype or Hangouts.
>>
>> Again, thank you all very much for your consideration and efforts to accommodate
our scenario.
>>
>> Kindly,
>> Dmitry
>>
>> -----Original Message-----
>> From: Dmitry Blotsky [mailto:dblotsky@microsoft.com]
>> Sent: Friday, March 11, 2016 2:53 PM
>> To: builds@apache.org; jfarrell@apache.org
>> Cc: David Nalley <david@gnsa.us>
>> Subject: RE: Self-Adding Jenkins Build Machines
>>
>> Hi Jake, David,
>>
>> Thank you, Jake, for clarifying the policy regarding a self-hosted Jenkins master.
What about connecting our own build machines to the ASF Jenkins master, similar to what we
did with Buildbot? David, Jake: is that option viable?
>>
>> Kindly,
>> Dmitry
>>
>> -----Original Message-----
>> From: Jake Farrell [mailto:jfarrell@apache.org]
>> Sent: Wednesday, March 9, 2016 4:02 AM
>> To: builds@apache.org
>> Cc: Nikhil Khandelwal <nikhilkh@microsoft.com>; Sarangan Rajamanickam
>> <sarajama@microsoft.com>
>> Subject: Re: Self-Adding Jenkins Build Machines
>>
>> Hey Dmitry
>> We do not open up any cross channels like that for externally controlled machines,
you would have to use jenkins polling and wait for a change to occur and then trigger the
build.
>>
>> -Jake
>>
>> On Tue, Mar 8, 2016 at 4:58 PM, Dmitry Blotsky
>> <dblotsky@microsoft.com>
>> wrote:
>>
>>> Hi David,
>>>
>>> Thanks for your response! Re: our own build physical machines, we
>>> want to use them because we expect to frequently change their
>>> environments (cumbersome with Docker) and to have physical devices
>>> connected (impossible with VMs). We've connected our own machines to
>>> the ASF Buildbot with no problems before; is Jenkins a very different
>>> scenario from Buildbot security-wise?
>>>
>>> To answer Alex's question: we want to run on builds.a.o because it
>>> already has the ASF Git and GitHub repos hooked up to it. We've
>>> definitely considered running our own CI like CloudStack does, but
>>> ASF Git/GitHub integration is a major hurdle. We've had a lot of
>>> friction setting this up in the past on Buildbot.
>>>
>>> David: if we do go the route of running our own Jenkins, how feasible
>>> would it be to set up ASF Git and GitHub integration with it?
>>>
>>> Kindly,
>>> Dmitry
>>>
>>> -----Original Message-----
>>> From: David Nalley [mailto:david@gnsa.us]
>>> Sent: Monday, March 7, 2016 9:00 PM
>>> To: builds@apache.org
>>> Cc: Nikhil Khandelwal <nikhilkh@microsoft.com>; Sarangan Rajamanickam
>>> < sarajama@microsoft.com>
>>> Subject: Re: Self-Adding Jenkins Build Machines
>>>
>>> Hi Dmitry,
>>>
>>> builds.a.o (like ci.a.o) is a relatively generic resources - we have
>>> ~20 Jenkins nodes and that is around to serve all of the ASF.
>>>
>>> In the past, we've created project-specific VMs or physical machines
>>> when they've had unusual requirements. That's been pretty mixed
>>> though
>>> - in some cases we have had VMs or even entire physical machines go
>>> unused for years despite them being absolutely necessary when requested.
>>>
>>> Technology has also moved along (or at least become more accessible),
>>> and containers have become ubiquitous among developers - thus
>>> esoteric build environments are pretty easy to fulfill with things
>>> like Docker (builds.a.o has support for Docker).
>>>
>>> Additionally, at present our build slaves are trusted to the same
>>> degree as out master - this means if we did accept these slaves, we'd
>>> probably want exclusive access. We are also trying to standardize;
>>> and having special snowflake nodes runs counter to that.
>>>
>>> I realize this means builds.a.o isn't a good fit for every project.
>>> In fact a number of projects have needs we simply can't easily meet,
>>> and they manage their own CI environment. (CloudStack, Traffic
>>> Server, Ignite, etc)
>>>
>>> --David
>>>
>>> On Mon, Mar 7, 2016 at 7:05 PM, Dmitry Blotsky
>>> <dblotsky@microsoft.com>
>>> wrote:
>>>> Dear Infra folks,
>>>>
>>>> My name is Dmitry Blotsky and I'm one of the committers on the
>>>> Apache
>>> Cordova project. I'm writing to please ask whether it's possible for
>>> us to connect our own build machines to the Jenkins build machine
>>> fleet. Our builds have a bunch of mobile-specific requirements
>>> (emulators, devices,
>>> SDKs) and cross-platform requirements (OS X, Windows), and we already
>>> have machines set up to handle them.
>>>>
>>>> Is there a process to do this? If not, could we help define one in
>>>> any
>>> way?
>>>>
>>>> Thank you in advance for your help!
>>>>
>>>> Kindly,
>>>> Dmitry
>>>


Mime
View raw message