mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianqi Chen <tqc...@cs.washington.edu>
Subject Re: [Proposal] Stabilizing Apache MXNet CI build system
Date Fri, 20 Oct 2017 20:51:32 GMT
+1

Tianqi
On Fri, Oct 20, 2017 at 1:39 PM Mu Li <muli.cmu@gmail.com> wrote:

> +1
>
>
> It seems that the Apache CI is quite overloaded these days, and MXNet's CI
> pipeline is too complex to run there. In addition, we may need to add more
> devices, e.g. macpro and rasbperry pi, into the server, and more tasks such
> as pip build. It means a lot of requests to the Infra team.
>
> We can reuse our previous Jenkins server at http://ci.mxnet.io/. But we
> probably need a dedicate developer to maintain it.
>
>
>
> On Fri, Oct 20, 2017 at 1:01 PM, sandeep krishnamurthy <
> sandeep.krishna98@gmail.com> wrote:
>
> > Hello all,
> >
> > I am hereby opening up a discussion thread on how we can stabilize Apache
> > MXNet CI build system.
> >
> > Problems:
> >
> > ========
> >
> > Recently, we have seen following issues with Apache MXNet CI build
> systems:
> >
> >    1. Apache Jenkins master is overloaded and we see issues like - unable
> >    to trigger builds, difficult to load and view the blue ocean and other
> >    Jenkins build status page.
> >    2. We are generating too many request/interaction on Apache Infra
> team.
> >       1. Addition/deletion of new slave: Caused from scaling activity,
> >       recycling, troubleshooting or any actions leading to change of
> slave
> >       machines.
> >       2. Plugins / other Jenkins Master configurations.
> >       3. Experimentation on CI pipelines.
> >    3. Harder to debug and resolve issues - Since access to master and
> slave
> >    is not with the same community, it requires Infra and community to
> dive
> >    deep together on all action items.
> >
> > Possible Solutions:
> >
> > ==============
> >
> >    1. Can we set up a separate Jenkins CI build system for Apache MXNet
> >    outside Apache Infra?
> >    2. Can we have a separate Jenkins Master in Apache Infra for MXNet?
> >    3. Review design of current setup, refine and fill the gaps.
> >
> > @ Mentors/Infra team/Community:
> >
> > ==========================
> >
> > Please provide your suggestions on how we can proceed further and work on
> > stabilizing the CI build systems for MXNet.
> >
> > Also, if the community decides on separate Jenkins CI build system, what
> > important points should be taken care of apart from the below:
> >
> >    1. Community being able to access the build page for build statuses.
> >    2. Committers being able to login with apache credentials.
> >    3. Hook setup from apache/incubator-mxnet repo to Jenkins master.
> >
> >
> > Irrespective of the solution we come up, I think we should initiate a
> > technical design discussion on how to setup the CI build system.
> Probably 1
> > or 2 pager documents with the architecture and review with Infra and
> > community members.
> >
> > ***There were few proposal and discussion on the slack channel, to reach
> > wider community members, moving that discussion formally to this list.
> >
> >
> > My Proposal: Option 1 - Set up separate Jenkins CI build system.
> >
> > Thanks,
> >
> > Sandeep
> >
> >
> >
> > --
> > Sandeep Krishnamurthy
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message