mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sandeep krishnamurthy <sandeep.krishn...@gmail.com>
Subject Fwd: [Proposal] Stabilizing Apache MXNet CI build system
Date Fri, 20 Oct 2017 20:01:12 GMT
Hello all,

I am hereby opening up a discussion thread on how we can stabilize Apache
MXNet CI build system.

Problems:

========

Recently, we have seen following issues with Apache MXNet CI build systems:

   1. Apache Jenkins master is overloaded and we see issues like - unable
   to trigger builds, difficult to load and view the blue ocean and other
   Jenkins build status page.
   2. We are generating too many request/interaction on Apache Infra team.
      1. Addition/deletion of new slave: Caused from scaling activity,
      recycling, troubleshooting or any actions leading to change of slave
      machines.
      2. Plugins / other Jenkins Master configurations.
      3. Experimentation on CI pipelines.
   3. Harder to debug and resolve issues - Since access to master and slave
   is not with the same community, it requires Infra and community to dive
   deep together on all action items.

Possible Solutions:

==============

   1. Can we set up a separate Jenkins CI build system for Apache MXNet
   outside Apache Infra?
   2. Can we have a separate Jenkins Master in Apache Infra for MXNet?
   3. Review design of current setup, refine and fill the gaps.

@ Mentors/Infra team/Community:

==========================

Please provide your suggestions on how we can proceed further and work on
stabilizing the CI build systems for MXNet.

Also, if the community decides on separate Jenkins CI build system, what
important points should be taken care of apart from the below:

   1. Community being able to access the build page for build statuses.
   2. Committers being able to login with apache credentials.
   3. Hook setup from apache/incubator-mxnet repo to Jenkins master.


Irrespective of the solution we come up, I think we should initiate a
technical design discussion on how to setup the CI build system. Probably 1
or 2 pager documents with the architecture and review with Infra and
community members.

***There were few proposal and discussion on the slack channel, to reach
wider community members, moving that discussion formally to this list.


My Proposal: Option 1 - Set up separate Jenkins CI build system.

Thanks,

Sandeep



-- 
Sandeep Krishnamurthy

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message