Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mxnet.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CABC2O9oK_=7rV6Zfai15YsAEa21DeuJqT=578OgdBvUrG2jbww@mail.gmail.com>
References: <CABC2O9rNcmnuuEPHTYuXc3bHnuYvFYoTqF6ySQKfiB8OOFKWOw@mail.gmail.com>
 <CABC2O9rjGCth2nY0WbT-6zq7C2WjRXKunmRQTWMaS9aGTJKbhg@mail.gmail.com>
 <CAAjLvt=2OHV+iRYEZr0a_fFrSjsN0=UNPX5=VUpDKDGZnSQSGQ@mail.gmail.com>
 <CACZfuxaU3OXGeX=SDOGfKz6rd0YJYYWmdE2om_7E=z4CYHO17Q@mail.gmail.com>
 <CABgAAfeAQiLEiK5WPf1X5RsBVADcemyekFiiYRH19ihyTZCUSg@mail.gmail.com>
 <CA+7OP3ipTWVMDU40ioFL8MWQ+UYRATozWhspQvgw9ygB0tFqYQ@mail.gmail.com>
 <CABgAAfcY9uQeH9U=E3ctaXAdiYTcZG3N2=djmgyaLmNhcvM4sg@mail.gmail.com>
 <CABgAAfcCt8WMp3R0qggAM4w9n1WTJW828WPF=7c6XsU9_Ag7ww@mail.gmail.com>
 <CA+7OP3ie8VWvVT6476s-qCBf3rCkvrumBstOUsA6Av4BSdyu_g@mail.gmail.com>
 <CA+7OP3gxRs8zFihMA8bnv5FAZjqcP=V5siXY4ZEk-bQUegw+yw@mail.gmail.com> <CABC2O9oK_=7rV6Zfai15YsAEa21DeuJqT=578OgdBvUrG2jbww@mail.gmail.com>
From: Chris Olivier <cjolivier01@gmail.com>
Date: Fri, 20 Oct 2017 16:38:01 -0700
Message-ID: <CABgAAfdGB8GLbLFC2RcYGXroo5RS0zYv+iW+JXruSz95sKRxjg@mail.gmail.com>
Subject: Re: [Proposal] Stabilizing Apache MXNet CI build system
To: dev@mxnet.incubator.apache.org
Content-Type: multipart/alternative; boundary="001a11447da4b326be055c02f681"
archived-at: Fri, 20 Oct 2017 23:38:10 -0000

--001a11447da4b326be055c02f681
Content-Type: text/plain; charset="UTF-8"

I believe that Mu already started that discussion about using old mxnet.io
Jenkins server.   I expect deciding whether to replace would hinge in large
part upon what it would be replaced with.

On Fri, Oct 20, 2017 at 4:30 PM, sandeep krishnamurthy <
sandeep.krishna98@gmail.com> wrote:

> Chris: If the community decides to go with separate setup, then there will
> be a tech design discussion and CodeCommit / Jenkins / Travis such
> proposals will be covered and discussed.
>
> Thanks,
> Sandeep
>
> On Fri, Oct 20, 2017 at 4:22 PM, Seb Kiureghian <sebouhdk@gmail.com>
> wrote:
>
> > But the feather can definitely be added once MXNet graduates.
> >
> > On Fri, Oct 20, 2017 at 4:21 PM, Seb Kiureghian <sebouhdk@gmail.com>
> > wrote:
> >
> > > The feather can only be used by Top Level Projects.
> > >
> > > On Fri, Oct 20, 2017 at 4:19 PM, Chris Olivier <cjolivier01@gmail.com>
> > > wrote:
> > >
> > >> When the word Apache is in the Hadoop logo (not always), it includes
> the
> > >> feather and color scheme.
> > >>
> > >> On Fri, Oct 20, 2017 at 4:18 PM, Chris Olivier <cjolivier01@gmail.com
> >
> > >> wrote:
> > >>
> > >>> Thanks.
> > >>>
> > >>> Is there any way to work the feather into it?
> > >>>
> > >>> i.e.  https://goo.gl/images/BU4dnG
> > >>>
> > >>> On Fri, Oct 20, 2017 at 4:11 PM, Seb Kiureghian <sebouhdk@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> https://imgur.com/a/aADkA
> > >>>>
> > >>>> On Fri, Oct 20, 2017 at 4:07 PM, Chris Olivier <
> cjolivier01@gmail.com
> > >
> > >>>> wrote:
> > >>>>
> > >>>> > Why don;t we look into fully managed AWS CodeBuild?  It maintains
> > >>>> > everything. It's also compatible with Jenkins.
> > >>>> >
> > >>>> > On Fri, Oct 20, 2017 at 1:51 PM, Tianqi Chen <
> > >>>> tqchen@cs.washington.edu>
> > >>>> > wrote:
> > >>>> >
> > >>>> > > +1
> > >>>> > >
> > >>>> > > Tianqi
> > >>>> > > On Fri, Oct 20, 2017 at 1:39 PM Mu Li <muli.cmu@gmail.com>
> wrote:
> > >>>> > >
> > >>>> > > > +1
> > >>>> > > >
> > >>>> > > >
> > >>>> > > > It seems that the Apache CI is quite overloaded these days,
> and
> > >>>> MXNet's
> > >>>> > > CI
> > >>>> > > > pipeline is too complex to run there. In addition, we may need
> > to
> > >>>> add
> > >>>> > > more
> > >>>> > > > devices, e.g. macpro and rasbperry pi, into the server, and
> more
> > >>>> tasks
> > >>>> > > such
> > >>>> > > > as pip build. It means a lot of requests to the Infra team.
> > >>>> > > >
> > >>>> > > > We can reuse our previous Jenkins server at
> http://ci.mxnet.io/
> > .
> > >>>> But
> > >>>> > we
> > >>>> > > > probably need a dedicate developer to maintain it.
> > >>>> > > >
> > >>>> > > >
> > >>>> > > >
> > >>>> > > > On Fri, Oct 20, 2017 at 1:01 PM, sandeep krishnamurthy <
> > >>>> > > > sandeep.krishna98@gmail.com> wrote:
> > >>>> > > >
> > >>>> > > > > Hello all,
> > >>>> > > > >
> > >>>> > > > > I am hereby opening up a discussion thread on how we can
> > >>>> stabilize
> > >>>> > > Apache
> > >>>> > > > > MXNet CI build system.
> > >>>> > > > >
> > >>>> > > > > Problems:
> > >>>> > > > >
> > >>>> > > > > ========
> > >>>> > > > >
> > >>>> > > > > Recently, we have seen following issues with Apache MXNet CI
> > >>>> build
> > >>>> > > > systems:
> > >>>> > > > >
> > >>>> > > > >    1. Apache Jenkins master is overloaded and we see issues
> > >>>> like -
> > >>>> > > unable
> > >>>> > > > >    to trigger builds, difficult to load and view the blue
> > ocean
> > >>>> and
> > >>>> > > other
> > >>>> > > > >    Jenkins build status page.
> > >>>> > > > >    2. We are generating too many request/interaction on
> Apache
> > >>>> Infra
> > >>>> > > > team.
> > >>>> > > > >       1. Addition/deletion of new slave: Caused from scaling
> > >>>> > activity,
> > >>>> > > > >       recycling, troubleshooting or any actions leading to
> > >>>> change of
> > >>>> > > > slave
> > >>>> > > > >       machines.
> > >>>> > > > >       2. Plugins / other Jenkins Master configurations.
> > >>>> > > > >       3. Experimentation on CI pipelines.
> > >>>> > > > >    3. Harder to debug and resolve issues - Since access to
> > >>>> master and
> > >>>> > > > slave
> > >>>> > > > >    is not with the same community, it requires Infra and
> > >>>> community to
> > >>>> > > > dive
> > >>>> > > > >    deep together on all action items.
> > >>>> > > > >
> > >>>> > > > > Possible Solutions:
> > >>>> > > > >
> > >>>> > > > > ==============
> > >>>> > > > >
> > >>>> > > > >    1. Can we set up a separate Jenkins CI build system for
> > >>>> Apache
> > >>>> > MXNet
> > >>>> > > > >    outside Apache Infra?
> > >>>> > > > >    2. Can we have a separate Jenkins Master in Apache Infra
> > for
> > >>>> > MXNet?
> > >>>> > > > >    3. Review design of current setup, refine and fill the
> > gaps.
> > >>>> > > > >
> > >>>> > > > > @ Mentors/Infra team/Community:
> > >>>> > > > >
> > >>>> > > > > ==========================
> > >>>> > > > >
> > >>>> > > > > Please provide your suggestions on how we can proceed
> further
> > >>>> and
> > >>>> > work
> > >>>> > > on
> > >>>> > > > > stabilizing the CI build systems for MXNet.
> > >>>> > > > >
> > >>>> > > > > Also, if the community decides on separate Jenkins CI build
> > >>>> system,
> > >>>> > > what
> > >>>> > > > > important points should be taken care of apart from the
> below:
> > >>>> > > > >
> > >>>> > > > >    1. Community being able to access the build page for
> build
> > >>>> > statuses.
> > >>>> > > > >    2. Committers being able to login with apache
> credentials.
> > >>>> > > > >    3. Hook setup from apache/incubator-mxnet repo to Jenkins
> > >>>> master.
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > > Irrespective of the solution we come up, I think we should
> > >>>> initiate a
> > >>>> > > > > technical design discussion on how to setup the CI build
> > system.
> > >>>> > > > Probably 1
> > >>>> > > > > or 2 pager documents with the architecture and review with
> > >>>> Infra and
> > >>>> > > > > community members.
> > >>>> > > > >
> > >>>> > > > > ***There were few proposal and discussion on the slack
> > channel,
> > >>>> to
> > >>>> > > reach
> > >>>> > > > > wider community members, moving that discussion formally to
> > this
> > >>>> > list.
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > > My Proposal: Option 1 - Set up separate Jenkins CI build
> > system.
> > >>>> > > > >
> > >>>> > > > > Thanks,
> > >>>> > > > >
> > >>>> > > > > Sandeep
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > > --
> > >>>> > > > > Sandeep Krishnamurthy
> > >>>> > > > >
> > >>>> > > >
> > >>>> > >
> > >>>> >
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> >
>
>
>
> --
> Sandeep Krishnamurthy
>

--001a11447da4b326be055c02f681--