mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@gmail.com>
Subject Re: MXNet CD pipelines cost savings
Date Fri, 14 Feb 2020 13:21:04 GMT
Excellent, thanks a lot for your efforts, sheng!

-Marco

Sheng Zha <zhasheng@apache.org> schrieb am Mi., 12. Feb. 2020, 22:57:

> Hi Marco,
>
> I've taken up the task to fix the CD pipeline [1] and it's pending CI
> checks and merge. I also made the adjustments to the namespace layout of
> the mxnet s3 bucket and updated the static page which is now accessible
> from https://repo.mxnet.io/dist/index.html.
>
> Meanwhile, the access from the CodeBuild to publish to the mxnet s3 bucket
> has been revoked so its status should no longer be relevant. I will update
> the installation doc to reflect only the artifacts published by Jenkins CD.
> I hope this brings conclusion to this situation.
>
> Let me know if there's any further question.
>
> -sz
>
> [1] https://github.com/apache/incubator-mxnet/pull/17551
>
> On 2020/02/10 23:57:47, Marco de Abreu <marco.g.abreu@gmail.com> wrote:
> > Thanks for bringing this to our attention.
> >
> > I'm quite devastated that our two vetoes have been ignored and the
> > CodeBuild pipeline is actually supplying our user-facing binaries.
> > Suggesting to turn of the Jenkins based CD now adds insult to injury.
> >
> > I'd like to hear a plan how to make the project compliant again. I
> already
> > announced that I will remove any mentions of that publishing method
> (speak,
> > all links on our website pointing to the bucket) if the sourcing system
> is
> > not our Jenkins CD. So far I believed that this was actually done, but
> > seems like we got played here.
> >
> > For the sake of our users, I'm giving one week (until 2/17) to come up
> with
> > a proposal and until the end of the month 2/29 to have CodeBuild turned
> off
> > and Jenkins CD fixed. Considering we have been tricked last time, I want
> to
> > have confirmation that CodeBuild has been turned off and a description
> how
> > we can verify that all artifacts are now coming from Jenkins CD.
> >
> > Best regards
> > Marco
> >
> > Zha, Sheng <zhasheng@amazon.com.invalid> schrieb am Mo., 10. Feb. 2020,
> > 19:40:
> >
> > > +dev@
> > >
> > > -sz
> > >
> > > On Feb 10, 2020, at 1:35 PM, Zha, Sheng <zhasheng@amazon.com> wrote:
> > >
> > >  As already stated in the public threads, I’ve vetoed the CodeBuild
> > > solution from becoming the long term solution as it’s not publicly
> > > manageable.
> > >
> > > As communicated before, the team should have put efforts in maintaining
> > > and fixing the Jenkins CD pipeline but has neglected to do so.
> Promoting
> > > the CodeBuild solution this way is a step in the wrong direction that
> has
> > > to be stopped.
> > >
> > > -sz
> > >
> > > On Feb 10, 2020, at 1:13 PM, Davydenko, Denis <dden@amazon.com> wrote:
> > >
> > > 
> > > Hello guys,
> > >
> > > I would like to start this discussion so that we can align on handling
> CD
> > > pipelines we currently have. There are two of them: one in Jenkins<
> > > http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/> and
> one
> > > in CodeBuild<https://tiny.amazon.com/1h49a01qg/IsenLink>. The one in
> > > Jenkins is currently functioning but its runs are always failing. The
> one
> > > in CodeBuild is currently functioning and publishing artifacts to S3
> bucket<
> > > https://tiny.amazon.com/39negmk0/IsenLink>.
> > >
> > > MXNet Engineering team proposal is to shut down Jenkins based CD
> > > completely as it is currently just a waste of resources and use
> CodeBuild
> > > based setup to continue publishing nightly builds to S3 bucket, which
> > > provides public access to all binaries stored in it. This doesn’t
> affect a
> > > discussion of whether to publish binaries to S3 or to pypi – once that
> > > concludes (if ever) we can switch destination of CodeBuild projects so
> that
> > > they would upload MXNet nightly binaries to pypi instead of S3.
> > >
> > > This is an effort to get alignment internally, if possible, before
> > > bringing this as a proposal for community discussion.
> > >
> > > --
> > > Thanks,
> > > Denis
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message