mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@googlemail.com.INVALID>
Subject Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Date Wed, 21 Nov 2018 17:31:04 GMT
Please notice that the "continuous-integration/jenkins/pr-merge" currently
is overlapping with the new pipelines. Please make sure all checks pass
(also the non-required ones) before merging the PRs. I will work on a fix
for this overlap.

-Marco

On Wed, Nov 21, 2018 at 5:42 PM Anton Chernov <mechernov@gmail.com> wrote:

> The ability to retrigger the pipelines separately is an amazing step
> forward. Great job Marco!
>
> ср, 21 нояб. 2018 г. в 15:03, Marco de Abreu
> <marco.g.abreu@googlemail.com.invalid>:
>
> > Hello,
> >
> > the PR has been merged and I've created the new pipelines at [1]. You can
> > see the new reports if you have a look at this example PR at [2].
> >
> > The new status messages will be the ones starting with
> > "ci/jenkins/mxnet-validation/".
> >
> > This now allows you to retrigger specific pipelines if they fail. For
> > example, if you're interested in the website pipeline, you can now go to
> > [3] and just retrigger that instead of running the entire suite. Whenever
> > there's a new commit, all pipelines will still be scheduled as before
> (the
> > overall behaviour or coverage of our pipeline did not change, I just
> > decoupled them and increased the usability).
> >
> > The next step will be the deprecation of the main Jenkinsfile (the one
> > which reports the status as "continuous-integration/jenkins/pr-merge")
> and
> > requesting these new statuses to be marked as required (protected master
> > branch). Since we have to change some reporting tools to point to the new
> > jobs and I'd like to observe the stability for some time, this will take
> > some times.
> >
> > You can now resume changes in the Jenkinsfiles. But please do not modify
> > the Jenkinsfile in the root directory but instead the ones at [4]. The
> > nightly Jenkinsfiles (or basically all Jenkinsfiles that are not part of
> > the main pipeline) have not been migrated yet and I will do that at a
> later
> > point in time.
> >
> > Best regards,
> > Marco
> >
> > [1]: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/
> > [2]: https://github.com/apache/incubator-mxnet/pull/13352
> > [3]:
> >
> >
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-13352/1/pipeline
> > [4]: https://github.com/apache/incubator-mxnet/tree/master/ci/jenkins
> >
> > On Tue, Nov 20, 2018 at 9:33 PM Marco de Abreu <
> > marco.g.abreu@googlemail.com>
> > wrote:
> >
> > > I have just submitted my PR at
> > > https://github.com/apache/incubator-mxnet/pull/13344. Test jobs are
> > > available at
> > > http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
> > >
> > > As soon as I'm done with my tests, I will mark it as ready for review.
> > >
> > > Best regards,
> > > Marco
> > >
> > > On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <
> > > marco.g.abreu@googlemail.com> wrote:
> > >
> > >> Thanks, Pedro!
> > >>
> > >> I have also been looking into that issue, but it seems like this would
> > >> require changes in the groovy interpreter of Jenkins. From what I can
> > tell,
> > >> a refactor will give us multiple benefits (clarity and speed) aside
> from
> > >> resolving this issue.
> > >>
> > >> Best regards,
> > >> Marco
> > >>
> > >> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
> > >> pedro.larroy.lists@gmail.com> geschrieben:
> > >>
> > >>> I think this is a big problem, which has blocked us before. I want
to
> > >>> point out that you are doing a great thing by avoiding everyone
> > >>> getting blocked by refactoring the pipelines.
> > >>>
> > >>> My concern is that we are kicking the can down the road and not
> > >>> addressing the root cause of the problem with is known
> > >>> https://issues.jenkins-ci.org/browse/JENKINS-37984
> > >>>
> > >>> Pedro.
> > >>>
> > >>>
> > >>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> > >>> <marco.g.abreu@googlemail.com.invalid> wrote:
> > >>> >
> > >>> > Hello Steffen,
> > >>> >
> > >>> > no, there won't be any impact on the PR process or nightly
> > regressions.
> > >>> > Only the reporting will have to be updated with the new job links,
> > but
> > >>> that
> > >>> > should be a minor issue. To avoid any outage, I have been thinking
> > >>> about
> > >>> > running both versions in parallel.
> > >>> >
> > >>> > Best regards,
> > >>> > Marco
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <
> > >>> steffenrochel@gmail.com>
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Marco - is there any impact on reporting, the PR process
or
> > >>> nightly
> > >>> > > regression beside reduction in TAT?  If yes, please elaborate.
> > >>> > > Steffen
> > >>> > >
> > >>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > >>> > > <marco.g.abreu@googlemail.com.invalid> wrote:
> > >>> > >
> > >>> > > > Hello,
> > >>> > > >
> > >>> > > > we ran into issues around the maximum filesize of the
> Jenkinsfile
> > >>> a few
> > >>> > > > times already. In order to resolve this issue, I'd like
to
> > combine
> > >>> this
> > >>> > > > with some refactors I have planned for quite some time.
> > >>> > > >
> > >>> > > > The idea is basically to move away from one big Jenkinsfile
and
> > >>> instead
> > >>> > > > split it into separate jobs that run in parallel and
report
> their
> > >>> status
> > >>> > > > individually. Besides avoiding the size restriction,
this will
> > >>> greatly
> > >>> > > > speed up the PR validation process by reducing the critical
> path.
> > >>> Instead
> > >>> > > > of having to wait for every single step within a stage
to
> finish
> > >>> before
> > >>> > > the
> > >>> > > > next stage (e.g. tests) is getting executed, these pipelines
> > would
> > >>> now be
> > >>> > > > able to move forward individually. I'm still in the
process of
> > >>> > > refactoring
> > >>> > > > and can't provide any numbers or documentation at this
time,
> but
> > I
> > >>> would
> > >>> > > > like to announce this early on to avoid conflicts:
> > >>> > > >
> > >>> > > > Since I will remove the original Jenkinsfile, this might
cause
> > >>> conflicts
> > >>> > > > with ongoing efforts that try to change the Jenkinsfile.
This
> > >>> poses the
> > >>> > > > risk that I might forget to port a change. Thus, I'd
like to
> ask
> > >>> all
> > >>> > > > contributors to wait with changes of Jenkinsfile and
would like
> > to
> > >>> > > request
> > >>> > > > fellow-committers to wait with merging any Jenkinsfile-related
> > PRs
> > >>> until
> > >>> > > > further notice.
> > >>> > > >
> > >>> > > > I expect to finish this refactor until the end of the
week.
> > Please
> > >>> don't
> > >>> > > > hesitate to ask if you've got further questions.
> > >>> > > >
> > >>> > > > Please excuse any caused inconveniences.
> > >>> > > >
> > >>> > > > Best regards,
> > >>> > > > Marco
> > >>> > > >
> > >>> > >
> > >>>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message