mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@googlemail.com.INVALID>
Subject Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Date Tue, 20 Nov 2018 20:09:58 GMT
Thanks, Pedro!

I have also been looking into that issue, but it seems like this would
require changes in the groovy interpreter of Jenkins. From what I can tell,
a refactor will give us multiple benefits (clarity and speed) aside from
resolving this issue.

Best regards,
Marco

Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <pedro.larroy.lists@gmail.com>
geschrieben:

> I think this is a big problem, which has blocked us before. I want to
> point out that you are doing a great thing by avoiding everyone
> getting blocked by refactoring the pipelines.
>
> My concern is that we are kicking the can down the road and not
> addressing the root cause of the problem with is known
> https://issues.jenkins-ci.org/browse/JENKINS-37984
>
> Pedro.
>
>
> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> <marco.g.abreu@googlemail.com.invalid> wrote:
> >
> > Hello Steffen,
> >
> > no, there won't be any impact on the PR process or nightly regressions.
> > Only the reporting will have to be updated with the new job links, but
> that
> > should be a minor issue. To avoid any outage, I have been thinking about
> > running both versions in parallel.
> >
> > Best regards,
> > Marco
> >
> >
> >
> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <steffenrochel@gmail.com>
> > wrote:
> >
> > > Hi Marco - is there any impact on reporting, the PR process or nightly
> > > regression beside reduction in TAT?  If yes, please elaborate.
> > > Steffen
> > >
> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > > <marco.g.abreu@googlemail.com.invalid> wrote:
> > >
> > > > Hello,
> > > >
> > > > we ran into issues around the maximum filesize of the Jenkinsfile a
> few
> > > > times already. In order to resolve this issue, I'd like to combine
> this
> > > > with some refactors I have planned for quite some time.
> > > >
> > > > The idea is basically to move away from one big Jenkinsfile and
> instead
> > > > split it into separate jobs that run in parallel and report their
> status
> > > > individually. Besides avoiding the size restriction, this will
> greatly
> > > > speed up the PR validation process by reducing the critical path.
> Instead
> > > > of having to wait for every single step within a stage to finish
> before
> > > the
> > > > next stage (e.g. tests) is getting executed, these pipelines would
> now be
> > > > able to move forward individually. I'm still in the process of
> > > refactoring
> > > > and can't provide any numbers or documentation at this time, but I
> would
> > > > like to announce this early on to avoid conflicts:
> > > >
> > > > Since I will remove the original Jenkinsfile, this might cause
> conflicts
> > > > with ongoing efforts that try to change the Jenkinsfile. This poses
> the
> > > > risk that I might forget to port a change. Thus, I'd like to ask all
> > > > contributors to wait with changes of Jenkinsfile and would like to
> > > request
> > > > fellow-committers to wait with merging any Jenkinsfile-related PRs
> until
> > > > further notice.
> > > >
> > > > I expect to finish this refactor until the end of the week. Please
> don't
> > > > hesitate to ask if you've got further questions.
> > > >
> > > > Please excuse any caused inconveniences.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message