mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@gmail.com>
Subject Re: [CI] Proposal for automatic PR labeling
Date Wed, 14 Oct 2020 23:01:50 GMT
Thanks chai!

Shall we remove the awaiting testing state? To me it doesn't really provide
any value and will just produce constant state switches and clutter in the
PR. I would say that the state should not change while CI is still running
until it reached a conclusion - whether that's one failed or all passed.

-Marco


On Thu, Oct 15, 2020, 00:15 Chaitanya Bapat <chai.bapat@gmail.com> wrote:

> Update: PR Status Label Bot is functional.
>
> You'll see your PRs getting labels such as
> - pr-awaiting-testing [When CI is kick-started for every commit]
> - pr-work-in-progress [When CI has failed or PR is marked draft or WIP in
> title]
> - pr-pending-review [When CI succeeded & no review]
> - pr-pending-merge [When CI succeeds & 2 Apache MXNet Committers approve]
> - pr-awaiting-response [When CI succeeded but reviewer request changes]
>
> PR: https://github.com/apache/incubator-mxnet-ci/pull/27
> Followup PR to enable add/remove label :
> https://github.com/apache/incubator-mxnet-ci/pull/33
>
>
> On Sun, 2 Aug 2020 at 13:56, Chaitanya Bapat <chai.bapat@gmail.com> wrote:
>
> > Yes. It has been disabled.
> >
> > On Sun, 2 Aug 2020 at 13:14, Marco de Abreu <marco.g.abreu@gmail.com>
> > wrote:
> >
> >> Great. As I said, please make sure that the bot is disabled until it has
> >> been reviewed and approved.
> >>
> >> -Marco
> >>
> >> On Sun, Aug 2, 2020, 10:12 PM Chaitanya Bapat <chai.bapat@gmail.com>
> >> wrote:
> >>
> >> > Thanks. webhook is setup!
> >> >
> >> > On Sun, 2 Aug 2020 at 02:43, Marco de Abreu <marco.g.abreu@gmail.com>
> >> > wrote:
> >> >
> >> > > Ticket created: https://issues.apache.org/jira/browse/INFRA-20624
> >> > >
> >> > > -Marco
> >> > >
> >> > > On Sun, Aug 2, 2020, 7:43 AM Chaitanya Bapat <chai.bapat@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Right.
> >> > > > I've sent you the email on your personal email address.
> >> > > > Thanks for helping setup the github webhook with Apache Infra.
> >> > > >
> >> > > > Cheers
> >> > > > Chai
> >> > > >
> >> > > > On Sat, 1 Aug 2020 at 03:25, Marco de Abreu <
> >> marco.g.abreu@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Maybe we should just remove the pr-awaiting-testing state
> >> entirely.
> >> > > > Either
> >> > > > > it's WIP or CI passed.
> >> > > > >
> >> > > > > -Marco
> >> > > > >
> >> > > > > On Sat, Aug 1, 2020 at 12:21 PM Marco de Abreu <
> >> > > marco.g.abreu@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > I generally like the idea, thanks for improving the
user
> >> experience
> >> > > for
> >> > > > > > the project, Chai! One thing that would be important
to me is
> >> > > avoiding
> >> > > > a
> >> > > > > > constant shifting between the states. The PR shouldn't
switch
> to
> >> > > > > > "awaiting-testing" every single time CI is retriggered
but
> >> rather
> >> > > have
> >> > > > > some
> >> > > > > > consistency. But that's something we can also tweak
as we go.
> >> > > > > >
> >> > > > > > Before the bot is run, a small heads up would be nice.
Just
> >> > running a
> >> > > > > > batch job on the repository - especially using others'
> >> credentials
> >> > -
> >> > > > > > without previous approval is not that nice. Hence,
please run
> >> your
> >> > > > tests
> >> > > > > a
> >> > > > > > non-intrusive way until the bot has been approved for
> >> deployment.
> >> > > > > >
> >> > > > > > With regards to the ticket, I'm happy to be of assistance.
> Just
> >> > send
> >> > > > the
> >> > > > > > secret to my personal email.
> >> > > > > >
> >> > > > > > Best regards
> >> > > > > > Marco
> >> > > > > >
> >> > > > > > On Fri, Jul 31, 2020 at 9:12 PM Sheng Zha <
> zhasheng@apache.org>
> >> > > wrote:
> >> > > > > >
> >> > > > > >> +1, thanks for driving this. Here's the pseudo
code for the
> PR
> >> > > > labeling
> >> > > > > >> for better readability:
> >> > > > > >>
> >> > > > > >> if WIP in title or PR is draft or CI failed:
> >> > > > > >>     pr-work-in-progress
> >> > > > > >> elif CI has not started yet or CI is in progress:
> >> > > > > >>     pr-awaiting-testing
> >> > > > > >> else: # CI passed checks
> >> > > > > >>     if pr has at least one approval and no request
changes:
> >> > > > > >>         pr-awaiting-merge
> >> > > > > >>     elif pr has no review or all reviews have been
> >> > > > > dismissed/re-requested:
> >> > > > > >>         pr-awaiting-review
> >> > > > > >>     else: # pr has a review that hasn't been dismissed
yet no
> >> > > approval
> >> > > > > >>         pr-awaiting-response
> >> > > > > >>
> >> > > > > >> Best,
> >> > > > > >> Sheng
> >> > > > > >>
> >> > > > > >> On 2020/07/31 18:08:34, Chaitanya Bapat <
> chai.bapat@gmail.com>
> >> > > wrote:
> >> > > > > >> > Hello MXNet community,
> >> > > > > >> >
> >> > > > > >> > Banking on the success of mxnet-bot that has
been
> triggering
> >> CI
> >> > > > > >> > successfully for the past 3 months, Sheng
proposed an idea
> >> for
> >> > > > > >> automatic PR
> >> > > > > >> > labeling. I am working on the implementation
of the same.
> >> > > > > >> >
> >> > > > > >> > *What's the idea?*
> >> > > > > >> > An automated way of labeling PRs based on
the status of CI.
> >> > > > > >> >
> >> > > > > >> > *Why?*
> >> > > > > >> >
> >> > > > > >> > This serves dual purpose
> >> > > > > >> >
> >> > > > > >> >    - Automates the labeling [currently manual]
> >> > > > > >> >    - Allows contributors to prioritize review
of PRs.
> >> > > > > >> >
> >> > > > > >> > *How?*
> >> > > > > >> > 1. Github webhook would send status update
events to the
> >> Lambda
> >> > > > > function
> >> > > > > >> > 2. Lambda would then label the CI based on
following
> >> conditions
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > *if WIP in title or PR is draft or CI failed:
> >> > > > > >> pr-work-in-progresselif CI
> >> > > > > >> > has not started yet or CI is in progress:
> >> > > > pr-awaiting-testingelse:
> >> > > > > #
> >> > > > > >> CI
> >> > > > > >> > passed checks    if pr has at least one approval
and no
> >> request
> >> > > > > changes:
> >> > > > > >> >     pr-awaiting-merge    elif pr has no review
or all
> reviews
> >> > have
> >> > > > > been
> >> > > > > >> > dismissed/re-requested:        pr-awaiting-review
   else:
> #
> >> pr
> >> > > has
> >> > > > a
> >> > > > > >> > review that hasn't been dismissed yet no approval
> >> > > > > >> > pr-awaiting-response*
> >> > > > > >> >
> >> > > > > >> > To that end, a POC was done to test if it's
possible &
> >> result:
> >> > it
> >> > > > > works.
> >> > > > > >> > Results of 1 successful run on Prod for
> >> apache/incubator-mxnet
> >> > > repo
> >> > > > > >> >
> >> > > > > >> > Open PRs : 207
> >> > > > > >> > PRs labeled : 18
> >> > > > > >> > PRs already labeled : 38
> >> > > > > >> > PRs with unknown jobs : 2
> >> > > > > >> > PRs with status failed : 149
> >> > > > > >> >
> >> > > > > >> > Lambda function labeled 18 PRs that had passed
all the
> tests.
> >> > > > > >> >
> >> > > > > >> > PR for POC:
> >> > https://github.com/apache/incubator-mxnet-ci/pull/27
> >> > > > > >> > However, Marco made a valid point regarding
leveraging
> Github
> >> > > > Webhooks
> >> > > > > >> > [instead of Jenkins] for retrieving status
of the PR.
> >> > > > > >> >
> >> > > > > >> > *What's required?*
> >> > > > > >> > Thus, to get this feature functional, I'd
need a separate
> >> Github
> >> > > > > webhook
> >> > > > > >> > that sends information related to PR Status.
Just like the
> >> > webhook
> >> > > > > >> request
> >> > > > > >> > for CI Bot, I can send the details over private
email to
> >> Marco's
> >> > > > > >> personal
> >> > > > > >> > email address & then he can create a ticket
with Apache
> >> Infra.
> >> > > > > >> >
> >> > > > > >> > Thanks to Sheng & Marco for guidance &
assistance on this.
> >> > > > > >> >
> >> > > > > >> > Thanks,
> >> > > > > >> > Chai
> >> > > > > >> >
> >> > > > > >> > --
> >> > > > > >> > *Chaitanya Prakash Bapat*
> >> > > > > >> > *+1 (973) 953-6299*
> >> > > > > >> >
> >> > > > > >> > [image: https://www.linkedin.com//in/chaibapat25]
> >> > > > > >> > <https://github.com/ChaiBapchya>[image:
> >> > > > > >> https://www.facebook.com/chaibapat]
> >> > > > > >> > <https://www.facebook.com/chaibapchya>[image:
> >> > > > > >> > https://twitter.com/ChaiBapchya] <
> >> > https://twitter.com/ChaiBapchya
> >> > > > > >> >[image:
> >> > > > > >> > https://www.linkedin.com//in/chaibapat25]
> >> > > > > >> > <https://www.linkedin.com//in/chaibapchya/>
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > *Chaitanya Prakash Bapat*
> >> > > > *+1 (973) 953-6299*
> >> > > >
> >> > > > [image: https://www.linkedin.com//in/chaibapat25]
> >> > > > <https://github.com/ChaiBapchya>[image:
> >> > > https://www.facebook.com/chaibapat
> >> > > > ]
> >> > > > <https://www.facebook.com/chaibapchya>[image:
> >> > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >> > > >[image:
> >> > > > https://www.linkedin.com//in/chaibapat25]
> >> > > > <https://www.linkedin.com//in/chaibapchya/>
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > *Chaitanya Prakash Bapat*
> >> > *+1 (973) 953-6299*
> >> >
> >> > [image: https://www.linkedin.com//in/chaibapat25]
> >> > <https://github.com/ChaiBapchya>[image:
> >> https://www.facebook.com/chaibapat
> >> > ]
> >> > <https://www.facebook.com/chaibapchya>[image:
> >> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >> >[image:
> >> > https://www.linkedin.com//in/chaibapat25]
> >> > <https://www.linkedin.com//in/chaibapchya/>
> >> >
> >>
> >
> >
> > --
> > *Chaitanya Prakash Bapat*
> > *+1 (973) 953-6299*
> >
> > [image: https://www.linkedin.com//in/chaibapat25]
> > <https://github.com/ChaiBapchya>[image:
> > https://www.facebook.com/chaibapat] <
> https://www.facebook.com/chaibapchya>[image:
> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >[image:
> > https://www.linkedin.com//in/chaibapat25]
> > <https://www.linkedin.com//in/chaibapchya/>
> >
>
>
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
>
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat
> ]
> <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message