mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@gmail.com>
Subject Re: MXNet Bot Demo
Date Mon, 23 Mar 2020 19:55:20 GMT
Sure, already done.

-Marco

On Mon, Mar 23, 2020 at 8:53 PM Chaitanya Bapat <chai.bapat@gmail.com>
wrote:

> Hello,
> Update: Apache Infra Ticket for MXNet Bot
> Thanks once again, Marco for opening the ticket. But turns out, Apache
> Infra folks closed it stating: "Security concerns around allowing unknown
> person to submit PR and run our hardware". Furthermore, it goes onto state
> that bot circumvents the dependence on Jenkins Admins which is like solving
> a problem that doesn't exist.
>
> I sense there is some confusion in the communication (maybe on my part). It
> turns out the security concerns aren't actually correct.
>
> 1. Unknown person can submit a PR (before & after bot proposal), and run
> our hardware (pre as well as post bot).
> 2. Code should be reviewed by somebody with an ICLA on file. This doesn't
> change either. Prior to merging a PR, code has to be approved by a
> committer just like before.
> Overall it looks like the job of the bot isn't clear to folks in Apache
> Infra. Bot simply is a means for triggering CI (which could be done
> manually by Log In to Jenkins -> PR -> Job -> Build) and doesn't quite
> tweak with merging procedure. Yes, only addition is now unknown person (PR
> Author) can trigger CI with a message (but that was possible anyway by
> pushing a commit. Bot just prevents users from pushing empty commits and
> building entire suite).
>
> As can be seen from last 10 open PRs as of Monday 23rd March, 12pm PT most
> PRs fail on 1/2 jobs. In such a scenario, the proposed MXNet bot would come
> in handy for just invoking CI on that specific build (instead of a
> non-committer PR Author to push empty commit : hurting on the resource,
> time & cost considerations apart from undesirable dev experience)
>
> @Marco Since I am a non-committer, I guess these 2 clarifications need to
> be conveyed to the Apache Infra by someone with Committer access.
>
> What do you think?
>
> Thanks,
> Chai
>
> On Sat, 21 Mar 2020 at 16:08, Marco de Abreu <marco.g.abreu@gmail.com>
> wrote:
>
> > Hello,
> >
> > the ticket has been created:
> > https://issues.apache.org/jira/browse/INFRA-20005
> >
> > Best regards,
> > Marco
> >
> > On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu <marco.g.abreu@gmail.com
> >
> > wrote:
> >
> > > Sounds like a good plan!
> > >
> > > Please send me the URL (please make sure it's backed by DNS and not
> just
> > > the gateway URL) of the webhook handler, GitHub events you're
> interested
> > in
> > > and the shared secret in a private email to my personal email address.
> I
> > > will then create the ticket with Apache infra.
> > >
> > > -Marco
> > >
> > > Chaitanya Bapat <chai.bapat@gmail.com> schrieb am Do., 19. März 2020,
> > > 23:07:
> > >
> > >> @Marco Alright, it makes total sense to test out the Bot feature
> > alongside
> > >> auto-trigger as a transition.
> > >>
> > >> Path Forward:
> > >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub WebHook and
> > >> Infra)
> > >> 2. We don't turn off automatic trigger of PR builds for now.
> > >> 3. Hopefully, bot is used by developers to trigger specific jobs
> > >> 4. Later on (say around April 20), let's discuss the possibility of
> > >> switching off auto-trigger (with appropriate data) if it makes sense.
> > >> Thanks Marco for volunteering to help enable the web hook on
> > >> apache/incubator-mxnet. Let me know if we can sync up on Slack channel
> > to
> > >> get the ball rolling.
> > >>
> > >> Thanks once again for the entire community to step in and help try out
> > >> this
> > >> Bot.
> > >> Chai
> > >>
> > >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu <marco.g.abreu@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi, that's correct. But as stated previously, it's not an option to
> > >> remove
> > >> > the hook. For now, I'd like to see how the system behaves while it's
> > >> > optional. Later on, we can talk about revisiting this decision. But
> to
> > >> me
> > >> > it's not an option to deploy an entirely new system and approach
> > without
> > >> > having a transition or even a timeframe in which we are able to fall
> > >> back.
> > >> >
> > >> > I'm happy to support the deployment of the bot and add an additional
> > >> > webhook to enable it's functionality to support selective triggering
> > by
> > >> PR
> > >> > authors and committers, but I will not support the disabling of
> > >> automatic
> > >> > triggering of branches or PRs.
> > >> >
> > >> > -Marco
> > >> >
> > >> > Chaitanya Bapat <chai.bapat@gmail.com> schrieb am Mi., 18. März
> 2020,
> > >> > 21:00:
> > >> >
> > >> > > Hey Marco,
> > >> > >
> > >> > > I thought currently every commit on PR and master triggers CI
> > >> > > because
> > >> > > a. github webhook points to Jenkins Server
> > >> > > b. GH Webhook events trigger builds on Jenkins for all commits to
> > any
> > >> > > branch in apache/incubator-mxnet
> > >> > > may it be master/PR/non-PR
> > >> > > Reason:
> > >> > > Because all the 3 types of branches are discovered by Jenkins
> > (non-PR
> > >> > > (including master) and PR)
> > >> > >
> > >> > > Proposal: Remove GitHub WebHook to Jenkins and replace with GH
> > >> Webhook to
> > >> > > Lambda
> > >> > > But after I remove the github webhook that points to Jenkins :
> *N**o
> > >> > commit
> > >> > > will trigger Jenkins build by default* (as Jenkins wont receive GH
> > >> > events)
> > >> > > Only those that Bot deems fit will be triggered (using Jenkins API
> > >> > invoked
> > >> > > by Lambda).
> > >> > > Hence its needed to handle that case of master merge.
> > >> > > Am I understanding this correctly?
> > >> > >
> > >> > > On Wed, 18 Mar 2020 at 04:23, Marco de Abreu <
> > marco.g.abreu@gmail.com
> > >> >
> > >> > > wrote:
> > >> > >
> > >> > > > Thanks Chai, sounds good to me. Could you elaborate a bit on the
> > >> point
> > >> > > > about triggering a CI run after the PR has been merged? We
> already
> > >> to
> > >> > > that
> > >> > > > automatically for the master, so what's the benefit to do it
> > twice?
> > >> > > >
> > >> > > > -Marco
> > >> > > >
> > >> > > > Chaitanya Bapat <chai.bapat@gmail.com> schrieb am Mi., 18. März
> > >> 2020,
> > >> > > > 09:30:
> > >> > > >
> > >> > > > > Update:
> > >> > > > >
> > >> > > > > >  we can ensure that all CI runs ran on the commit that will
> be
> > >> > merged
> > >> > > > > @Sam Skalicky <samskalicky@gmail.com> Branch Protection is
> > added
> > >> to
> > >> > > > public
> > >> > > > > MXNet repo. It ensures that for every PR to be merged, the CI
> > >> passes.
> > >> > > All
> > >> > > > > the jobs selected "required" jobs will have to be green for
> the
> > >> PR to
> > >> > > be
> > >> > > > > merged. Ofcourse, users with "Adminstrator" access can merge
> > >> without
> > >> > it
> > >> > > > but
> > >> > > > > that's just a backdoor. It is the case now and will continue
> to
> > be
> > >> > the
> > >> > > > case
> > >> > > > > with the inclusion of Bot.
> > >> > > > >
> > >> > > > > > easily verify that the CI has executed all runs on the
> commit
> > >> that
> > >> > > will
> > >> > > > > be merged
> > >> > > > > GitHub UI shows all the jobs and the status corresponding to
> it
> > on
> > >> > > every
> > >> > > > > commit. That should suffice. For the merged commits, Repo ->
> > >> Commits
> > >> > ->
> > >> > > > > Commit ID (Status) can be tracked currently (only way that I
> > know
> > >> > of).
> > >> > > > > Moreover, it is beyond the scope of this project (and possibly
> > >> out of
> > >> > > our
> > >> > > > > control since this is purely GitHub UI specific use-case).
> > >> > > > >
> > >> > > > > Thanks @przemyslaw for supporting the opt-in.
> > >> > > > >
> > >> > > > > Thanks everyone in the community for sharing concerns, voicing
> > >> your
> > >> > > > opinion
> > >> > > > > and participating in the discussion.
> > >> > > > > Thanks to those who attended the demo last Friday.
> > >> > > > >
> > >> > > > > Action items from that discussion
> > >> > > > > 1. Handle master merge builds [Done]
> > >> > > > > Bot runs entire CI suite after the PR is merged and comments
> on
> > >> the
> > >> > PR
> > >> > > > > about the same.
> > >> > > > > Design decision :
> > >> > > > > MXNet Bot comment about master merge build on the *merge
> commit
> > vs
> > >> > PR*.
> > >> > > > > After the PR is merged, Bot runs entire CI and comments the
> > >> result of
> > >> > > CI
> > >> > > > > trigger on the PR (because it is easy to track on a PR rather
> > than
> > >> > > > > commenting inside the merge commit)
> > >> > > > >
> > >> > > > > 2. Idempotent condition
> > >> > > > > In case of already running build, if an attempt is made to
> > >> retrigger
> > >> > > the
> > >> > > > > job then what should be the response
> > >> > > > > a. Not to re-trigger, let the ongoing build continue till
> > >> completion
> > >> > > > > b. End the ongoing build and re-trigger
> > >> > > > > c. Let the ongoing build continue, re-trigger new build
> > >> > > > >
> > >> > > > > From resource saving point of view, *c* looks costly and a can
> > be
> > >> > > > > avoided/optimized by B.
> > >> > > > > In case when a re-trigger was started "erroneously" then
> killing
> > >> > > ongoing
> > >> > > > > build and re-trigger is a waste.
> > >> > > > > In case when ongoing build failed in one sub-part, then
> > >> re-triggering
> > >> > > is
> > >> > > > > justified.
> > >> > > > > Erroneous re-triggers would be less often while conscious
> > >> re-triggers
> > >> > > to
> > >> > > > > suppress failure is more common use-case. It looks like a safe
> > >> > > assumption
> > >> > > > > to make given the trade-off.
> > >> > > > > [Open to debate]
> > >> > > > >
> > >> > > > > 3. Add security consideration [Use of secret manager, but
> > without
> > >> > > > > auto-rotation due to Jenkins manual config requirement] [Done]
> > >> > > > > 4. New PR Instruction message by the Bot [Done]
> > >> > > > > Thanks to the suggestion of Leonard, supported by others. I've
> > now
> > >> > > added
> > >> > > > > the feature where the Bot comments a help message. [For
> > reference
> > >> -
> > >> > > > > https://github.com/ChaiBapchya/incubator-mxnet/pull/52]
> > >> > > > >
> > >> > > > > Barring the opt-in vs opt-out debate & idempotency, consensus
> > was
> > >> > > quickly
> > >> > > > > reached for the rest.
> > >> > > > >
> > >> > > > > In the coming days, I hope to roll-out this feature into Prod
> > >> (public
> > >> > > > > MXNet) for all devs to use.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Chai
> > >> > > > >
> > >> > > > > On Mon, 16 Mar 2020 at 11:57, Marco de Abreu <
> > >> > marco.g.abreu@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Well that's generally a problem with a deferred CI approach
> > (CI
> > >> is
> > >> > > run
> > >> > > > at
> > >> > > > > > commit and not at merge time). This can either be solved
> > through
> > >> > the
> > >> > > > > other
> > >> > > > > > proposal that's currently on dev@, by having a bot which
> does
> > >> > merges
> > >> > > > by
> > >> > > > > > having a global lock and a merge queue or by accepting the
> > >> issue.
> > >> > > > Reality
> > >> > > > > > right now is that we're running that model where two PRs
> which
> > >> are
> > >> > > > merged
> > >> > > > > > in parallel might break one another. One thing to consider
> > >> though
> > >> > is
> > >> > > > that
> > >> > > > > > this breakage would have to be introduced in two separate
> > parts
> > >> > since
> > >> > > > > > otherwise there'd be merge conflicts. I think we had that
> > >> situation
> > >> > > > twice
> > >> > > > > > so far and the result was a quick revert, so I'd say that
> > it's a
> > >> > > > problem
> > >> > > > > > that can happily be accepted. All other solutions basically
> > >> require
> > >> > > > some
> > >> > > > > > form of single-threaded and globally locked solution which
> > >> limits
> > >> > us
> > >> > > in
> > >> > > > > > scalability. I'd recommend to just accept that risk and
> revert
> > >> a PR
> > >> > > in
> > >> > > > > case
> > >> > > > > > it actually had a conflict.
> > >> > > > > >
> > >> > > > > > -Marco
> > >> > > > > >
> > >> > > > > > On Mon, Mar 16, 2020 at 6:29 PM Skalicky, Sam
> > >> > > > <sskalic@amazon.com.invalid
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > We probably need some way to track which CI runs ran for
> > which
> > >> > > commit
> > >> > > > > > too,
> > >> > > > > > > that way we can ensure that all CI runs ran on the commit
> > that
> > >> > will
> > >> > > > be
> > >> > > > > > > merged.  Maybe the bot can comment with the commit hash
> when
> > >> > users
> > >> > > > > > command
> > >> > > > > > > it to do something. Although since users can trigger
> > >> individual
> > >> > CI
> > >> > > > runs
> > >> > > > > > its
> > >> > > > > > > possible to have some commits run some CI runs but not
> > >> others. We
> > >> > > > need
> > >> > > > > > some
> > >> > > > > > > way to easily verify that the CI has executed all runs on
> > the
> > >> > > commit
> > >> > > > > that
> > >> > > > > > > will be merged.
> > >> > > > > > >
> > >> > > > > > > Sam
> > >> > > > > > >
> > >> > > > > > > > On Mar 13, 2020, at 8:28 PM, Przemysław Trędak <
> > >> > > ptrendx@apache.org
> > >> > > > >
> > >> > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > CAUTION: This email originated from outside of the
> > >> > organization.
> > >> > > Do
> > >> > > > > not
> > >> > > > > > > click links or open attachments unless you can confirm the
> > >> sender
> > >> > > and
> > >> > > > > > know
> > >> > > > > > > the content is safe.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > I personally like the idea of opt-in more than opt-out:
> > >> > > > > > > > - ultimately PR author wants the PR to be merged so they
> > (or
> > >> > > > > committer
> > >> > > > > > > reviewing the PR) will trigger the CI
> > >> > > > > > > > - if it is easy to trigger the PR via the bot command
> then
> > >> the
> > >> > > > amount
> > >> > > > > > of
> > >> > > > > > > work per PR should be less than with opt-out (since most
> of
> > >> the
> > >> > > > commits
> > >> > > > > > > should then be marked as [skip ci] or something similar
> > >> > > > > > > >
> > >> > > > > > > > +1 to the bot making a comment on each new PR with its
> > >> commands
> > >> > > > (and
> > >> > > > > > > also explaining, or at least giving links to the general
> PR
> > >> > process
> > >> > > > so
> > >> > > > > > new
> > >> > > > > > > PR authors are not lost). Maybe we could make the bot
> > >> recognize
> > >> > if
> > >> > > > the
> > >> > > > > PR
> > >> > > > > > > author is new or existing contributor and offer advice
> based
> > >> on
> > >> > > that?
> > >> > > > > > > >
> > >> > > > > > > > Thanks
> > >> > > > > > > > Przemek
> > >> > > > > > > >
> > >> > > > > > > > On 2020/03/13 22:06:58, Marco de Abreu <
> > >> > marco.g.abreu@gmail.com>
> > >> > > > > > wrote:
> > >> > > > > > > >> Hi,
> > >> > > > > > > >>
> > >> > > > > > > >> since it's no longer necessary to push a new commit to
> > >> trigger
> > >> > > CI,
> > >> > > > > it
> > >> > > > > > > will
> > >> > > > > > > >> already reduce the costs. But to me, requiring an
> action
> > to
> > >> > > enable
> > >> > > > > CI
> > >> > > > > > > after
> > >> > > > > > > >> a PR has been created initially, is a no go. User can
> opt
> > >> out
> > >> > of
> > >> > > > CI,
> > >> > > > > > but
> > >> > > > > > > >> the default has to be CI being triggered automatically
> > for
> > >> > every
> > >> > > > > > commit
> > >> > > > > > > >> unless specifically disabled by a participant. I'm also
> > >> fine
> > >> > > with
> > >> > > > > > > >> triggering certain additional jobs (think about
> running a
> > >> > > nightly
> > >> > > > > job
> > >> > > > > > > upon
> > >> > > > > > > >> request for a PR) to require a manual step, but the PR
> > >> > > validation
> > >> > > > > > > pipelines
> > >> > > > > > > >> have to run automatically. Every check that is marked
> as
> > >> > > > "Required"
> > >> > > > > in
> > >> > > > > > > >> GitHub has to be automatically kicked off.
> > >> > > > > > > >>
> > >> > > > > > > >> -Marco
> > >> > > > > > > >>
> > >> > > > > > > >> On Fri, Mar 13, 2020 at 9:50 PM Chaitanya Bapat <
> > >> > > > > chai.bapat@gmail.com
> > >> > > > > > >
> > >> > > > > > > >> wrote:
> > >> > > > > > > >>
> > >> > > > > > > >>> Firstly,
> > >> > > > > > > >>> Sorry I missed out on attaching the mail thread that
> was
> > >> sent
> > >> > > on
> > >> > > > > 12th
> > >> > > > > > > >>> February for notifying the community of the upcoming
> > >> changes
> > >> > to
> > >> > > > the
> > >> > > > > > > MXNet
> > >> > > > > > > >>> CI
> > >> > > > > > > >>> For reference :
> > >> > > > > > > >>>
> > >> > > > > > > >>>
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://lists.apache.org/thread.html/r09a6ab2803a996fc80e00fe39ed312fa4865e8805e08df847f1addad%40%3Cdev.mxnet.apache.org%3E
> > >> > > > > > > >>>
> > >> > > > > > > >>> Now to the questions,
> > >> > > > > > > >>>> Is it possible for re-triggering a single job to be
> > >> abused?
> > >> > > > > > > >>> @Tao In the case when a user re-triggers a single job
> > >> > multiple
> > >> > > > > times,
> > >> > > > > > > that
> > >> > > > > > > >>> will be visible in the PR conversation thread. A
> > >> committer,
> > >> > > even
> > >> > > > > > after
> > >> > > > > > > he
> > >> > > > > > > >>> has approved the PR before, generally takes a look at
> > the
> > >> > final
> > >> > > > > state
> > >> > > > > > > of
> > >> > > > > > > >>> the PR before merging. Would it be fair to assume the
> > >> > committer
> > >> > > > > could
> > >> > > > > > > take
> > >> > > > > > > >>> the multiple re-trigger of a single job into account
> > >> before
> > >> > > > > merging?
> > >> > > > > > > The
> > >> > > > > > > >>> committer then has the option to invoke `@mxnet-bot
> run
> > ci
> > >> > > [all]
> > >> > > > `
> > >> > > > > to
> > >> > > > > > > >>> trigger the entire build pipeline one last to counter
> > the
> > >> > > abuse.
> > >> > > > > This
> > >> > > > > > > is
> > >> > > > > > > >>> aligned with what @Leonard said.
> > >> > > > > > > >>>
> > >> > > > > > > >>> @Sandeep Thanks a lot for collecting and sharing
> > valuable
> > >> > data.
> > >> > > > I'd
> > >> > > > > > > concur
> > >> > > > > > > >>> with the opinion that given the existing things
> > >> committers &
> > >> > PR
> > >> > > > > > Authors
> > >> > > > > > > >>> take care of, invoking CI shouldn't be that big of an
> > >> > > additional
> > >> > > > > > > burden.
> > >> > > > > > > >>>
> > >> > > > > > > >>> @Marco With the opt-out, the onus remains on the PR
> > >> Author.
> > >> > It
> > >> > > > > > doesn't
> > >> > > > > > > help
> > >> > > > > > > >>> reduce the resource usage. Hence, it was suggested to
> > >> switch
> > >> > to
> > >> > > > > > > >>> opt-in. @Leo's suggestion for proactive commenting on
> > the
> > >> > part
> > >> > > of
> > >> > > > > bot
> > >> > > > > > > makes
> > >> > > > > > > >>> sense and is doable.
> > >> > > > > > > >>>
> > >> > > > > > > >>> Default : opt-out and User initiated opt-in (with
> > >> addressing
> > >> > > > Leo's
> > >> > > > > > fix
> > >> > > > > > > for
> > >> > > > > > > >>> the usability issue you correctly pointed out )
> > >> > > > > > > >>> @Marco How does this sound to you?
> > >> > > > > > > >>>
> > >> > > > > > > >>> Again, thank you all for chiming in and voicing your
> > >> opinion.
> > >> > > > > > > Appreciate
> > >> > > > > > > >>> it.
> > >> > > > > > > >>> We can take ahead these discussions in today's demo
> > >> meeting.
> > >> > > > > [Design
> > >> > > > > > > Doc
> > >> > > > > > > >>> <
> > >> > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot
> > >> > > > >]
> > >> > > > > > > [Demo
> > >> > > > > > > >>> Video <https://www.youtube.com/watch?v=gfOGwZId8aU>]
> > >> > > > > > > >>>
> > >> > > > > > > >>> Thanks,
> > >> > > > > > > >>> Chai
> > >> > > > > > > >>>
> > >> > > > > > > >>> On Fri, 13 Mar 2020 at 12:34, Marco de Abreu <
> > >> > > > > > marco.g.abreu@gmail.com>
> > >> > > > > > > >>> wrote:
> > >> > > > > > > >>>
> > >> > > > > > > >>>> I'd recommend that the bot makes an initial comment
> > when
> > >> a
> > >> > PR
> > >> > > > gets
> > >> > > > > > > opened
> > >> > > > > > > >>>> and informs the users of its commands. It then tells
> > the
> > >> > user
> > >> > > > the
> > >> > > > > > > commend
> > >> > > > > > > >>>> to opt out of CI.
> > >> > > > > > > >>>>
> > >> > > > > > > >>>> -Marco
> > >> > > > > > > >>>>
> > >> > > > > > > >>>> Lausen, Leonard <lausen@amazon.com.invalid> schrieb
> am
> > >> Fr.,
> > >> > > 13.
> > >> > > > > > März
> > >> > > > > > > >>> 2020,
> > >> > > > > > > >>>> 20:27:
> > >> > > > > > > >>>>
> > >> > > > > > > >>>>> On opt-out: People may be unaware of opt-out would
> not
> > >> use
> > >> > > it.
> > >> > > > > > There
> > >> > > > > > > is
> > >> > > > > > > >>>> no
> > >> > > > > > > >>>>> incentive to use opt-out, as the PR author doesn't
> pay
> > >> any
> > >> > > > money
> > >> > > > > > for
> > >> > > > > > > CI
> > >> > > > > > > >>>>> run.
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>> I agree with Marco though that opt-in alone may
> cause
> > >> > > usability
> > >> > > > > > > issues,
> > >> > > > > > > >>>> as
> > >> > > > > > > >>>>> contributors may not be aware of how to trigger the
> > CI.
> > >> > > > > > > >>>>> One solution is that the bot proactively comments on
> > >> the PR
> > >> > > and
> > >> > > > > > > reminds
> > >> > > > > > > >>>> the
> > >> > > > > > > >>>>> author to trigger running CI once the author deems
> the
> > >> PR
> > >> > > > ready.
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>> But even if we choose opt-out, the bot will still
> add
> > a
> > >> lot
> > >> > > of
> > >> > > > > > value,
> > >> > > > > > > >>> as
> > >> > > > > > > >>>> PR
> > >> > > > > > > >>>>> authors can retrigger single jobs that have failed
> due
> > >> to
> > >> > > > > > flakiness.
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>>> Is it possible for re-triggering a single job to be
> > >> > abused?
> > >> > > > For
> > >> > > > > > > >>>> example,
> > >> > > > > > > >>>>>> the author spends two days re-triggering a flaky
> job
> > to
> > >> > make
> > >> > > > it
> > >> > > > > > > pass.
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>> Yes, this is possible. I suggest the committer who
> > >> likes to
> > >> > > > > merge a
> > >> > > > > > > PR
> > >> > > > > > > >>>>> needs to
> > >> > > > > > > >>>>> make a good judgement here if a PR is abusing the
> > >> feature,
> > >> > > and
> > >> > > > if
> > >> > > > > > so,
> > >> > > > > > > >>>>> retrigger
> > >> > > > > > > >>>>> all CI runs.
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>> Best regards
> > >> > > > > > > >>>>> Leonard
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>> On Fri, 2020-03-13 at 08:07 +0100, Marco de Abreu
> > wrote:
> > >> > > > > > > >>>>>> Thanks for the data Sandeep. In these cases it
> sounds
> > >> like
> > >> > > it
> > >> > > > > > would
> > >> > > > > > > >>>> have
> > >> > > > > > > >>>>>> rather been better when people explicitly turned
> off
> > >> CI in
> > >> > > > that
> > >> > > > > > > case.
> > >> > > > > > > >>>>>> What's the argument against an opt-out instead of
> an
> > >> > opt-in?
> > >> > > > > > > >>>>>>
> > >> > > > > > > >>>>>> My intention is that I consider it quite cumbersome
> > to
> > >> > make
> > >> > > > it a
> > >> > > > > > > >>>>> *required*
> > >> > > > > > > >>>>>> step to always trigger CI manually, even if just
> > >> > submitting
> > >> > > a
> > >> > > > > > small
> > >> > > > > > > >>> PR.
> > >> > > > > > > >>>>> I'd
> > >> > > > > > > >>>>>> rather see people explicitly turning off CI if they
> > >> > wouldn't
> > >> > > > > like
> > >> > > > > > to
> > >> > > > > > > >>>> use
> > >> > > > > > > >>>>> it
> > >> > > > > > > >>>>>> - and there's also the "draft" stage for a PR which
> > >> some
> > >> > > > > > > contributors
> > >> > > > > > > >>>> are
> > >> > > > > > > >>>>>> using.
> > >> > > > > > > >>>>>>
> > >> > > > > > > >>>>>> With regards to WIP and do not review: I think
> these
> > >> are
> > >> > use
> > >> > > > > cases
> > >> > > > > > > >>>> where
> > >> > > > > > > >>>>>> you want CI feedback, as otherwise you wouldn't
> have
> > >> > opened
> > >> > > > the
> > >> > > > > > PR.
> > >> > > > > > > >>> If
> > >> > > > > > > >>>>> you
> > >> > > > > > > >>>>>> don't want human feedback and neither machine
> > feedback,
> > >> > why
> > >> > > > open
> > >> > > > > > the
> > >> > > > > > > >>> PR
> > >> > > > > > > >>>>> at
> > >> > > > > > > >>>>>> all?
> > >> > > > > > > >>>>>>
> > >> > > > > > > >>>>>> -Marco
> > >> > > > > > > >>>>>>
> > >> > > > > > > >>>>>>
> > >> > > > > > > >>>>>> sandeep krishnamurthy <sandeep.krishna98@gmail.com
> >
> > >> > schrieb
> > >> > > > am
> > >> > > > > > Fr.,
> > >> > > > > > > >>>> 13.
> > >> > > > > > > >>>>>> März 2020, 05:24:
> > >> > > > > > > >>>>>>
> > >> > > > > > > >>>>>>> I tried to gather some data for us to discuss this
> > >> topic
> > >> > in
> > >> > > > > this
> > >> > > > > > > >>>>> thread. I
> > >> > > > > > > >>>>>>> tried to count number of un-necessary builds by
> > >> looking
> > >> > at
> > >> > > > most
> > >> > > > > > > >>>> recent
> > >> > > > > > > >>>>> (as
> > >> > > > > > > >>>>>>> of 12, March 9 PM PST) 50 PRs merged to master and
> > 50
> > >> > PRs.
> > >> > > > > > > >>>>>>> Identifying un-necessary builds is bit
> subjective. I
> > >> > tried
> > >> > > to
> > >> > > > > be
> > >> > > > > > > >>> more
> > >> > > > > > > >>>>>>> conservative where I didn't count a build as
> > >> un-necessary
> > >> > > if
> > >> > > > I
> > >> > > > > > was
> > >> > > > > > > >>> in
> > >> > > > > > > >>>>>>> doubt. Hence, I was not able to automate, but I
> made
> > >> an
> > >> > > > effort
> > >> > > > > to
> > >> > > > > > > >>> go
> > >> > > > > > > >>>>>>> through PRs manually and use below criteria to
> > >> identify
> > >> > > > > > > >>> un-necessary
> > >> > > > > > > >>>>>>> commits triggering the builds.
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>>   1. Explicitly marked as WIP / do not review  PR
> > >> > > > > > > >>>>>>>   2. Incremental WIP commit and finally
> commenting a
> > >> > commit
> > >> > > > > > > >>> “trigger
> > >> > > > > > > >>>>> CI”
> > >> > > > > > > >>>>>>>   3. Multiple commits to address all comments from
> > >> single
> > >> > > > > review.
> > >> > > > > > > >>>>> This is
> > >> > > > > > > >>>>>>>   assuming we see a comment, address them, commit,
> > >> next
> > >> > the
> > >> > > > > > > >>>> following
> > >> > > > > > > >>>>>>> comment
> > >> > > > > > > >>>>>>>   4. Sequence of documentation only changes
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> I found there were around 42 avoidable builds from
> > >> most
> > >> > > > recent
> > >> > > > > 50
> > >> > > > > > > >>>>> merged
> > >> > > > > > > >>>>>>> PRs and around 86 builds from recent 50 open PRs.
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> I synced up with other contributors (Joe Evans,
> > Chai)
> > >> > from
> > >> > > > > Amazon
> > >> > > > > > > >>> who
> > >> > > > > > > >>>>> is
> > >> > > > > > > >>>>>>> contributing to MXNet CI system. I was told that
> on
> > an
> > >> > > > average
> > >> > > > > it
> > >> > > > > > > >>>> costs
> > >> > > > > > > >>>>>>> around $84 per build and on an average 6 commits
> per
> > >> > merged
> > >> > > > PR
> > >> > > > > > (for
> > >> > > > > > > >>>>> year
> > >> > > > > > > >>>>>>> 2019). Going by that, it is approximately 1/6
> builds
> > >> are
> > >> > > > > > avoidable.
> > >> > > > > > > >>>>> [100 /
> > >> > > > > > > >>>>>>> 300 + 300 ]
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> Usability should be top most priority. But, since
> > >> either
> > >> > a
> > >> > > > > > reviewer
> > >> > > > > > > >>>> or
> > >> > > > > > > >>>>> pr
> > >> > > > > > > >>>>>>> author can trigger the bot, is it really a hurdle
> > for
> > >> pr
> > >> > > > author
> > >> > > > > > or
> > >> > > > > > > >>>>> reviewer
> > >> > > > > > > >>>>>>> to call a bot to trigger CI? Given that PR author
> > and
> > >> > > > reviewer
> > >> > > > > is
> > >> > > > > > > >>>>> already
> > >> > > > > > > >>>>>>> actively commenting various details such as - PR
> > >> > > description,
> > >> > > > > > > >>> review
> > >> > > > > > > >>>>>>> comments and responses, adding labels etc.
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> Me too curious to know the behavior for Tao's
> above
> > >> use
> > >> > > case.
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> Best,
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> Sandeep
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> On Thu, Mar 12, 2020 at 7:18 PM Tao Lv <
> > >> > mutouorz@gmail.com
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>>> Is it possible for re-triggering a single job to
> be
> > >> > > abused?
> > >> > > > > For
> > >> > > > > > > >>>>> example,
> > >> > > > > > > >>>>>>>> the author spends two days re-triggering a flaky
> > job
> > >> to
> > >> > > make
> > >> > > > > it
> > >> > > > > > > >>>>> pass. But
> > >> > > > > > > >>>>>>>> other jobs which have passed the validation may
> be
> > >> > broken
> > >> > > by
> > >> > > > > > > >>> other
> > >> > > > > > > >>>>>>> commits
> > >> > > > > > > >>>>>>>> during the two day without being noticed. And
> > finally
> > >> > the
> > >> > > PR
> > >> > > > > is
> > >> > > > > > > >>>>> merged
> > >> > > > > > > >>>>>>> with
> > >> > > > > > > >>>>>>>> underlying problems.
> > >> > > > > > > >>>>>>>>
> > >> > > > > > > >>>>>>>> On Fri, Mar 13, 2020 at 6:19 AM Marco de Abreu <
> > >> > > > > > > >>>>> marco.g.abreu@gmail.com>
> > >> > > > > > > >>>>>>>> wrote:
> > >> > > > > > > >>>>>>>>
> > >> > > > > > > >>>>>>>>> In the end it only comes down to money,
> > considering
> > >> > that
> > >> > > > the
> > >> > > > > > > >>>>> system is
> > >> > > > > > > >>>>>>>> auto
> > >> > > > > > > >>>>>>>>> scaling, making the execution time constant.
> > >> > > > > > > >>>>>>>>>
> > >> > > > > > > >>>>>>>>> If we're trading money for usability, I
> certainly
> > >> would
> > >> > > > > prefer
> > >> > > > > > > >>>>>>> usability.
> > >> > > > > > > >>>>>>>>> I'd rather recommend to spend time on
> > parallelizing
> > >> > test
> > >> > > > > > > >>>> execution
> > >> > > > > > > >>>>> or
> > >> > > > > > > >>>>>>>>> getting rid of integration tests in the PR stage
> > >> > instead
> > >> > > > > > > >>> reducing
> > >> > > > > > > >>>>> the
> > >> > > > > > > >>>>>>>> costs
> > >> > > > > > > >>>>>>>>> by making people not use it. But taking a step
> > back
> > >> to
> > >> > > > > > > >>> requiring
> > >> > > > > > > >>>>> people
> > >> > > > > > > >>>>>>>> to
> > >> > > > > > > >>>>>>>>> manually trigger CI again doesn't feel right.
> > >> > > > > > > >>>>>>>>>
> > >> > > > > > > >>>>>>>>> I'm happy to see that bot deployed, but I do not
> > >> agree
> > >> > > with
> > >> > > > > > > >>>>> removing
> > >> > > > > > > >>>>>>> the
> > >> > > > > > > >>>>>>>>> auto trigger functionality for new commits.
> > >> > > > > > > >>>>>>>>>
> > >> > > > > > > >>>>>>>>> -Marco
> > >> > > > > > > >>>>>>>>>
> > >> > > > > > > >>>>>>>>> Chaitanya Bapat <chai.bapat@gmail.com> schrieb
> am
> > >> Do.,
> > >> > > 12.
> > >> > > > > > > >>> März
> > >> > > > > > > >>>>> 2020,
> > >> > > > > > > >>>>>>>>> 22:47:
> > >> > > > > > > >>>>>>>>>
> > >> > > > > > > >>>>>>>>>> @Marco Thanks for pointing that out.
> > >> > > > > > > >>>>>>>>>> Tomorrow i.e. Friday, March 13, 2020 at 3:00
> PM -
> > >> 3:30
> > >> > > PM
> > >> > > > in
> > >> > > > > > > >>>>>>>> (UTC-08:00)
> > >> > > > > > > >>>>>>>>>> Pacific Time (US & Canada).
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>> When do we expect this bot to be deployed?
> > >> > > > > > > >>>>>>>>>> @Lin If all goes well in the next week I can
> > >> deploy it
> > >> > > to
> > >> > > > > > > >>>> public
> > >> > > > > > > >>>>>>> Apache
> > >> > > > > > > >>>>>>>>>> (provided I get permissions from Apache Infra)
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>> @Marco Thanks for your feedback.
> > >> > > > > > > >>>>>>>>>>> CI system has to support the community without
> > >> > > requiring
> > >> > > > > > > >>>>> people to
> > >> > > > > > > >>>>>>>>>> constantly shepherd every single run
> > >> > > > > > > >>>>>>>>>> We have data for the number of times CI was
> > >> triggered
> > >> > > > > > > >>>>> unnecessarily
> > >> > > > > > > >>>>>>>> which
> > >> > > > > > > >>>>>>>>>> includes
> > >> > > > > > > >>>>>>>>>> - Entire build triggered instead of specific
> > build
> > >> > > > > > > >>>>>>>>>> - CI triggered when PR is still work in
> progress
> > or
> > >> > not
> > >> > > > yet
> > >> > > > > > > >>>> ready
> > >> > > > > > > >>>>>>> (say
> > >> > > > > > > >>>>>>>> -
> > >> > > > > > > >>>>>>>>>> intermediate commits)
> > >> > > > > > > >>>>>>>>>> At the end its a trade-off
> > >> > > > > > > >>>>>>>>>> Money, Resources, Time to build for each and
> > every
> > >> > > commit
> > >> > > > vs
> > >> > > > > > > >>>>> Pain of
> > >> > > > > > > >>>>>>>>>> triggering builds
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>> Scan trigger plugin would poll SCM. Can we use
> > >> plugin
> > >> > > at
> > >> > > > > > > >>>>> scale?
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>> 1. I haven't tested it on scale. But I think
> with
> > >> the
> > >> > > > > current
> > >> > > > > > > >>>>> scale
> > >> > > > > > > >>>>>>> of
> > >> > > > > > > >>>>>>>>>> MXNet repo (191 open PRs i.e. checking for
> > changes
> > >> to
> > >> > > 191
> > >> > > > > > > >>>>> branches -
> > >> > > > > > > >>>>>>> It
> > >> > > > > > > >>>>>>>>>> should be manageable)
> > >> > > > > > > >>>>>>>>>> 2. What's the purpose of the plugin? tldr;
> Branch
> > >> > > > discovery
> > >> > > > > > > >>> or
> > >> > > > > > > >>>>> branch
> > >> > > > > > > >>>>>>>>>> indexing.
> > >> > > > > > > >>>>>>>>>> Scan trigger plugin comes into the picture only
> > >> once
> > >> > per
> > >> > > > PR
> > >> > > > > > > >>> per
> > >> > > > > > > >>>>> job
> > >> > > > > > > >>>>>>>>> (i.e. 8
> > >> > > > > > > >>>>>>>>>> times per PR for 8 jobs). It is basically done
> > >> when a
> > >> > > new
> > >> > > > PR
> > >> > > > > > > >>> is
> > >> > > > > > > >>>>> made
> > >> > > > > > > >>>>>>>> and
> > >> > > > > > > >>>>>>>>>> the job (say unix-cpu hasn't discovered the new
> > PR
> > >> > > branch
> > >> > > > > > > >>> yet).
> > >> > > > > > > >>>>>>> That's
> > >> > > > > > > >>>>>>>>> it.
> > >> > > > > > > >>>>>>>>>> So it shouldn't be a problem for public MXNet
> > repo.
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>> Thanks,
> > >> > > > > > > >>>>>>>>>> Chai
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>> On Thu, 12 Mar 2020 at 14:22, Marco de Abreu <
> > >> > > > > > > >>>>>>> marco.g.abreu@gmail.com>
> > >> > > > > > > >>>>>>>>>> wrote:
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>> Btw you forgot to set a date and time for the
> > >> metting
> > >> > > > > > > >>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>> On Thu, Mar 12, 2020 at 10:18 PM Marco de
> Abreu
> > <
> > >> > > > > > > >>>>>>>>> marco.g.abreu@gmail.com
> > >> > > > > > > >>>>>>>>>>> wrote:
> > >> > > > > > > >>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>> Thanks Chai, I generally like the idea of the
> > >> bot.
> > >> > But
> > >> > > > > > > >>> I'm
> > >> > > > > > > >>>>> not a
> > >> > > > > > > >>>>>>>>>>> supporter
> > >> > > > > > > >>>>>>>>>>>> of the idea to disable any automatic
> triggering
> > >> > > > > > > >>> (disabling
> > >> > > > > > > >>>>> the
> > >> > > > > > > >>>>>>>>> webhook
> > >> > > > > > > >>>>>>>>>> is
> > >> > > > > > > >>>>>>>>>>>> also not an option, considering that this
> will
> > >> > disable
> > >> > > > > > > >>>> master
> > >> > > > > > > >>>>>>>>>> triggers).
> > >> > > > > > > >>>>>>>>>>>> The CI system has to support the community
> > >> without
> > >> > > > > > > >>>> requiring
> > >> > > > > > > >>>>>>> people
> > >> > > > > > > >>>>>>>>> to
> > >> > > > > > > >>>>>>>>>>>> constantly shepherd every single run.
> Disabling
> > >> > > > automatic
> > >> > > > > > > >>>>>>>> triggering
> > >> > > > > > > >>>>>>>>>>> seems
> > >> > > > > > > >>>>>>>>>>>> like a step back to me.
> > >> > > > > > > >>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>> Instead, I'd recommend that CI gets triggered
> > >> upon
> > >> > > every
> > >> > > > > > > >>>>> commit
> > >> > > > > > > >>>>>>> as
> > >> > > > > > > >>>>>>>>>> usual,
> > >> > > > > > > >>>>>>>>>>>> but people have the possibility to call a
> > >> "command"
> > >> > > > (i.e.
> > >> > > > > > > >>>>> make a
> > >> > > > > > > >>>>>>>>>> message
> > >> > > > > > > >>>>>>>>>>>> which results in the bot setting a label) to
> > >> disable
> > >> > > CI
> > >> > > > > > > >>>> until
> > >> > > > > > > >>>>>>> they
> > >> > > > > > > >>>>>>>>>> revoke
> > >> > > > > > > >>>>>>>>>>>> it. But the standard should still be that a
> new
> > >> > commit
> > >> > > > > > > >>>>> triggers a
> > >> > > > > > > >>>>>>>> new
> > >> > > > > > > >>>>>>>>>> CI
> > >> > > > > > > >>>>>>>>>>>> run.
> > >> > > > > > > >>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>
> > >> > > > > > > >>>>
> > >> > https://plugins.jenkins.io/multibranch-scan-webhook-trigger/
> > >> > > > > > > >>>>>>>> seems
> > >> > > > > > > >>>>>>>>>> like
> > >> > > > > > > >>>>>>>>>>>> this would poll SCM. This will incur high
> quota
> > >> > > > > > > >>>>> restrictions. Are
> > >> > > > > > > >>>>>>>> you
> > >> > > > > > > >>>>>>>>>>> sure
> > >> > > > > > > >>>>>>>>>>>> that you can use that plugin at scale?
> > >> > > > > > > >>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>> -Marco
> > >> > > > > > > >>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:04 PM Lin Yuan <
> > >> > > > > > > >>>>> apeforest@gmail.com>
> > >> > > > > > > >>>>>>>>> wrote:
> > >> > > > > > > >>>>>>>>>>>>> Chai,
> > >> > > > > > > >>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>> Awesome work. When do we expect this bot to
> be
> > >> > > > > > > >>> deployed?
> > >> > > > > > > >>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>> Best,
> > >> > > > > > > >>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>> Lin
> > >> > > > > > > >>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>> On Thu, Mar 12, 2020 at 2:00 PM Chaitanya
> > Bapat
> > >> <
> > >> > > > > > > >>>>>>>>> chai.bapat@gmail.com
> > >> > > > > > > >>>>>>>>>>>>> wrote:
> > >> > > > > > > >>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> Hello MXNet community,
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> I have built an MXNet Bot <
> > >> > > > > > > >>>> https://github.com/mxnet-bot>
> > >> > > > > > > >>>>> that
> > >> > > > > > > >>>>>>>>>> allows
> > >> > > > > > > >>>>>>>>>>> PR
> > >> > > > > > > >>>>>>>>>>>>>> Authors, Committers and Jenkins Admins to
> > >> trigger
> > >> > CI
> > >> > > > > > > >>>>> manually.
> > >> > > > > > > >>>>>>>>>>>>>> It handles 2 problems
> > >> > > > > > > >>>>>>>>>>>>>> 1. Manual CI trigger instead of existing
> > >> automated
> > >> > > CI
> > >> > > > > > > >>>>> trigger
> > >> > > > > > > >>>>>>>>>>>>>> 2. Gives permissions to PR Authors (in
> > >> addition to
> > >> > > > > > > >>>> MXNet
> > >> > > > > > > >>>>>>>>> Committers
> > >> > > > > > > >>>>>>>>>>> and
> > >> > > > > > > >>>>>>>>>>>>>> Jenkins Admins)
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> Design Doc :
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>
> > >> > > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot
> > >> > > > > > > >>>>>>>>>>>>>> I urge you all to attend the demonstration
> > >> meeting
> > >> > > > > > > >>> and
> > >> > > > > > > >>>>> lend
> > >> > > > > > > >>>>>>> your
> > >> > > > > > > >>>>>>>>>> views
> > >> > > > > > > >>>>>>>>>>>>> on
> > >> > > > > > > >>>>>>>>>>>>>> the same.
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> Thank you,
> > >> > > > > > > >>>>>>>>>>>>>> Chai
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> *Meeting Details*:
> > >> > > > > > > >>>>>>>>>>>>>> ==============Conference Bridge
> > >> > > > > > > >>>> Information==============
> > >> > > > > > > >>>>>>>>>>>>>> You have been invited to an online meeting,
> > >> > powered
> > >> > > > > > > >>> by
> > >> > > > > > > >>>>> Amazon
> > >> > > > > > > >>>>>>>>> Chime.
> > >> > > > > > > >>>>>>>>>>>>>> *Chime meeting ID*: *9272158344*
> > >> > > > > > > >>>>>>>>>>>>>> Join via Chime clients (manually): Select
> > >> > 'Meetings
> > >> > > >
> > >> > > > > > > >>>>> Join a
> > >> > > > > > > >>>>>>>>>> Meeting',
> > >> > > > > > > >>>>>>>>>>>>> and
> > >> > > > > > > >>>>>>>>>>>>>> enter 9272158344
> > >> > > > > > > >>>>>>>>>>>>>> Join via Chime clients (auto-call): If you
> > >> invite
> > >> > > > > > > >>>>> auto-call as
> > >> > > > > > > >>>>>>>>>>> attendee,
> > >> > > > > > > >>>>>>>>>>>>>> Chime will call you when the meeting
> starts,
> > >> > select
> > >> > > > > > > >>>>> 'Answer'
> > >> > > > > > > >>>>>>>>>>>>>> *Join via browser screen share*:
> > >> > > > > > > >>>>> https://chime.aws/9272158344
> > >> > > > > > > >>>>>>>>>>>>>> *Join via phone* (US):
> > >> > +1-929-432-4463,,,9272158344#
> > >> > > > > > > >>>>>>>>>>>>>> *Join via phone (US toll-free)*:
> > >> > > > > > > >>>>> +1-855-552-4463,,,9272158344#
> > >> > > > > > > >>>>>>>>>>>>>> International dial-in:
> > >> > > > > > > >>>> https://chime.aws/dialinnumbers/
> > >> > > > > > > >>>>>>>>>>>>>> In-room video system: Ext: 62000, Meeting
> > PIN:
> > >> > > > > > > >>>>> 9272158344#
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> --
> > >> > > > > > > >>>>>>>>>>>>>> *Chaitanya Prakash Bapat*
> > >> > > > > > > >>>>>>>>>>>>>> *+1 (973) 953-6299*
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>>>>> [image:
> > >> https://www.linkedin.com//in/chaibapat25]
> > >> > > > > > > >>>>>>>>>>>>>> <https://github.com/ChaiBapchya>[image:
> > >> > > > > > > >>>>>>>>>>>>> https://www.facebook.com/chaibapat
> > >> > > > > > > >>>>>>>>>>>>>> ]
> > >> > > > > > > >>>>>>>>>>>>>> <https://www.facebook.com/chaibapchya
> > >[image:
> > >> > > > > > > >>>>>>>>>>>>>> https://twitter.com/ChaiBapchya] <
> > >> > > > > > > >>>>>>>> https://twitter.com/ChaiBapchya
> > >> > > > > > > >>>>>>>>>>>>>> [image:
> > >> > > > > > > >>>>>>>>>>>>>> https://www.linkedin.com//in/chaibapat25]
> > >> > > > > > > >>>>>>>>>>>>>> <https://www.linkedin.com//in/chaibapchya/
> >
> > >> > > > > > > >>>>>>>>>>>>>>
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>> --
> > >> > > > > > > >>>>>>>>>> *Chaitanya Prakash Bapat*
> > >> > > > > > > >>>>>>>>>> *+1 (973) 953-6299*
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>>>> [image:
> https://www.linkedin.com//in/chaibapat25
> > ]
> > >> > > > > > > >>>>>>>>>> <https://github.com/ChaiBapchya>[image:
> > >> > > > > > > >>>>>>>>> https://www.facebook.com/chaibapat
> > >> > > > > > > >>>>>>>>>> ]
> > >> > > > > > > >>>>>>>>>> <https://www.facebook.com/chaibapchya>[image:
> > >> > > > > > > >>>>>>>>>> https://twitter.com/ChaiBapchya] <
> > >> > > > > > > >>>>> https://twitter.com/ChaiBapchya
> > >> > > > > > > >>>>>>>>>> [image:
> > >> > > > > > > >>>>>>>>>> https://www.linkedin.com//in/chaibapat25]
> > >> > > > > > > >>>>>>>>>> <https://www.linkedin.com//in/chaibapchya/>
> > >> > > > > > > >>>>>>>>>>
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>>> --
> > >> > > > > > > >>>>>>> Sandeep Krishnamurthy
> > >> > > > > > > >>>>>>>
> > >> > > > > > > >>>>>
> > >> > > > > > > >>>>
> > >> > > > > > > >>>
> > >> > > > > > > >>>
> > >> > > > > > > >>> --
> > >> > > > > > > >>> *Chaitanya Prakash Bapat*
> > >> > > > > > > >>> *+1 (973) 953-6299*
> > >> > > > > > > >>>
> > >> > > > > > > >>> [image: https://www.linkedin.com//in/chaibapat25]
> > >> > > > > > > >>> <https://github.com/ChaiBapchya>[image:
> > >> > > > > > > https://www.facebook.com/chaibapat
> > >> > > > > > > >>> ]
> > >> > > > > > > >>> <https://www.facebook.com/chaibapchya>[image:
> > >> > > > > > > >>> https://twitter.com/ChaiBapchya] <
> > >> > > > https://twitter.com/ChaiBapchya
> > >> > > > > > > >[image:
> > >> > > > > > > >>> https://www.linkedin.com//in/chaibapat25]
> > >> > > > > > > >>> <https://www.linkedin.com//in/chaibapchya/>
> > >> > > > > > > >>>
> > >> > > > > > > >>
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > *Chaitanya Prakash Bapat*
> > >> > > > > *+1 (973) 953-6299*
> > >> > > > >
> > >> > > > > [image: https://www.linkedin.com//in/chaibapat25]
> > >> > > > > <https://github.com/ChaiBapchya>[image:
> > >> > > > https://www.facebook.com/chaibapat
> > >> > > > > ]
> > >> > > > > <https://www.facebook.com/chaibapchya>[image:
> > >> > > > > https://twitter.com/ChaiBapchya] <
> > https://twitter.com/ChaiBapchya
> > >> > > > >[image:
> > >> > > > > https://www.linkedin.com//in/chaibapat25]
> > >> > > > > <https://www.linkedin.com//in/chaibapchya/>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > *Chaitanya Prakash Bapat*
> > >> > > *+1 (973) 953-6299*
> > >> > >
> > >> > > [image: https://www.linkedin.com//in/chaibapat25]
> > >> > > <https://github.com/ChaiBapchya>[image:
> > >> > https://www.facebook.com/chaibapat
> > >> > > ]
> > >> > > <https://www.facebook.com/chaibapchya>[image:
> > >> > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> > >> > >[image:
> > >> > > https://www.linkedin.com//in/chaibapat25]
> > >> > > <https://www.linkedin.com//in/chaibapchya/>
> > >> > >
> > >> >
> > >>
> > >>
> > >> --
> > >> *Chaitanya Prakash Bapat*
> > >> *+1 (973) 953-6299*
> > >>
> > >> [image: https://www.linkedin.com//in/chaibapat25]
> > >> <https://github.com/ChaiBapchya>[image:
> > >> https://www.facebook.com/chaibapat]
> > >> <https://www.facebook.com/chaibapchya>[image:
> > >> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> > >[image:
> > >> https://www.linkedin.com//in/chaibapat25]
> > >> <https://www.linkedin.com//in/chaibapchya/>
> > >>
> > >
> >
>
>
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
>
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat
> ]
> <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message