openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <csantan...@gmail.com>
Subject Re: How to best run non-local tests in ASF (was: Performance tests for OpenWhisk)
Date Sun, 08 Apr 2018 01:36:07 GMT
Sorry Sebastian about that, I thought you were the one I had the discussion
in the infra chat.

I think it’s a good suggestion to follow up in the infra mailing list.

Michael could you direct the inquiries on the infra list?

— Carlos
On Sat, Apr 7, 2018 at 6:47 AM sebb <sebbaz@gmail.com> wrote:

> On 6 April 2018 at 15:49, Carlos Santana <csantana23@gmail.com> wrote:
> > Sebastian Bazley from ASF INFRA would be good resource to contact to
> drive
> > this.
>
> I'm not sure why you used my name.
> Although I do some infra-related work, I am not actually part of Infra.
> (nor do I wish to be).
>
> Nor am I familiar with setting up VMs.
>
> Sorry, but this is not something I could take on, nor do I wish to.
>
> Please contact Infra via its user mailing list.
>
> > In the past when I brought this with infra I think I understood from him
> > that companies can donate funds and explicitly state that are for the
> > OpenWhisk project then the INFRA team will allocate VMs to OpenWhisk to
> run
> > load/performance tests, same way that Hadoop project has decimated VMs in
> > Jenkins today.
> >
> > — Carlos
> >
> > On Fri, Apr 6, 2018 at 9:38 AM Michael Marth <mmarth@adobe.com.invalid>
> > wrote:
> >>
> >> Hi mentors (and others),
> >>
> >> Had an offline discussion this week in which the question came up how an
> >> ASF project should best go about running
> performance/throughput/scalability
> >> tests – i.e. tests that cannot be run locally and require a repeatable
> >> environment.
> >>
> >> Some options:
> >> * interested companies run the tests on their own infra and publish
> >> results. Pretty lame, especially because typically only that company’s
> >> engineers can access the env and investigate further.
> >> * interested companies donate cash to sponsor compute resources,
> >> committers can run and investigate the tests. Ideal from tech
> perspective,
> >> but I have no idea how that cash would make its way from the ASF to a
> >> particular project.
> >> * maybe a middle-ground: interested party that happens to have a public
> >> cloud offering gives credentials to committers
> >>
> >> I am mainly interested to learn if there are other ASF projects (e.g. in
> >> the Big Data/Hadoop ecosystem) that do something similar. Or if there
> is an
> >> ASF-recommended way to do this. Or else, where I could ask this
> question?
> >>
> >> Thanks!
> >> Michael
> >>
> >> From: Michael Marth <mmarth@adobe.com>
> >> Date: Wednesday 3 May 2017 20:57
> >> To: "dev@openwhisk.apache.org" <dev@openwhisk.apache.org>
> >> Subject: Re: Performance tests for OpenWhisk
> >>
> >> Markus,
> >>
> >> Quick update: sent the below to users@infra. So far no reaction. The
> >> archive is here [1] but Bertrand tells me only ASF member have access
> - for
> >> whatever reason.
> >>
> >> Michael
> >>
> >> [1]
> >>
> https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E
> >>
> >>
> >> On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth
> >> <mmarth@adobe.com<mailto:mmarth@adobe.com>> wrote:
> >> Dear Infra team,
> >>
> >> I am enquiring on behalf of the OpenWhisk project (currently in
> Incubator)
> >> [1].
> >>
> >> We would like to periodically run performance tests on a distributed
> >> environment (OpenWhisk typically runs on more than 1 machine). So we are
> >> basically looking for an ability to spin up/tear down a number of
> >> (virtual)
> >> machines and exclusively use them for a certain amount of time (so that
> >> the
> >> VMs are not shared and the performance test results are comparable over
> >> time).
> >> The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week.
> >>
> >> I would like to find out if there is an ASF-supported mechanism to do
> >> that.
> >> For example, can Infra provide such infrastructure? Or is there a cloud
> >> provider (like Azure) that might sponsor such efforts with VMs? Or maybe
> >> there is an established way for commercial companies that are interested
> >> in
> >> an ASF project to sponsor (fund) such tests?
> >>
> >> If none of the above exists, then it would also be helpful for us to get
> >> to
> >> know how other projects run such sort of tests.
> >>
> >> Thanks a lot!
> >> Michael
> >>
> >>
> >> [1]
> >>
> >>
> https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E
> >>
> >>
> >>
> >>
> >> From: Markus Thömmes <markusthoemmes@me.com<mailto:
> markusthoemmes@me.com>>
> >> Reply-To: "dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>"
> >> <dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>>
> >> Date: Wednesday 26 April 2017 12:59
> >> To: "dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>"
> >> <dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>>
> >> Subject: Re: Performance tests for OpenWhisk
> >>
> >> Hi Michael,
> >>
> >> yeah that sounds pretty much spot on. I'd like to have at least 2 VMs
> with
> >> 4+ cores and 8GB memory. One VM would host the management stack while
> one
> >> would be dedicated to an Invoker only. That way we could assert
> >> single-invoker performance the easiest.
> >>
> >> Thanks for helping!
> >>
> >> Cheers,
> >> Markus
> >>
> >> Am 26. April 2017 um 11:36 schrieb Michael Marth
> >> <mmarth@adobe.com<mailto:mmarth@adobe.com>>:
> >> Markus,
> >>
> >> Does what I describe reflect what you are looking for?
> >> If yes, I am happy to ask on infra.
> >>
> >> Let me know
> >> Michael
> >>
> >>
> >>
> >> On 26/04/17 07:52, "Bertrand Delacretaz"
> >> <bdelacretaz@apache.org<mailto:bdelacretaz@apache.org>> wrote:
> >>
> >>
> >> Hi Michael,
> >>
> >> On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth
> >> <mmarth@adobe.com<mailto:mmarth@adobe.com>> wrote:
> >> ...Maybe our mentors can chime in. Has this been discussed in the ASF
> >> board or so?...
> >>
> >> Best would be to ask the ASF infrastructure team via
> >> users@infra.apache.org<mailto:users@infra.apache.org> - briefly
> describe
> >> what you need to see what's
> >> possible.
> >>
> >> -Bertrand
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message