openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <csantan...@gmail.com>
Subject Re: How to best run non-local tests in ASF (was: Performance tests for OpenWhisk)
Date Fri, 06 Apr 2018 14:49:21 GMT
Sebastian Bazley from ASF INFRA would be good resource to contact to drive
this.

In the past when I brought this with infra I think I understood from him
that companies can donate funds and explicitly state that are for the
OpenWhisk project then the INFRA team will allocate VMs to OpenWhisk to run
load/performance tests, same way that Hadoop project has decimated VMs in
Jenkins today.

— Carlos

On Fri, Apr 6, 2018 at 9:38 AM Michael Marth <mmarth@adobe.com.invalid>
wrote:

> Hi mentors (and others),
>
> Had an offline discussion this week in which the question came up how an
> ASF project should best go about running performance/throughput/scalability
> tests – i.e. tests that cannot be run locally and require a repeatable
> environment.
>
> Some options:
> * interested companies run the tests on their own infra and publish
> results. Pretty lame, especially because typically only that company’s
> engineers can access the env and investigate further.
> * interested companies donate cash to sponsor compute resources,
> committers can run and investigate the tests. Ideal from tech perspective,
> but I have no idea how that cash would make its way from the ASF to a
> particular project.
> * maybe a middle-ground: interested party that happens to have a public
> cloud offering gives credentials to committers
>
> I am mainly interested to learn if there are other ASF projects (e.g. in
> the Big Data/Hadoop ecosystem) that do something similar. Or if there is an
> ASF-recommended way to do this. Or else, where I could ask this question?
>
> Thanks!
> Michael
>
> From: Michael Marth <mmarth@adobe.com>
> Date: Wednesday 3 May 2017 20:57
> To: "dev@openwhisk.apache.org" <dev@openwhisk.apache.org>
> Subject: Re: Performance tests for OpenWhisk
>
> Markus,
>
> Quick update: sent the below to users@infra. So far no reaction. The
> archive is here [1] but Bertrand tells me only ASF member have access  -
> for whatever reason.
>
> Michael
>
> [1]
> https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E
>
>
> On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth <mmarth@adobe.com<mailto:
> mmarth@adobe.com>> wrote:
> Dear Infra team,
>
> I am enquiring on behalf of the OpenWhisk project (currently in Incubator)
> [1].
>
> We would like to periodically run performance tests on a distributed
> environment (OpenWhisk typically runs on more than 1 machine). So we are
> basically looking for an ability to spin up/tear down a number of (virtual)
> machines and exclusively use them for a certain amount of time (so that the
> VMs are not shared and the performance test results are comparable over
> time).
> The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week.
>
> I would like to find out if there is an ASF-supported mechanism to do that.
> For example, can Infra provide such infrastructure? Or is there a cloud
> provider (like Azure) that might sponsor such efforts with VMs? Or maybe
> there is an established way for commercial companies that are interested in
> an ASF project to sponsor (fund) such tests?
>
> If none of the above exists, then it would also be helpful for us to get to
> know how other projects run such sort of tests.
>
> Thanks a lot!
> Michael
>
>
> [1]
>
> https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E
>
>
>
>
> From: Markus Thömmes <markusthoemmes@me.com<mailto:markusthoemmes@me.com>>
> Reply-To: "dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>" <
> dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>>
> Date: Wednesday 26 April 2017 12:59
> To: "dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>" <
> dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>>
> Subject: Re: Performance tests for OpenWhisk
>
> Hi Michael,
>
> yeah that sounds pretty much spot on. I'd like to have at least 2 VMs with
> 4+ cores and 8GB memory. One VM would host the management stack while one
> would be dedicated to an Invoker only. That way we could assert
> single-invoker performance the easiest.
>
> Thanks for helping!
>
> Cheers,
> Markus
>
> Am 26. April 2017 um 11:36 schrieb Michael Marth <mmarth@adobe.com<mailto:
> mmarth@adobe.com>>:
> Markus,
>
> Does what I describe reflect what you are looking for?
> If yes, I am happy to ask on infra.
>
> Let me know
> Michael
>
>
>
> On 26/04/17 07:52, "Bertrand Delacretaz" <bdelacretaz@apache.org<mailto:
> bdelacretaz@apache.org>> wrote:
>
>
> Hi Michael,
>
> On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth <mmarth@adobe.com<mailto:
> mmarth@adobe.com>> wrote:
> ...Maybe our mentors can chime in. Has this been discussed in the ASF
> board or so?...
>
> Best would be to ask the ASF infrastructure team via
> users@infra.apache.org<mailto:users@infra.apache.org> - briefly describe
> what you need to see what's
> possible.
>
> -Bertrand
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message