openwhisk-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From markusthoem...@apache.org
Subject [incubator-openwhisk] branch master updated: Move performance repository to core repository. (#3475)
Date Wed, 04 Apr 2018 13:55:11 GMT
This is an automated email from the ASF dual-hosted git repository.

markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk.git


The following commit(s) were added to refs/heads/master by this push:
     new f3f1cb9  Move performance repository to core repository. (#3475)
f3f1cb9 is described below

commit f3f1cb9171e0b95a27ac8f43dd8eafd5bce2f938
Author: Christian Bickel <github@cbickel.de>
AuthorDate: Wed Apr 4 15:55:05 2018 +0200

    Move performance repository to core repository. (#3475)
    
    Co-authored-by: Markus Thömmes <markusthoemmes@me.com>
    Co-authored-by: Jeremias Werner <jeremias.werner@gmail.com>
    Co-authored-by: Alexandre Nicastro <kryndex@gmail.com>
---
 performance/.travis.yml             | 14 ++++++++++++++
 performance/README.md               | 33 +++++++++++++++++++++++++++++++++
 performance/preparation/create.sh   | 17 +++++++++++++++++
 performance/preparation/deploy.sh   | 25 +++++++++++++++++++++++++
 performance/wrk_tests/latency.sh    | 12 ++++++++++++
 performance/wrk_tests/post.lua      |  1 +
 performance/wrk_tests/throughput.sh | 31 +++++++++++++++++++++++++++++++
 7 files changed, 133 insertions(+)

diff --git a/performance/.travis.yml b/performance/.travis.yml
new file mode 100644
index 0000000..b425c60
--- /dev/null
+++ b/performance/.travis.yml
@@ -0,0 +1,14 @@
+sudo: required
+language: java
+services:
+  - docker
+
+env:
+  - TERM=dumb
+
+install:
+  - ./preparation/deploy.sh
+
+script:
+  - ./wrk_tests/latency.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)"
2m
+  - ./wrk_tests/throughput.sh "http://172.17.0.1:10001" "$(cat openwhisk/ansible/files/auth.guest)"
4 2 2m
diff --git a/performance/README.md b/performance/README.md
new file mode 100644
index 0000000..c1d668f
--- /dev/null
+++ b/performance/README.md
@@ -0,0 +1,33 @@
+# :electric_plug: Apache OpenWhisk - Performance Tests
+A few simple but efficient test suites for determining the maximum throughput and end-user
latency of the Apache OpenWhisk system.
+
+## Workflow
+- A standard OpenWhisk system is deployed. (_Note that the edge NGINX router and API Gateway
are currently left out. As a consequence, the tests talk directly to the controller._)
+- All limits are set to 999999, which in our current use case means "No throttling at all".
+- The deployment is using the docker setup proposed by the OpenWhisk development team: `overlay`
driver and HTTP API enabled via a UNIX port.
+
+The load is driven by the blazingly fast [`wrk`](https://github.com/wg/wrk).
+
+#### Travis Machine Setup
+The [machine provided by Travis](https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments)
has ~2 CPU cores (likely shared through virtualization) and 7.5GB memory.
+
+## Suites
+
+### Latency Test
+Determines the end-to-end latency a user experience when doing a blocking invocation. The
action used is a no-op so the numbers returned are the plain overhead of the OpenWhisk system.
+
+- 1 HTTP request at a time (concurrency: 1)
+- You can specify how long this test will run. Default are 30s.
+- no-op action
+
+**Note:** The throughput number has a 100% correlation with the latency in this case. This
test does not serve to determine the maximum throughput of the system.
+
+### Throughput Test
+Determines the maximum throughput a user can get out of the system while using a single action.
The action used is a no-op, so the numbers are plain OpenWhisk overhead. Note that the throughput
does not directly correlate to end-to-end latency here, as the system does more processing
in the background as it shows to the user in a blocking invocation.
+
+- 4 HTTP requests at a time (concurrency: 4) (using CPU cores * 2 to exploit some buffering)
+- 10.000 samples with a single user
+- no-op action
+
+## Running tests against your own system is simple too!
+All you have to do is use the corresponding script located in /*_tests folder, remembering
that the parameters are defined inline.
diff --git a/performance/preparation/create.sh b/performance/preparation/create.sh
new file mode 100755
index 0000000..1a79649
--- /dev/null
+++ b/performance/preparation/create.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+set -e
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# Name of the action to create and test.
+action=$3
+
+# create a noop action
+echo "Creating action $action"
+curl -k -u "$credentials" "$host/api/v1/namespaces/_/actions/$action" -XPUT -d '{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function
main(){return {};}"}}' -H "Content-Type: application/json"
+
+# run the noop action
+echo "Running $action once to assert an intact system"
+curl -k -u "$credentials" "$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
diff --git a/performance/preparation/deploy.sh b/performance/preparation/deploy.sh
new file mode 100755
index 0000000..5628176
--- /dev/null
+++ b/performance/preparation/deploy.sh
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+set -e
+currentDir="$(cd "$(dirname "$0")"; pwd)"
+
+# common docker setup
+sudo gpasswd -a travis docker
+sudo -E bash -c 'echo '\''DOCKER_OPTS="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
--storage-driver=overlay --userns-remap=default"'\'' > /etc/default/docker'
+sudo service docker restart
+
+# install ansible
+pip install --user ansible==2.4.2.0
+
+cd $currentDir/../../ansible
+ANSIBLE_CMD="ansible-playbook -i environments/local -e docker_image_prefix=openwhisk -e docker_registry=docker.io/
-e limit_invocations_per_minute=999999 -e limit_invocations_concurrent=999999 -e limit_invocations_concurrent_system=999999"
+
+$ANSIBLE_CMD setup.yml
+$ANSIBLE_CMD prereq.yml
+$ANSIBLE_CMD couchdb.yml
+$ANSIBLE_CMD initdb.yml
+$ANSIBLE_CMD wipe.yml
+
+$ANSIBLE_CMD kafka.yml
+$ANSIBLE_CMD controller.yml
+$ANSIBLE_CMD invoker.yml
diff --git a/performance/wrk_tests/latency.sh b/performance/wrk_tests/latency.sh
new file mode 100755
index 0000000..bee706b
--- /dev/null
+++ b/performance/wrk_tests/latency.sh
@@ -0,0 +1,12 @@
+#!/bin/sh
+set -e
+currentDir="$(cd "$(dirname "$0")"; pwd)"
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# How long to run the test
+duration=${3:-30s}
+
+$currentDir/throughput.sh $host $credentials 1 1 $duration
diff --git a/performance/wrk_tests/post.lua b/performance/wrk_tests/post.lua
new file mode 100644
index 0000000..e3e664c
--- /dev/null
+++ b/performance/wrk_tests/post.lua
@@ -0,0 +1 @@
+wrk.method = "POST"
\ No newline at end of file
diff --git a/performance/wrk_tests/throughput.sh b/performance/wrk_tests/throughput.sh
new file mode 100755
index 0000000..6ed2453
--- /dev/null
+++ b/performance/wrk_tests/throughput.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+set -e
+currentDir="$(cd "$(dirname "$0")"; pwd)"
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# concurrency level of the throughput test: How many requests should
+# open in parallel.
+concurrency=$3
+# How many threads to utilize, directly correlates to the number
+# of CPU cores
+threads=${4:-4}
+# How long to run the test
+duration=${5:-30s}
+
+action="noopThroughput"
+"$currentDir/../preparation/create.sh" "$host" "$credentials" "$action"
+
+# run throughput tests
+encodedAuth=$(echo "$credentials" | tr -d '\n' | base64 | tr -d '\n')
+docker run --pid=host --userns=host --rm -v "$currentDir":/data williamyeh/wrk \
+  --threads "$threads" \
+  --connections "$concurrency" \
+  --duration "$duration" \
+  --header "Authorization: basic $encodedAuth" \
+  "$host/api/v1/namespaces/_/actions/$action?blocking=true" \
+  --latency \
+  --timeout 10s \
+  --script post.lua

-- 
To stop receiving notification emails like this one, please contact
markusthoemmes@apache.org.

Mime
View raw message