Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A5631200C1A for ; Mon, 13 Feb 2017 12:29:45 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id A4265160B60; Mon, 13 Feb 2017 11:29:45 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 08250160B87 for ; Mon, 13 Feb 2017 12:29:42 +0100 (CET) Received: (qmail 76874 invoked by uid 500); 13 Feb 2017 11:29:42 -0000 Mailing-List: contact commits-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list commits@ignite.apache.org Received: (qmail 75710 invoked by uid 99); 13 Feb 2017 11:29:41 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Feb 2017 11:29:41 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 958E6E042B; Mon, 13 Feb 2017 11:29:41 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: vozerov@apache.org To: commits@ignite.apache.org Date: Mon, 13 Feb 2017 11:30:09 -0000 Message-Id: <5f557544596747ee89aa13c1816ed229@git.apache.org> In-Reply-To: <386653f669a24bbd84e5e61fe97baced@git.apache.org> References: <386653f669a24bbd84e5e61fe97baced@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [29/40] ignite git commit: IGNITE-4212 Ignite Benchmarking Simplification and Automation / IGNITE-4478 Fixing documentation and resolving lack of usability archived-at: Mon, 13 Feb 2017 11:29:45 -0000 IGNITE-4212 Ignite Benchmarking Simplification and Automation / IGNITE-4478 Fixing documentation and resolving lack of usability Project: http://git-wip-us.apache.org/repos/asf/ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/605d946a Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/605d946a Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/605d946a Branch: refs/heads/ignite-2.0 Commit: 605d946ada5ca4e562e256fa0edb29cc25dea7dc Parents: e1c3dda Author: oleg-ostanin Authored: Thu Feb 9 20:08:57 2017 +0300 Committer: Anton Vinogradov Committed: Thu Feb 9 20:08:57 2017 +0300 ---------------------------------------------------------------------- modules/yardstick/DEVNOTES-standalone.txt | 15 +++ modules/yardstick/DEVNOTES.txt | 20 ++-- modules/yardstick/README.txt | 119 +++++++++++++------ .../config/benchmark-sample.properties | 62 ++++++++++ modules/yardstick/pom.xml | 4 +- pom.xml | 1 + 6 files changed, 172 insertions(+), 49 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/DEVNOTES-standalone.txt ---------------------------------------------------------------------- diff --git a/modules/yardstick/DEVNOTES-standalone.txt b/modules/yardstick/DEVNOTES-standalone.txt new file mode 100644 index 0000000..d14dde2 --- /dev/null +++ b/modules/yardstick/DEVNOTES-standalone.txt @@ -0,0 +1,15 @@ +Building from standalone sources +================================ + +Run + +mvn clean package + +Artifacts can be found in `/target/assembly directory.` + + +Writing Ignite Benchmarks +========================= + +All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this abstract class +and implement `test` method. This is the method which actually tests performance. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/DEVNOTES.txt ---------------------------------------------------------------------- diff --git a/modules/yardstick/DEVNOTES.txt b/modules/yardstick/DEVNOTES.txt index 6921243..b8b1f53 100644 --- a/modules/yardstick/DEVNOTES.txt +++ b/modules/yardstick/DEVNOTES.txt @@ -1,16 +1,20 @@ -Yardstick Ignite Maven Build Instructions +Building from Ignite sources ========================================= -Yardstick can be build from standalone sources using following maven command: +Run -mvn clean package +mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests -Artifacts can be found in /target/assembly directory. +in the Apache Ignite root directory. -To build yardstick from Apache Ignite sources use: +This command will compile the project and also unpack scripts from `yardstick-resources.zip` file to +`modules/yardstick/target/assembly/bin` directory. -mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests +Artifacts can be found in `modules/yardstick/target/assembly` directory. + +Writing Ignite Benchmarks +========================= -in Apache Ignite root directory +All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this abstract class +and implement `test` method. This is the method which actually tests performance. -Artifacts can be found in modules/yardstick/target/assembly directory. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/README.txt ---------------------------------------------------------------------- diff --git a/modules/yardstick/README.txt b/modules/yardstick/README.txt index 353ddd8..dcfe0e6 100644 --- a/modules/yardstick/README.txt +++ b/modules/yardstick/README.txt @@ -1,55 +1,92 @@ Yardstick Ignite Benchmarks =========================== -Yardstick Ignite is a set of Ignite Grid (http://ignite.apache.org/) benchmarks written on top of Yardstick framework. +Apache Ignite benchmarks are written on top of Yardstick Framework (https://github.com/gridgain/yardstick) and allow +you to measure performance of various Apache Ignite components and modules. -Yardstick Framework -=================== -Visit Yardstick Repository (https://github.com/gridgain/yardstick) for detailed information on how to run Yardstick benchmarks and how to generate graphs. +The documentation below describes how to execute and configure already assembled benchmarks. If you need to add new +benchmarks or build existing one then refer to instruction from `DEVNOTES.txt` in source directory. -The documentation below describes configuration parameters in addition to standard Yardstick parameters. +Visit Yardstick Repository (https://github.com/gridgain/yardstick) for detailed information in regards resulting graphs +generation and how the frameworks works. -Building from Ignite sources -============ -1. Create a local clone of Ignite repository -2. Run -mvn package +Running Ignite Benchmarks Locally +========================================== -command for Yardstick Ignite POM +The simplest way to start with benchmarking is to use one of the executable scripts available under `benchmarks\bin` directory: -Building from standalone sources -===================== -Run +./bin/benchmark-run-all.sh config/benchmark-atomic-put.properties -mvn package +The command above will benchmark the cache put operation for a distributed atomic cache. The results of the benchmark will be added +to auto-generated `results-{DATE-TIME}` directory. -command for Yardstick Ignite POM +If `./bin/benchmark-run-all.sh` command is executed as is without any parameters and modifications in configurations files then +all the available benchmarks will be executed on a local machine using `config/benchmark.properties` configuration. -Provided Benchmarks -=================== -The following benchmarks are provided: +To get more information about available benchmarks and configuration parameters refer to “Provided Benchmarks” and +“Properties And Command Line Arguments” sections below. + + +Running Ignite Benchmarks Remotely +========================================= + +For running Ignite benchmarks on remote hosts you need to upload Ignite-Yardstick to each one of your remote hosts. + +NOTE: The path to the uploaded Ignite-Yardstick should be exactly the same on each host. + +Then you need to make some changes in config files: -1. `GetBenchmark` - benchmarks atomic distributed cache get operation -2. `PutBenchmark` - benchmarks atomic distributed cache put operation -3. `PutGetBenchmark` - benchmarks atomic distributed cache put and get operations together -4. `PutTxBenchmark` - benchmarks transactional distributed cache put operation -5. `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get operations together -6. `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data -7. `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over cached data -8. `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous cache updates +1. You need to comment or delete the + +line in `config/ignite-localhost-config.xml` file. -Writing Ignite Benchmarks -========================= -All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this abstract class -and implement `test` method. This is the method that is actually benchmarked. +2. You need to replace all the `127.0.0.1` addresses in `ignite-localhost-config.xml` file by actual IPs of your remote +servers. You can add or delete lines with IP addresses if you want to run benchmarks on different number of hosts. +There must be at least one IP address in the list. -Running Ignite Benchmarks -========================= -Before running Ignite benchmarks, run: +3. You need to replace the `localhost` strings by actual IP of your servers in the lines +SERVERS='localhost,localhost' +DRIVERS='localhost' +in `config/benchmark-atomic-put.properties` file. + +Then use the following command: + +./bin/benchmark-run-all.sh config/benchmark-atomic-put.properties + + +It is recommended to create some copies of original config files, edit these copies and then use as a +parameter for `./bin/benchmark-run-all.sh` script. + + +Provided Benchmarks +=================== +The following benchmarks are provided: -mvn package +1. `GetBenchmark` - benchmarks atomic distributed cache get operation +2. `PutBenchmark` - benchmarks atomic distributed cache put operation +3. `PutGetBenchmark` - benchmarks atomic distributed cache put and get operations together +4. `PutTxBenchmark` - benchmarks transactional distributed cache put operation +5. `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get operations together +6. `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data +7. `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over cached data +8. `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous cache updates +9. `AffinityCallBenchmark` - benchmarks affinity call operation +10. `ApplyBenchmark` - benchmarks apply operation +11. `BroadcastBenchmark` - benchmarks broadcast operations +12. `ExecuteBenchmark` - benchmarks execute operations +13. `RunBenchmark` - benchmarks running task operations +14. `PutGetOffHeapBenchmark` - benchmarks atomic distributed cache put and get operations together off heap +15. `PutGetOffHeapValuesBenchmark` - benchmarks atomic distributed cache put value operations off heap +16. `PutOffHeapBenchmark` - benchmarks atomic distributed cache put operations off heap +17. `PutOffHeapValuesBenchmark` - benchmarks atomic distributed cache get value operations off heap +18. `PutTxOffHeapBenchmark` - benchmarks transactional distributed cache put operation off heap +19. `PutTxOffHeapValuesBenchmark` - benchmarks transactional distributed cache put value operation off heap +20. `SqlQueryOffHeapBenchmark` -benchmarks distributed SQL query over cached data off heap +21. `SqlQueryJoinOffHeapBenchmark` - benchmarks distributed SQL query with a Join over cached data off heap +22. `SqlQueryPutOffHeapBenchmark` - benchmarks distributed SQL query with simultaneous cache updates off heap +23. `PutAllBenchmark` - benchmarks atomic distributed cache batch put operation +24. `PutAllTxBenchmark` - benchmarks transactional distributed cache batch put operation -command. This command will compile the project and also will unpack scripts from `yardstick-resources.zip` file to `bin` directory. Properties And Command Line Arguments ===================================== @@ -67,15 +104,19 @@ The following Ignite benchmark properties can be defined in the benchmark config * `-cs` or `--cacheStore` - Enable or disable cache store readThrough, writeThrough * `-cl` or `--client` - Client flag * `-nc` or `--nearCache` - Near cache flag -* `-nn ` or `--nodeNumber ` - Number of nodes (automatically set in `benchmark.properties`), used to wait for the specified number of nodes to start +* `-nn ` or `--nodeNumber ` - Number of nodes (automatically set in `benchmark.properties`), used to wait for + the specified number of nodes to start * `-sm ` or `-syncMode ` - Synchronization mode (defined in `CacheWriteSynchronizationMode`) * `-ot` or `--offheapTiered` - Flag indicating whether tiered off-heap mode is on * `-ov` or `--offheapValuesOnly` - Flag indicating whether off-heap mode is on and only cache values are stored off-heap * `-r ` or `--range` - Range of keys that are randomly generated for cache operations +* `-rd or --restartdelay` - Restart delay in seconds +* `-rs or --restartsleep` - Restart sleep in seconds * `-rth ` or `--restHost ` - REST TCP host * `-rtp ` or `--restPort ` - REST TCP port, indicates that a Ignite node is ready to process Ignite Clients * `-ss` or `--syncSend` - Flag indicating whether synchronous send is used in `TcpCommunicationSpi` -* `-txc ` or `--txConcurrency ` - Cache transaction concurrency control, either `OPTIMISTIC` or `PESSIMISTIC` (defined in `CacheTxConcurrency`) +* `-txc ` or `--txConcurrency ` - Cache transaction concurrency control, either `OPTIMISTIC` or + `PESSIMISTIC` (defined in `CacheTxConcurrency`) * `-txi ` or `--txIsolation ` - Cache transaction isolation (defined in `CacheTxIsolation`) * `-wb` or `--writeBehind` - Enable or disable writeBehind for cache store * `-wom ` or `--writeOrderMode ` - Write order mode for ATOMIC caches (defined in `CacheAtomicWriteOrderMode`) @@ -91,7 +132,7 @@ SERVER_HOSTS=localhost,localhost Note that -dn and -sn, which stand for data node and server node, are native Yardstick parameters and are documented in Yardstick framework. =========================================================================================================================================== -CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark -sn IgniteNode" +CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark`IgniteNode" ``` Issues http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/config/benchmark-sample.properties ---------------------------------------------------------------------- diff --git a/modules/yardstick/config/benchmark-sample.properties b/modules/yardstick/config/benchmark-sample.properties new file mode 100644 index 0000000..fee7cbb --- /dev/null +++ b/modules/yardstick/config/benchmark-sample.properties @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Contains benchmarks for ATOMIC cache. +# + +# JVM options. +JVM_OPTS=${JVM_OPTS}" -DIGNITE_QUIET=false" + +# Uncomment to enable concurrent garbage collection (GC) if you encounter long GC pauses. +# JVM_OPTS=${JVM_OPTS}" \ +# -XX:+UseParNewGC \ +# -XX:+UseConcMarkSweepGC \ +# -XX:+UseTLAB \ +# -XX:NewSize=128m \ +# -XX:MaxNewSize=128m \ +# -XX:MaxTenuringThreshold=0 \ +# -XX:SurvivorRatio=1024 \ +# -XX:+UseCMSInitiatingOccupancyOnly \ +# -XX:CMSInitiatingOccupancyFraction=60 \ +#" + +# List of default probes. +# Add DStatProbe or VmStatProbe if your OS supports it (e.g. if running on Linux). +BENCHMARK_DEFAULT_PROBES=ThroughputLatencyProbe,PercentileProbe + +# Packages where the specified benchmark is searched by reflection mechanism. +BENCHMARK_PACKAGES=org.yardstickframework,org.apache.ignite.yardstick + +# Probe point writer class name. +# BENCHMARK_WRITER= + +# Comma-separated list of the hosts to run BenchmarkServers on. 2 nodes on local host are enabled by default. +SERVER_HOSTS=localhost,localhost + +# Comma-separated list of the hosts to run BenchmarkDrivers on. 1 node on local host is enabled by default. +DRIVER_HOSTS=localhost + +# Remote username. +# REMOTE_USER= + +# Number of nodes, used to wait for the specified number of nodes to start. +nodesNum=$((`echo ${SERVER_HOSTS} | tr ',' '\n' | wc -l` + `echo ${DRIVER_HOSTS} | tr ',' '\n' | wc -l`)) + +# Run configuration. +# Note that each benchmark is set to run for 300 seconds (5 mins) with warm-up set to 60 seconds (1 minute). +CONFIGS="\ +-cfg ${SCRIPT_DIR}/../config/ignite-localhost-config.xml -nn ${nodesNum} -b 1 -w 60 -d 300 -t 64 -sm PRIMARY_SYNC -dn IgnitePutBenchmark -sn IgniteNode -ds atomic-put-1-backup,\ +" http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/pom.xml ---------------------------------------------------------------------- diff --git a/modules/yardstick/pom.xml b/modules/yardstick/pom.xml index 4c4d138..d96fcc6 100644 --- a/modules/yardstick/pom.xml +++ b/modules/yardstick/pom.xml @@ -35,7 +35,7 @@ http://ignite.apache.org - 0.8.0 + 0.8.2 4.1.0.RELEASE @@ -56,7 +56,7 @@ org.apache.ignite ignite-indexing ${project.version} - + org.apache.ignite http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 790cd4b..ea76053 100644 --- a/pom.xml +++ b/pom.xml @@ -506,6 +506,7 @@ +