ignite-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From voze...@apache.org
Subject [29/40] ignite git commit: IGNITE-4212 Ignite Benchmarking Simplification and Automation / IGNITE-4478 Fixing documentation and resolving lack of usability
Date Mon, 13 Feb 2017 11:30:09 GMT
IGNITE-4212 Ignite Benchmarking Simplification and Automation / IGNITE-4478 Fixing documentation
and resolving lack of usability


Project: http://git-wip-us.apache.org/repos/asf/ignite/repo
Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/605d946a
Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/605d946a
Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/605d946a

Branch: refs/heads/ignite-2.0
Commit: 605d946ada5ca4e562e256fa0edb29cc25dea7dc
Parents: e1c3dda
Author: oleg-ostanin <oostanin@gridgain.com>
Authored: Thu Feb 9 20:08:57 2017 +0300
Committer: Anton Vinogradov <av@apache.org>
Committed: Thu Feb 9 20:08:57 2017 +0300

----------------------------------------------------------------------
 modules/yardstick/DEVNOTES-standalone.txt       |  15 +++
 modules/yardstick/DEVNOTES.txt                  |  20 ++--
 modules/yardstick/README.txt                    | 119 +++++++++++++------
 .../config/benchmark-sample.properties          |  62 ++++++++++
 modules/yardstick/pom.xml                       |   4 +-
 pom.xml                                         |   1 +
 6 files changed, 172 insertions(+), 49 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/DEVNOTES-standalone.txt
----------------------------------------------------------------------
diff --git a/modules/yardstick/DEVNOTES-standalone.txt b/modules/yardstick/DEVNOTES-standalone.txt
new file mode 100644
index 0000000..d14dde2
--- /dev/null
+++ b/modules/yardstick/DEVNOTES-standalone.txt
@@ -0,0 +1,15 @@
+Building from standalone sources
+================================
+
+Run
+
+mvn clean package
+
+Artifacts can be found in `/target/assembly directory.`
+
+
+Writing Ignite Benchmarks
+=========================
+
+All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this
abstract class
+and implement `test` method. This is the method which actually tests performance.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/DEVNOTES.txt
----------------------------------------------------------------------
diff --git a/modules/yardstick/DEVNOTES.txt b/modules/yardstick/DEVNOTES.txt
index 6921243..b8b1f53 100644
--- a/modules/yardstick/DEVNOTES.txt
+++ b/modules/yardstick/DEVNOTES.txt
@@ -1,16 +1,20 @@
-Yardstick Ignite Maven Build Instructions
+Building from Ignite sources
 =========================================
 
-Yardstick can be build from standalone sources using following maven command:
+Run
 
-mvn clean package
+mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests
 
-Artifacts can be found in /target/assembly directory.
+in the Apache Ignite root directory.
 
-To build yardstick from Apache Ignite sources use:
+This command will compile the project and also unpack scripts from `yardstick-resources.zip`
file to
+`modules/yardstick/target/assembly/bin` directory.
 
-mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests
+Artifacts can be found in `modules/yardstick/target/assembly` directory.
+
+Writing Ignite Benchmarks
+=========================
 
-in Apache Ignite root directory
+All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this
abstract class
+and implement `test` method. This is the method which actually tests performance.
 
-Artifacts can be found in modules/yardstick/target/assembly directory.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/README.txt
----------------------------------------------------------------------
diff --git a/modules/yardstick/README.txt b/modules/yardstick/README.txt
index 353ddd8..dcfe0e6 100644
--- a/modules/yardstick/README.txt
+++ b/modules/yardstick/README.txt
@@ -1,55 +1,92 @@
 Yardstick Ignite Benchmarks
 ===========================
-Yardstick Ignite is a set of Ignite Grid (http://ignite.apache.org/) benchmarks written on
top of Yardstick framework.
+Apache Ignite benchmarks are written on top of Yardstick Framework (https://github.com/gridgain/yardstick)
and allow
+you to measure performance of various Apache Ignite components and modules.
 
-Yardstick Framework
-===================
-Visit Yardstick Repository (https://github.com/gridgain/yardstick) for detailed information
on how to run Yardstick benchmarks and how to generate graphs.
+The documentation below describes how to execute and configure already assembled benchmarks.
If you need to add new
+benchmarks or build existing one then refer to instruction from `DEVNOTES.txt` in source
directory.
 
-The documentation below describes configuration parameters in addition to standard Yardstick
parameters.
+Visit Yardstick Repository (https://github.com/gridgain/yardstick) for detailed information
in regards resulting graphs
+generation and how the frameworks works.
 
-Building from Ignite sources
-============
-1. Create a local clone of Ignite repository
-2. Run
 
-mvn package
+Running Ignite Benchmarks Locally
+==========================================
 
-command for Yardstick Ignite POM
+The simplest way to start with benchmarking is to use one of the executable scripts available
under `benchmarks\bin` directory:
 
-Building from standalone sources
-=====================
-Run
+./bin/benchmark-run-all.sh config/benchmark-atomic-put.properties
 
-mvn package
+The command above will benchmark the cache put operation for a distributed atomic cache.
The results of the benchmark will be added
+to auto-generated `results-{DATE-TIME}` directory.
 
-command for Yardstick Ignite POM
+If `./bin/benchmark-run-all.sh` command is executed as is without any parameters and modifications
in configurations files then
+all the available benchmarks will be executed on a local machine using `config/benchmark.properties`
configuration.
 
-Provided Benchmarks
-===================
-The following benchmarks are provided:
+To get more information about available benchmarks and configuration parameters refer to
“Provided Benchmarks” and
+“Properties And Command Line Arguments” sections below.
+
+
+Running Ignite Benchmarks Remotely
+=========================================
+
+For running Ignite benchmarks on remote hosts you need to upload Ignite-Yardstick to each
one of your remote hosts.
+
+NOTE: The path to the uploaded Ignite-Yardstick should be exactly the same on each host.
+
+Then you need to make some changes in config files:
 
-1. `GetBenchmark` - benchmarks atomic distributed cache get operation
-2. `PutBenchmark` - benchmarks atomic distributed cache put operation
-3. `PutGetBenchmark` - benchmarks atomic distributed cache put and get operations together
-4. `PutTxBenchmark` - benchmarks transactional distributed cache put operation
-5. `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get operations
together
-6. `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data
-7. `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over cached data
-8. `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous cache updates
+1. You need to comment or delete the
+        <property name="localHost" value="127.0.0.1"/>
+line in `config/ignite-localhost-config.xml` file.
 
-Writing Ignite Benchmarks
-=========================
-All benchmarks extend `AbstractBenchmark` class. A new benchmark should also extend this
abstract class
-and implement `test` method. This is the method that is actually benchmarked.
+2. You need to replace all the `127.0.0.1` addresses in `ignite-localhost-config.xml` file
by actual IPs of your remote
+servers. You can add or delete lines with IP addresses if you want to run benchmarks on different
number of hosts.
+There must be at least one IP address in the list.
 
-Running Ignite Benchmarks
-=========================
-Before running Ignite benchmarks, run:
+3. You need to replace the `localhost` strings by actual IP of your servers in the lines
+SERVERS='localhost,localhost'
+DRIVERS='localhost'
+in `config/benchmark-atomic-put.properties` file.
+
+Then use the following command:
+
+./bin/benchmark-run-all.sh config/benchmark-atomic-put.properties
+
+
+It is recommended to create some copies of original config files, edit these copies and then
use as a
+parameter for `./bin/benchmark-run-all.sh` script.
+
+
+Provided Benchmarks
+===================
+The following benchmarks are provided:
 
-mvn package
+1.  `GetBenchmark` - benchmarks atomic distributed cache get operation
+2.  `PutBenchmark` - benchmarks atomic distributed cache put operation
+3.  `PutGetBenchmark` - benchmarks atomic distributed cache put and get operations together
+4.  `PutTxBenchmark` - benchmarks transactional distributed cache put operation
+5.  `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get operations
together
+6.  `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data
+7.  `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over cached data
+8.  `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous cache updates
+9.  `AffinityCallBenchmark` - benchmarks affinity call operation
+10. `ApplyBenchmark` - benchmarks apply operation
+11. `BroadcastBenchmark` - benchmarks broadcast operations
+12. `ExecuteBenchmark` - benchmarks execute operations
+13. `RunBenchmark` - benchmarks running task operations
+14. `PutGetOffHeapBenchmark` - benchmarks atomic distributed cache put and get operations
together off heap
+15. `PutGetOffHeapValuesBenchmark` - benchmarks atomic distributed cache put value operations
off heap
+16. `PutOffHeapBenchmark` - benchmarks atomic distributed cache put operations off heap
+17. `PutOffHeapValuesBenchmark` - benchmarks atomic distributed cache get value operations
off heap
+18. `PutTxOffHeapBenchmark` - benchmarks transactional distributed cache put operation off
heap
+19. `PutTxOffHeapValuesBenchmark` - benchmarks transactional distributed cache put value
operation off heap
+20. `SqlQueryOffHeapBenchmark` -benchmarks distributed SQL query over cached data off heap
+21. `SqlQueryJoinOffHeapBenchmark` - benchmarks distributed SQL query with a Join over cached
data off heap
+22. `SqlQueryPutOffHeapBenchmark` - benchmarks distributed SQL query with simultaneous cache
updates off heap
+23. `PutAllBenchmark` - benchmarks atomic distributed cache batch put operation
+24. `PutAllTxBenchmark` - benchmarks transactional distributed cache batch put operation
 
-command. This command will compile the project and also will unpack scripts from `yardstick-resources.zip`
file to `bin` directory.
 
 Properties And Command Line Arguments
 =====================================
@@ -67,15 +104,19 @@ The following Ignite benchmark properties can be defined in the benchmark
config
 * `-cs` or `--cacheStore` - Enable or disable cache store readThrough, writeThrough
 * `-cl` or `--client` - Client flag
 * `-nc` or `--nearCache` - Near cache flag
-* `-nn <num>` or `--nodeNumber <num>` - Number of nodes (automatically set in
`benchmark.properties`), used to wait for the specified number of nodes to start
+* `-nn <num>` or `--nodeNumber <num>` - Number of nodes (automatically set in
`benchmark.properties`), used to wait for
+    the specified number of nodes to start
 * `-sm <mode>` or `-syncMode <mode>` - Synchronization mode (defined in `CacheWriteSynchronizationMode`)
 * `-ot` or `--offheapTiered` - Flag indicating whether tiered off-heap mode is on
 * `-ov` or `--offheapValuesOnly` - Flag indicating whether off-heap mode is on and only cache
values are stored off-heap
 * `-r <num>` or `--range` - Range of keys that are randomly generated for cache operations
+* `-rd or --restartdelay` - Restart delay in seconds
+* `-rs or --restartsleep` - Restart sleep in seconds
 * `-rth <host>` or `--restHost <host>` - REST TCP host
 * `-rtp <num>` or `--restPort <num>` - REST TCP port, indicates that a Ignite
node is ready to process Ignite Clients
 * `-ss` or `--syncSend` - Flag indicating whether synchronous send is used in `TcpCommunicationSpi`
-* `-txc <value>` or `--txConcurrency <value>` - Cache transaction concurrency
control, either `OPTIMISTIC` or `PESSIMISTIC` (defined in `CacheTxConcurrency`)
+* `-txc <value>` or `--txConcurrency <value>` - Cache transaction concurrency
control, either `OPTIMISTIC` or
+    `PESSIMISTIC` (defined in `CacheTxConcurrency`)
 * `-txi <value>` or `--txIsolation <value>` - Cache transaction isolation (defined
in `CacheTxIsolation`)
 * `-wb` or `--writeBehind` - Enable or disable writeBehind for cache store
 * `-wom <mode>` or `--writeOrderMode <mode>` - Write order mode for ATOMIC caches
(defined in `CacheAtomicWriteOrderMode`)
@@ -91,7 +132,7 @@ SERVER_HOSTS=localhost,localhost
 Note that -dn and -sn, which stand for data node and server node, are native Yardstick parameters
and are documented in Yardstick framework.
 ===========================================================================================================================================
 
-CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark -sn IgniteNode"
+CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark`IgniteNode"
 ```
 
 Issues

http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/config/benchmark-sample.properties
----------------------------------------------------------------------
diff --git a/modules/yardstick/config/benchmark-sample.properties b/modules/yardstick/config/benchmark-sample.properties
new file mode 100644
index 0000000..fee7cbb
--- /dev/null
+++ b/modules/yardstick/config/benchmark-sample.properties
@@ -0,0 +1,62 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Contains benchmarks for ATOMIC cache.
+#
+
+# JVM options.
+JVM_OPTS=${JVM_OPTS}" -DIGNITE_QUIET=false"
+
+# Uncomment to enable concurrent garbage collection (GC) if you encounter long GC pauses.
+# JVM_OPTS=${JVM_OPTS}" \
+#  -XX:+UseParNewGC \
+#  -XX:+UseConcMarkSweepGC \
+#  -XX:+UseTLAB \
+#  -XX:NewSize=128m \
+#  -XX:MaxNewSize=128m \
+#  -XX:MaxTenuringThreshold=0 \
+#  -XX:SurvivorRatio=1024 \
+#  -XX:+UseCMSInitiatingOccupancyOnly \
+#  -XX:CMSInitiatingOccupancyFraction=60 \
+#"
+
+# List of default probes.
+# Add DStatProbe or VmStatProbe if your OS supports it (e.g. if running on Linux).
+BENCHMARK_DEFAULT_PROBES=ThroughputLatencyProbe,PercentileProbe
+
+# Packages where the specified benchmark is searched by reflection mechanism.
+BENCHMARK_PACKAGES=org.yardstickframework,org.apache.ignite.yardstick
+
+# Probe point writer class name.
+# BENCHMARK_WRITER=
+
+# Comma-separated list of the hosts to run BenchmarkServers on. 2 nodes on local host are
enabled by default.
+SERVER_HOSTS=localhost,localhost
+
+# Comma-separated list of the hosts to run BenchmarkDrivers on. 1 node on local host is enabled
by default.
+DRIVER_HOSTS=localhost
+
+# Remote username.
+# REMOTE_USER=
+
+# Number of nodes, used to wait for the specified number of nodes to start.
+nodesNum=$((`echo ${SERVER_HOSTS} | tr ',' '\n' | wc -l` + `echo ${DRIVER_HOSTS} | tr ','
'\n' | wc -l`))
+
+# Run configuration.
+# Note that each benchmark is set to run for 300 seconds (5 mins) with warm-up set to 60
seconds (1 minute).
+CONFIGS="\
+-cfg ${SCRIPT_DIR}/../config/ignite-localhost-config.xml -nn ${nodesNum} -b 1 -w 60 -d 300
-t 64 -sm PRIMARY_SYNC -dn IgnitePutBenchmark -sn IgniteNode -ds atomic-put-1-backup,\
+"

http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/modules/yardstick/pom.xml
----------------------------------------------------------------------
diff --git a/modules/yardstick/pom.xml b/modules/yardstick/pom.xml
index 4c4d138..d96fcc6 100644
--- a/modules/yardstick/pom.xml
+++ b/modules/yardstick/pom.xml
@@ -35,7 +35,7 @@
     <url>http://ignite.apache.org</url>
 
     <properties>
-        <yardstick.version>0.8.0</yardstick.version>
+        <yardstick.version>0.8.2</yardstick.version>
         <spring.version>4.1.0.RELEASE</spring.version>
     </properties>
 
@@ -56,7 +56,7 @@
             <groupId>org.apache.ignite</groupId>
             <artifactId>ignite-indexing</artifactId>
             <version>${project.version}</version>
-          </dependency>
+        </dependency>
 
         <dependency>
             <groupId>org.apache.ignite</groupId>

http://git-wip-us.apache.org/repos/asf/ignite/blob/605d946a/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 790cd4b..ea76053 100644
--- a/pom.xml
+++ b/pom.xml
@@ -506,6 +506,7 @@
                                             <fileset dir="${basedir}/target/release-package/benchmarks/config/">
                                                 <include name="*.*" />
                                                 <exclude name="benchmark.properties"/>
+                                                <exclude name="benchmark-sample.properties"/>
                                                 <exclude name="benchmark-multicast.properties"/>
                                                 <exclude name="ignite-base-config.xml"/>
                                                 <exclude name="ignite-localhost-config.xml"/>


Mime
View raw message