accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bhava...@apache.org
Subject [4/6] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.1-SNAPSHOT
Date Thu, 26 Jun 2014 21:29:27 GMT
Merge branch '1.5.2-SNAPSHOT' into 1.6.1-SNAPSHOT

Conflicts:
	test/system/stress/README.md


Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2a17a7a2
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2a17a7a2
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2a17a7a2

Branch: refs/heads/master
Commit: 2a17a7a2b4eebdfb0a89f51b97adaab52904bc95
Parents: 934ca6f e01a720
Author: Bill Havanki <bhavanki@cloudera.com>
Authored: Thu Jun 26 17:29:00 2014 -0400
Committer: Bill Havanki <bhavanki@cloudera.com>
Committed: Thu Jun 26 17:29:00 2014 -0400

----------------------------------------------------------------------
 test/system/stress/README.md             | 95 +++++++++++++++++++++++----
 test/system/stress/reader.sh             | 42 ++++++------
 test/system/stress/readers               | 17 +++++
 test/system/stress/start-readers.sh      | 40 +++++++++++
 test/system/stress/start-writers.sh      | 40 +++++++++++
 test/system/stress/stop-readers.sh       | 36 ++++++++++
 test/system/stress/stop-writers.sh       | 36 ++++++++++
 test/system/stress/stress-env.sh.example | 56 ++++++++++++++++
 test/system/stress/writer.sh             | 44 ++++++-------
 test/system/stress/writers               | 17 +++++
 10 files changed, 367 insertions(+), 56 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/accumulo/blob/2a17a7a2/test/system/stress/README.md
----------------------------------------------------------------------
diff --cc test/system/stress/README.md
index 9546b21,e400d4b..9eb9fdc
--- a/test/system/stress/README.md
+++ b/test/system/stress/README.md
@@@ -15,20 -15,86 +15,91 @@@ See the License for the specific langua
  limitations under the License.
  -->
  
- 
  Accumulo Stress Test
- --------------------
+ ====================
+ 
+ This is an attempt to observe the behavior Accumulo displays when compacting and
+ reading cells. There are two components to this package:
+ `org.apache.accumulo.test.stress.random.Write` and
+ `org.apache.accumulo.test.stress.random.Scan`.
+ 
+ Since this test is meant to stress the system, there is no pre-defined test
+ period and no success criteria that we can programmatically check. During a
+ successful test period, the Accumulo cluster should remain stable and
+ functional. Possible failure conditions include component failures, such as
+ tablet servers running out of memory.
+ 
+ The `org.apache.accumulo.test.stress.random.Write` utility provides facilities
+ for writing random sized cells. Users can configure minimum and maximum sized
+ portions of a cell. The portions users can configure are the row, column family,
+ column qualifier and value. Note that the sizes are uniformly distributed
+ between the minimum and maximum values.
+ 
+ The `org.apache.accumulo.test.stress.random.Scan` utility provides users with
+ the ability to query tables generated by the Write. It will pick a tablet at
+ random and scan the entire range. The amount of times this process is performed
+ is user configurable. By default, it happens 1,024 times. Users can also specify
+ whether or not the scan should be isolated.
+ 
+ There is no shared state intended by either of these services. This allows
+ multiple clients to be run in parallel, either on the same host or distributed
+ across hosts.
+ 
+ ## Prerequisites
+ 
+ The utilities are run on a normal, initialized Accumulo cluster. By default,
+ they only work with a table named "stress_test".
+ 
+ The start and stop scripts rely on pssh. Before running them, you will need
+ to install pssh on the machines that will be controlled by them.
+ 
+ ## Running
+ 
+ Copy `stress-env.sh.example` to `stress-env.sh` and edit it, supplying at least
+ correct values for the USERPASS and INSTANCE variables. The remainder of this
+ file describes and provides defaults for many of the configuration options
+ for the stress utilities.
+ 
+ ### Individual Execution
+ 
+ Before starting a fresh write test, clear out the test table using the Accumulo
+ shell.
+ 
+     > deletetable -f stress_test
+     > createtable stress_test
+ 
+ To run a writer:
+ 
+     $ ./writer.sh
+ 
+ The writer will begin logging into a "logs" subdirectory, using timestamped
+ filenames. You can stop the writer by killing it or using Control-C.
+ 
+ To run a reader:
+ 
+     $ ./reader.sh
+ 
+ The reader logs like the writer. By default, the reader reads forever, but
+ it can be configured for a fixed number of scan iterations.
  
- This is an attempt to observe the behavior Accumulo displays when compacting and reading
these cells. There are two components to this package: `org.apache.accumulo.test.stress.random.Write`
and `org.apache.accumulo.test.stress.random.Scan`.
+ ### Group Execution
  
- Since this test is meant to stress the system, there is no pre-defined test period and no
success criteria that we can programmatically check. During a successful test period, the
Accumulo cluster should remain stable and functional. Possible failure conditions include
component failures, such as tablet servers running out of memory.
+ To run writers across multiple machines, create a writers file and list each
+ host that will run a writer, one per line.
  
- The `org.apache.accumulo.test.stress.random.Write` utility provides facilities for writing
random sized cells. Users can configure minimum and maximum sized portions of a cell. The
portions users can configure are the row, column family, column qualifier and value. Note
that the sizes are uniformly distributed between the minimum and maximum values
+ Finally, to start and stop the writers:
  
- The `org.apache.accumulo.test.stress.random.Scan` utility provides users with the ability
to query tables generated by the Write. It will pick a tablet at random and scan the entire
range. The amount of times this process is performed is user configurable. By default, it
happens 1,024 times. Users can also specify whether or not the scan should be isolated or
not.
+     $ ./start-writers.sh
+     $ ./stop-writers.sh
  
- There is no shared state intended by either of these services. This allows multiple clients
to be run in parallel, either on the same host or distributedacross hosts.
+ For readers, make a similar readers file, and use `start-readers.sh` and
+ `stop-readers.sh`.
  
- See the provided `test/system/stress/writer.sh` and `test/system/stress/reader.sh` scripts
for example configuration options and how to launch the processes.
+ The start scripts copy out stress-env.sh to each of the writers and readers,
+ overwriting what is there, so you only need to edit the file on the host from
+ which you run those scripts.
 +
- Note that in versions of Accumulo greater than 1.6.0, users should verify that the cell
sizes they supply meet the criteria set forth by the key constraints on their tables. By default,
tables are configured to reject keys that are are greater than 1MB.
++Note that in versions of Accumulo greater than 1.6.0, users should verify that
++the cell sizes they supply meet the criteria set forth by the key constraints
++on their tables. By default, tables are configured to reject keys that are
++greater than 1MB.


Mime
View raw message