hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/ServiceLatencyOverview" by PatrickHunt
Date Tue, 27 Oct 2009 20:38:42 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/ServiceLatencyOverview" page has been changed by PatrickHunt.
http://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview?action=diff&rev1=11&rev2=12

--------------------------------------------------

  
  All systems had 16gig memory available, however unless specifically noted the JVM's -Xmx
option was used to limit the size of the JVM heap to 512m.
  
- All systems had 7200RPM SATA drives.
+ All systems had 7200RPM SATA drives. hdparm reported:
+  * Timing cached reads:   22980 MB in  1.99 seconds = 11532.40 MB/sec
+  * Timing buffered disk reads:  266 MB in  3.01 seconds =  88.29 MB/sec
  
  All systems are interconnected with 1gig ethernet, approximate ping time is steady at 0.09ms
roundtrip.
  
@@ -45, +47 @@

   * zk-latencies.py from http://github.com/phunt/zk-smoketest was used as the test client.
   * python 2.4.3
  
+ == Results ==
+ 
+ the results are as follows for standalone and 3 node ensemble
+ 
+ Note: in each case below I give only a single client's results, regardless of the number
of clients run in the tests. I do this to limit the length of this document, the results that
I do show are for the last, longest running client, so in general these are the worst performing
results of any client in the test. I also verify that all clients completed successfully and
without warnings or errors in the client log.
+ 
+ == Results for Standalone Server ==
+ 
+ The Standalone version of ZooKeeper was run on a single host ("host60").
+ 
- The client was run as follows unless otherwise noted:
+ The client(s) was run as follows unless otherwise noted:
  
  {{{
- zk-latencies.py --servers "host60:2181" --root_znode=/`hostname` --znode_size=100 --znode_count=10000
--timeout=5000 --watch_multiple=5
+ zk-latencies.py --cluster "host60:2181" --root_znode=/`hostname` --znode_size=100 --znode_count=10000
--timeout=5000 --watch_multiple=5
  }}}
  
  This means each client:
@@ -57, +69 @@

   * sets 5 watches on each of the znodes it creates, for 50k znodes by each client
  
  As a result, each client will send ~110,000 requests to the server during each test.
- 
- == Results ==
- 
- the results are as follows for standalone and 3 node ensemble
- 
- Note: in each case below I give only a single client's results, regardless of the number
of clients run in the tests. I do this to limit the length of this document, the results that
I do show are for the last, longest running client, so in general these are the worst performing
results of any client in the test. I also verify that all clients completed successfully and
without warnings or errors in the client log.
- 
- == Results for Standalone Server ==
  
  === 1 Core - 1 Client ===
  
@@ -237, +241 @@

  host65: 0inputs+0outputs (0major+75539minor)pagefaults 0swaps
  }}}
  
+ == Results for 3 Server Ensemble ==
+ 
+ A 3 Server Ensemble (3 distinct hosts) was used for this test ("host60", "host62" &
"host63").
+ 
+ The client was run as follows unless otherwise noted:
+ 
+ {{{
+ zk-latencies.py --cluster "host60:2181,host61:2181,host63:2181" --root_znode=/`hostname`
--znode_size=100 --znode_count=10000 --timeout=5000 --watch_multiple=5
+ }}}
+ 
+ This means each client:
+  * has a 5 second timeout
+  * creates 10k znodes of size 100 bytes
+  * sets 5 watches on each of the znodes it creates, for 50k znodes by each client
+ 
+ As a result, each client will send ~110,000 requests to the server during each test.
+ 
+ === 1 Core - 1 Client ===
+ 
+ 110k operations against the server with elapsed time of 18 seconds
+ 
+ {{{
+ }}}
+ 
+ === 1 Core - 10 Client ===
+ 
+ 1.1 million operations against the server with elapsed time of 58 seconds
+ 
+ {{{
+ }}}
+ 
+ === 1 Core - 20 Client ===
+ 
+ 2.2 million operations against the server with elapsed time of 150 seconds
+ 
+ {{{
+ }}}
+ 
+ === 2 Core - 1 Client ===
+ 
+ 110k operations against the server with elapsed time of 20 seconds
+ 
+ {{{
+ }}}
+ 
+ === 2 Core - 10 Client ===
+ 
+ 1.1 million operations against the server with elapsed time of 53 seconds
+ 
+ {{{
+ }}}
+ 
+ === 2 Core - 20 Client ===
+ 
+ 2.2 million operations against the server with elapsed time of 108 seconds
+ 
+ {{{
+ }}}
+ 
+ === 4 Core - 1 Client ===
+ 
+ 110k operations against the server with elapsed time of 20 seconds
+ 
+ {{{
+ }}}
+ 
+ === 4 Core - 10 Client ===
+ 
+ 1.1 million operations against the server with elapsed time of 43 seconds
+ 
+ {{{
+ }}}
+ 
+ === 4 Core - 20 Client ===
+ 
+ 2.2 million operations against the server with elapsed time of 95 seconds
+ 
+ {{{
+ }}}
+ 

Mime
View raw message