hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/HBaseUseCases" by PatrickHunt
Date Thu, 05 Nov 2009 20:13:21 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/HBaseUseCases" page has been changed by PatrickHunt.
http://wiki.apache.org/hadoop/ZooKeeper/HBaseUseCases?action=diff&rev1=7&rev2=8

--------------------------------------------------

  
  See [[http://bit.ly/4ekN8G|this perf doc]] for some ideas, 20 clients doing 50k watches
each - 1 million watches on a single core standalone server and still << 5ms avg response
time (async ops, keep that in mind re implementation time) YMMV of course but your numbers
are well below this. 
  
- Worst-case scenarion -- cascade if all RS become disconnected
+ Worst-case scenarios -- say a cascade failure where all RS become disconnected and sessions
expire
+  # master will get notified of 1000 RS gone
+  # master will delete all nodes in 2) - 1000 RS znodes, 100 regions each RS znode, 1 state
(typ) each of the 100 reg
+   * 200k nodes deleted (hint: use async) for each RS
+ 
+ Another worst case:
+  # some set of RS are flapping (this may actually be much worse than just dying)
+   * consider running some sort of health check on RS before assigning work, in case it just
dies 
+   * or, slowly ramp up the number of regions assigned to the RS, allow it to prove itself
vs dumping a number of regions on it and then have it flap... (donno enough about hbase to
comment resonably, but thing about something like this)
+  # for each RS master is deleting 200 znodes
  
  [PDH end]
  

Mime
View raw message