hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/HBaseUseCases" by PatrickHunt
Date Thu, 05 Nov 2009 19:14:00 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/HBaseUseCases" page has been changed by PatrickHunt.
http://wiki.apache.org/hadoop/ZooKeeper/HBaseUseCases?action=diff&rev1=3&rev2=4

--------------------------------------------------

  == Near Future Usecases 0.21 HBase ==
  === Case 1 ===
  Summary: HBase Table State and Schema Changes
+ 
  Expected scale: Thousands of RegionServers watching ready to react to changes with about
100 tables each of which can have 1 or 2 states and an involved schema
  
+ [PDH] the link is very useful, let's do the math here so that it's more clear, correct where
I get it wrong. So if I get this right:
+  * 100 tables
+  * each table may have 1000+ regions (let's say 1000 for this calculation)
+  * 1 region server will carry a region from each table (per the link)
+  * if I understand correctly, region servers don't "own" the region znode in the table,
just watch tables for regions it carries
+ 
+ [PDH] So this means a region server watches each of 100 tables. 
+  * 100 * 1000 = 100k watches, as each region server watching 100 table nodes
+  * watches typically fire as a group/table (ie on/off/ro/drop each table)
+   * 1000 watches would fire notifying 1000 region servers each time a table changes
+ 
  General recipe implemented: A better description of problem and sketch of the solution can
be found at [[http://wiki.apache.org/hadoop/Hbase/MasterRewrite#tablestate|Master Rewrite:
Table State]]
+ 
+ [PDH] this is essentially "dynamic configuration" usecase - we are telling each region server
the state of the table containing a region it manages, when the master changes the state the
watchers are notified
  
  === Case 2 ===
  Summary: HBase Region Transitions from unassigned to open and from open to unassigned with
some intermediate states

Mime
View raw message