hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Trivial Update of "Hbase/MasterRewrite" by stack
Date Tue, 20 Oct 2009 19:36:15 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hbase/MasterRewrite" page has been changed by stack.
http://wiki.apache.org/hadoop/Hbase/MasterRewrite?action=diff&rev1=7&rev2=8

--------------------------------------------------

  Initial Master Rewrite design came of conversations had at the hbase hackathon held at StumbleUpon,
August 5-7, 2009 ([[https://issues.apache.org/jira/secure/attachment/12418561/HBase+Hackathon+Notes+-+Sunday.pdf|Jon
Gray kept notes]]).  The umbrella issue for the master rewrite is [[https://issues.apache.org/jira/browse/HBASE-1816|HBASE-1816]].
 Timeline is hbase 0.20.1.
  
  == Table of Contents ==
-  * What does the Master do now?
+  * [[#now|What does the Master do now?]]
-  * Problems with current Master
+  * [[#problems|Problems with current Master]]
-  * Design
-   * 
+  * [[#design|Design]]
+   * [[#all|Move all region state transitions to zookeeper]]
+   * [[#distinct|In Zookeeper, a State and a Schema section]]
  
+ <<Anchor(now)>>
  == What does the Master do now? ==
  Here's a bit of a refresher on what Master currently does:
   * Region Assignment
@@ -22, +24 @@

    * Distributes out administered close, flush, compact messages
   * Watches ZK for its own lease and for regionservers so knows when to run recovery
  
+ 
+ <<Anchor(problems)>>
  == Problems with current Master ==
  There is a good list in the [[https://issues.apache.org/jira/secure/ManageLinks.jspa?id=12434794|Issue
Links]] section of HBASE-1816.
  	  
- 
+ <<Anchor(design)>>
  == Design ==
  
+ <<Anchor(all)>>
  === Move all region state transitions to zookeeper ===
  Run state transitions by changing state in zookeeper rather than inside in Master
  
+ <<Anchor(distinct)>>
  === In Zookeeper, a State and a Schema section ===
- 
- == Notes ==
- To be organized...
- 
- === Schema vs State ===
- Chatting w/ Berk, Ryan, and J-D:
- 
- Two locations in zk; one for schema and then one for state.  For example, could have hierarchy
in zk as follows:
+ Two locations in zk; one for schema and then one for state.  No connection.  For example,
could have hierarchy in zk as follows:
  
  {{{/hbase/tables/name/schema/{family1, family2}
  /hbase/tables/name/state/{attributes [read-only, enabled, nocompact, noflush]}
@@ -47, +46 @@

  /hbase/regionserver/to_open/{list of regions....}
  /hbase/regionservers/to_close/{list of regions...}
  
- Open of a region does not change state -- do not compact or rewrite anything -- in case
when we go back to zk to report open, its been opened elsewhere.
+ === State changes are clean, minimal, and comprehensive ===
+ Currently, moving a region from opening to open may involve a region compaction -- i.e.
a change to content in filesystem.  Better if modification of filesystem content was done
when no question of ownership involved.
  
- Need to come up with state transitions.  Eliminate fuzzy state.
+ In current o.a.h.h.master.RegionManager.RegionState inner class, here are possible states:
+ {{{    private volatile boolean unassigned = false;
+     private volatile boolean pendingOpen = false;
+     private volatile boolean open = false;
+     private volatile boolean closing = false;
+     private volatile boolean pendingClose = false;
+     private volatile boolean closed = false;
+     private volatile boolean offlined = false;}}}
+ 
  
  == More Notes ==
  

Mime
View raw message