hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
Date Tue, 26 Mar 2013 19:57:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614484#comment-13614484

Jonathan Hsieh commented on HBASE-5487:

Thanks for writing this up.  I read the first two sections and haven't spent time reading
the details of the others yet.  (already have a bunch of quetsions).

So the only problem is that the assignment manager and ssh is difficult to reason about? 
Is the assignment manager the only "master coordinated" task in scope? 

I think there are more problems than that that we should enumerate:
* Instead of asserting it is not clear if table (+region) locks scale, let's find out.  
* Master operations and processes can clash and we should understand where we need concurrency
control.  (I'm working on a table -- here's an draft distilled version [1], there exists an
overly detailed version that I'll share once i get it fixed)
* Should there be a notion of queuing operations?  (locking, or an actual queue) Should these
operations be generically logged so they can complete if a master goes down in the middle?
(ex: master goes down during a "move" operation after the close but before the open on the
new rs).

The "design principles" is actually more of a proposed design.  

Design principles::region record
* how do we deal with operations where we need "locks" on multiple region because we are reading
or modifying multiple regions -- e.g. splits, merges, snapshots?  [~mbertozzi] had suggested
in another jira making a the meta row per table, or maybe part of the solution is using the
multi-row single meta region transaction.

What are alternatives?  why this approach vs others? 

[1]  https://docs.google.com/spreadsheet/ccc?key=0AiVCAt6zRttFdDRVaG56RnpQZlNNZUNsRVJsSXM3YlE&usp=sharing
> Generic framework for Master-coordinated tasks
> ----------------------------------------------
>                 Key: HBASE-5487
>                 URL: https://issues.apache.org/jira/browse/HBASE-5487
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver, Zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>         Attachments: Region management in Master.pdf
> Need a framework to execute master-coordinated tasks in a fault-tolerant manner. 
> Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s)
based on start/end key) can make use of this framework.
> The advantages of framework are
> 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated
> 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK
> 3. Easy to plugin new master-coordinated tasks without adding code to core components

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message