hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
Date Tue, 26 Mar 2013 19:57:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614484#comment-13614484
] 

Jonathan Hsieh commented on HBASE-5487:
---------------------------------------

Thanks for writing this up.  I read the first two sections and haven't spent time reading
the details of the others yet.  (already have a bunch of quetsions).

So the only problem is that the assignment manager and ssh is difficult to reason about? 
Is the assignment manager the only "master coordinated" task in scope? 

I think there are more problems than that that we should enumerate:
* Instead of asserting it is not clear if table (+region) locks scale, let's find out.  
* Master operations and processes can clash and we should understand where we need concurrency
control.  (I'm working on a table -- here's an draft distilled version [1], there exists an
overly detailed version that I'll share once i get it fixed)
* Should there be a notion of queuing operations?  (locking, or an actual queue) Should these
operations be generically logged so they can complete if a master goes down in the middle?
(ex: master goes down during a "move" operation after the close but before the open on the
new rs).

The "design principles" is actually more of a proposed design.  

Design principles::region record
* how do we deal with operations where we need "locks" on multiple region because we are reading
or modifying multiple regions -- e.g. splits, merges, snapshots?  [~mbertozzi] had suggested
in another jira making a the meta row per table, or maybe part of the solution is using the
multi-row single meta region transaction.

What are alternatives?  why this approach vs others? 

[1]  https://docs.google.com/spreadsheet/ccc?key=0AiVCAt6zRttFdDRVaG56RnpQZlNNZUNsRVJsSXM3YlE&usp=sharing
                
> Generic framework for Master-coordinated tasks
> ----------------------------------------------
>
>                 Key: HBASE-5487
>                 URL: https://issues.apache.org/jira/browse/HBASE-5487
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver, Zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>         Attachments: Region management in Master.pdf
>
>
> Need a framework to execute master-coordinated tasks in a fault-tolerant manner. 
> Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s)
based on start/end key) can make use of this framework.
> The advantages of framework are
> 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated
tasks
> 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK
> 3. Easy to plugin new master-coordinated tasks without adding code to core components

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message