hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-5487) Generic framework for Master-coordinated tasks
Date Wed, 16 Oct 2013 23:03:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13797395#comment-13797395
] 

Sergey Shelukhin edited comment on HBASE-5487 at 10/16/13 11:03 PM:
--------------------------------------------------------------------

You'd need some way to connect these "end" with what AM is doing, so AM will have to support
operations attached to its actions even if there's separate operation management. 
Moreover, you'd find out that these are not "steps", they are state goals.
For example, if you are disabling table, you want to close regions. So in case of separate
operation manager, you might create tasks to close all regions. But what if some server fails?
Now some of your regions are already closed. Separate operations to close region might fail
now, but the goal is achieved. If I start disabling table and then kill all RS-es, the table
is now disabled :) But all operations would fail.
State goals fit much more naturally in AM than "steps". I want to avoid steps as much as possible.

Stateful (as in, having separate state) multi-step operations are also hard to coordinate.
In the above example, during recovery, you don't want to reopen region if the table is disabling,
but you don't know until it's actually disabled if the table disable is an external operation.




was (Author: sershe):
You'd need some way to connect these "end" with what AM is doing, so AM will have to support
operations attached to its actions even if there's separate operation management. 
Moreover, you'd find out that these are not "steps", they are state goals.
For example, if you are disabling table, you want to close regions. So in case of separate
operation manager, you might create tasks to close all regions. But what if some server fails?
Now some of your regions are already closed. Separate operations to close region might fail
now, but the goal is achieved. If I start disabling table and then kill all RS-es, the table
is now disabled :) But all operations would fail.
State goals fit much more naturally in AM than "steps". I want to avoid steps as much as possible.

Stateful (as in, having separate step) multi-step operations are also hard to coordinate.
In the above example, during recovery, you don't want to reopen region if the table is disabling,
but you don't know until it's actually disabled if the table disable is an external operation.



> Generic framework for Master-coordinated tasks
> ----------------------------------------------
>
>                 Key: HBASE-5487
>                 URL: https://issues.apache.org/jira/browse/HBASE-5487
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver, Zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>         Attachments: Region management in Master5.docx, Region management in Master.pdf
>
>
> Need a framework to execute master-coordinated tasks in a fault-tolerant manner. 
> Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s)
based on start/end key) can make use of this framework.
> The advantages of framework are
> 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated
tasks
> 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK
> 3. Easy to plugin new master-coordinated tasks without adding code to core components



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message