hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
Date Tue, 15 Jan 2013 19:14:15 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13554176#comment-13554176
] 

Enis Soztutar commented on HBASE-5487:
--------------------------------------

bq. FATE would violate one of our "Invariants", no permanent data in zk:
I think it depends on the definition of permanent. FATE keeps the stack of operation states
in zk, but when the whole thing finishes, no zk data is left. You can still wipe out zk, if
you are in a clean state. 
bq. FATE's approach of serializing the steps along the way is easily replicated via WAL. Write
a step to the WAL to initiate, then write the next step, and the next, &c as you go. When
the whole thing is complete, write a completion record.
Agreed. 
bq. I think using the WAL is not the ideal mechanism; it makes master failover a very heavy-weight
operation
Not sure why it is so heavy weight. It requires to replay only the non-finished steps of the
operation of undo the operation, so that we can clean state. As in fate, we can divide every
operation (create table, disable, etc) as a sequence of steps, which are idempotent, and undo/redo'able.
Before the operation, and before every step master syncs the step to the WAL, after the step
is done, master does another sync to record a DONE state for that step. On replay, it just
buffers non-finished operations (ignores already finished ones), and undos/redos the steps
not finished. 
We can do size based and periodic log rolling, and for every non-finished operation keep the
smallest seqNum. When operation is done, we can just release that seqId, so that we can use
the same fshlog infrastructure. 
bq. An idea that Chris Trezzo just proposed (and I really like) is to move this to a 'system
level table'. 
I must have missed this. This will still be a WAL I guess, but kept as an hbase table. Considering
megastore also keeps its WAL in bigtable, 
I think it is doable.
bq. Why not write to a system table, (like META) using the existing wal? This question came
up for snapshot clone/restore recovery, and is a potential solution for maintaining table
enable/disable state (which is in zk and violates the rule)
I never liked the table enabled/disabled state, and agreed that it is a violation, as well
as a major cause of bugs. I guess, regardless of where we keep it (fshlog or system table),
it is clear that we need a WAL. 
                
> Generic framework for Master-coordinated tasks
> ----------------------------------------------
>
>                 Key: HBASE-5487
>                 URL: https://issues.apache.org/jira/browse/HBASE-5487
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver, Zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>            Assignee: Nick Dimiduk
>
> Need a framework to execute master-coordinated tasks in a fault-tolerant manner. 
> Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s)
based on start/end key) can make use of this framework.
> The advantages of framework are
> 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated
tasks
> 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK
> 3. Easy to plugin new master-coordinated tasks without adding code to core components

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message