hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Honghua (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
Date Mon, 14 Oct 2013 02:17:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793881#comment-13793881
] 

Feng Honghua commented on HBASE-5487:
-------------------------------------

[~jxiang]
bq. to the uncertainty due to ZK, I don't think it is because the way how we use it. It is
more because ZK doesn't support continuous events. You have to set the watch again after each
event callback. The problem is that after an event is triggered, when we try to get the data,
the data could be changed again so an event is missed that will cause state jump.
Agree. 'one-time watch' and 'asynchronous event notification' are the root cause of current
AM problem ( I mentioned in above comment, you can find it:-) ). And when I said 'because
the way we use it', I meant we use ZK's watch/event mechanism: A process(RS) updates ZK, and
B process(master) gets notified the update via watch event. If we use ZK just as a reliable
storage, just the way of using meta table, it makes no difference we use meta table or ZK
(except performance difference)
In the theme of using meta table, we adopt another communication pattern for tasks(assign/split/merge):
master requests RS to do something(and master stores the task progress/state to meta table),
RS responses master of its progress periodically, master changes the task progress in both
memory and meta table... ---under this theme we can use ZK to replace meta table, and avoid
previous state transition miss problem as well, since we don't use ZK's watch/event mechanism,
just using it as a reliable storage. right?
Just clarify, I think we share the same understanding of this problem, you can check my above
comments :-)

> Generic framework for Master-coordinated tasks
> ----------------------------------------------
>
>                 Key: HBASE-5487
>                 URL: https://issues.apache.org/jira/browse/HBASE-5487
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver, Zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>            Priority: Critical
>         Attachments: Region management in Master.pdf
>
>
> Need a framework to execute master-coordinated tasks in a fault-tolerant manner. 
> Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s)
based on start/end key) can make use of this framework.
> The advantages of framework are
> 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated
tasks
> 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK
> 3. Easy to plugin new master-coordinated tasks without adding code to core components



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message