hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
Date Tue, 08 Oct 2013 19:46:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789598#comment-13789598

Sergey Shelukhin commented on HBASE-5487:

Wrt coprocs - that is bad imho, that is not the kind of modular that we want. Core parts of
the system should depend on well-defined interfaces, not a generic extension points. Imho,
the litmus test for coproc, as a plugin interface, is - can you run HBase without it? If yes,
then it's ok to be a coproc (e.g. accesscontrol). Otherwise we should have proper interfaces
that have some meaning to the caller.

bq. Our master is only complex because of our use of zk to hold and mutate state.
That is not due to ZK as such, that is due to multi-state-machine reconciliation model and
truth in multiple places that it requires.
System table can have exact same problem of state in the table + state in memory, question
is how you split and manage state between them, storage substrate doesn't matter as much.
If truth was in ZK and nowhere else that wouldn't be a problem, same way as with system table.
Also, by reliable I meant that ZK is multiple nodes with built-in master recovery by design,
whereas with master you need at least HA, and still it's probably worse than ZK in case of
There are also other things that I mentioned.

bq. With the meta/system wal I think we can be within an order of magnitude.
So, why would we write a bunch of new code to get "within an order of magnitude"? I don't
see an advantage, or ZK disadvantage that you mention compared to multiple advantages of ZK.
Esp. if we cannot totally get rid of it, so we'll have an extra service regardless.

> Generic framework for Master-coordinated tasks
> ----------------------------------------------
>                 Key: HBASE-5487
>                 URL: https://issues.apache.org/jira/browse/HBASE-5487
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver, Zookeeper
>    Affects Versions: 0.94.0
>            Reporter: Mubarak Seyed
>            Priority: Critical
>         Attachments: Region management in Master.pdf
> Need a framework to execute master-coordinated tasks in a fault-tolerant manner. 
> Master-coordinated tasks such as online-scheme change and delete-range (deleting region(s)
based on start/end key) can make use of this framework.
> The advantages of framework are
> 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for master-coordinated
> 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK
> 3. Easy to plugin new master-coordinated tasks without adding code to core components

This message was sent by Atlassian JIRA

View raw message