hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nkeywal (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-7246) Assignment#nodeChildrenChanged calls listChildrenAndWatchForNewChildren, overloading master & zookeper needlessly
Date Fri, 30 Nov 2012 17:41:58 GMT
nkeywal created HBASE-7246:

             Summary: Assignment#nodeChildrenChanged calls listChildrenAndWatchForNewChildren,
overloading master & zookeper needlessly
                 Key: HBASE-7246
                 URL: https://issues.apache.org/jira/browse/HBASE-7246
             Project: HBase
          Issue Type: Improvement
          Components: master, Region Assignment
    Affects Versions: 0.96.0
            Reporter: nkeywal

Assignment#nodeChildrenChanged listens to change event on hbase/unassigned, then calls listChildrenAndWatchForNewChildren.
As its name says, this function gets the whole list of unnasigned regions from ZK.

The internal comments says it's only for split creation. I tried to remove it, it seems to

The point is:
- ZK does not allow us to know upfront if it's a split or not
- 50% of the events will be deletions, we don't care about them but we can't know in advance
- for large assignment, we will have a lot of children (hence events). Common practise in
ZK is to limit the number of subnodes (see the threads http://www.mail-archive.com/zookeeper-user@hadoop.apache.org/msg01954.html)

Monitoring the master, it was spending 10% of it's time on this during assignment. It seems
it was not on the critical path (yet...), so removing it did not change the overall performances
on a test scenario. In real life however, this typically adds workload to ZK, and ZK workload
is precious on large failures... Same as master actually.

If it's used only for split, we should have a znode specific for split and not a global znode.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message