hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeffrey Zhong (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-8368) Improve ZK notification handling in Master
Date Wed, 17 Apr 2013 22:33:16 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeffrey Zhong updated HBASE-8368:
---------------------------------

    Description: 
In ZooKeeperWatcher, we handle ZK notifications in a single queue(except AM) and each listener
is handling notifications synchronously. 

This cause potential performance issue if Master is handling an event which may jam the queue
so that ZooKeeperWatcher won't get new notifications in time. For example, we have several
region un-assigned notifications in pipeline and logsplitting or SSH handling etc handling
will be delayed.

I'm proposing:

During registerListeners, we can let a user to choose which queue current listener want to
register. For example, 
registerListener(QueueName, ZNodePathPrefix, listener);

Such as:
registerListener("unsigned_regions", "/hbase/unassigned", AMlistener);
registerListener("logSplitting", "/hbase/splitlog", LogSplittingManagerlistener);
...

For each queue, we use a single thread(consumer) to process events in order.

The ZNodePathPrefix is to pre-filter out notifications that current listener is only interested
in. 

For listeners without specifying a queue name, the listener will be put on a new "default"
queue.

Please let me know if you have better ideas!
Thanks.

  was:
In ZooKeeperWatcher, we handle ZK notifications in a single queue and each listener is handling
notifications synchronously. 

This cause potential performance issue if Master is handling an event which may jam the queue
so that ZooKeeperWatcher won't get new notifications in time. For example, we have several
region un-assigned notifications in pipeline and logsplitting or SSH handling etc handling
will be delayed.

I'm proposing:

During registerListeners, we can let a user to choose which queue current listener want to
register. For example, 
registerListener(QueueName, ZNodePathPrefix, listener);

Such as:
registerListener("unsigned_regions", "/hbase/unassigned", AMlistener);
registerListener("logSplitting", "/hbase/splitlog", LogSplittingManagerlistener);
...

For each queue, we use a single thread(consumer) to process events in order.

The ZNodePathPrefix is to pre-filter out notifications that current listener is only interested
in. 

For listeners without specifying a queue name, the listener will be put on a new "default"
queue.

Please let me know if you have better ideas!
Thanks.

    
> Improve ZK notification handling in Master
> ------------------------------------------
>
>                 Key: HBASE-8368
>                 URL: https://issues.apache.org/jira/browse/HBASE-8368
>             Project: HBase
>          Issue Type: Improvement
>          Components: master
>    Affects Versions: 0.95.0
>            Reporter: Jeffrey Zhong
>            Assignee: Jeffrey Zhong
>             Fix For: 0.98.0
>
>
> In ZooKeeperWatcher, we handle ZK notifications in a single queue(except AM) and each
listener is handling notifications synchronously. 
> This cause potential performance issue if Master is handling an event which may jam the
queue so that ZooKeeperWatcher won't get new notifications in time. For example, we have several
region un-assigned notifications in pipeline and logsplitting or SSH handling etc handling
will be delayed.
> I'm proposing:
> During registerListeners, we can let a user to choose which queue current listener want
to register. For example, 
> registerListener(QueueName, ZNodePathPrefix, listener);
> Such as:
> registerListener("unsigned_regions", "/hbase/unassigned", AMlistener);
> registerListener("logSplitting", "/hbase/splitlog", LogSplittingManagerlistener);
> ...
> For each queue, we use a single thread(consumer) to process events in order.
> The ZNodePathPrefix is to pre-filter out notifications that current listener is only
interested in. 
> For listeners without specifying a queue name, the listener will be put on a new "default"
queue.
> Please let me know if you have better ideas!
> Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message