Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 11 Sep 2015 20:38:45 +0000 (UTC)
From: "Ted Yu (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12862030.1441469716000.318592.1442003925896@Atlassian.JIRA>
In-Reply-To: <JIRA.12862030.1441469716000@Atlassian.JIRA>
References: <JIRA.12862030.1441469716000@Atlassian.JIRA>
 <JIRA.12862030.1441469716226@arcas>
Subject: [jira] [Commented] (HBASE-14370) Use separate thread for calling
 ZKPermissionWatcher#refreshNodes()
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-14370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741503#comment-14741503 ] 

Ted Yu commented on HBASE-14370:
--------------------------------

For 0.98 QA run:
{code}
fht https://builds.apache.org/job/PreCommit-HBASE-Build/15567/console
Fetching the console output from the URL
Printing hanging tests
Hanging test : org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1
Hanging test : org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat
Printing Failing tests
{code}
The above hanging tests were not related to the patch.

Planning to commit in the near future.

> Use separate thread for calling ZKPermissionWatcher#refreshNodes()
> ------------------------------------------------------------------
>
>                 Key: HBASE-14370
>                 URL: https://issues.apache.org/jira/browse/HBASE-14370
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.0
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>             Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3
>
>         Attachments: 14370-0.98-v10.txt, 14370-branch-1-v10.txt, 14370-branch-1-v10.txt, 14370-v1.txt, 14370-v10.txt, 14370-v3.txt, 14370-v5.txt, 14370-v7.txt, 14370-v8.txt, 14370-wait-nofity-v2.txt, 14370-wait-nofity.txt, hbase-14370_v4.patch, test-acl3-branch-1.stack
>
>
> I came off a support case (0.98.0) where main zk thread was seen doing the following:
> {code}
>   at org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.refreshAuthManager(ZKPermissionWatcher.java:152)
>   at org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.refreshNodes(ZKPermissionWatcher.java:135)
>   at org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.nodeChildrenChanged(ZKPermissionWatcher.java:121)
>   at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:348)
>   at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
>   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> {code}
> There were 62000 nodes under /acl due to lack of fix from HBASE-12635, leading to slowness in table creation because zk notification for region offline was blocked by the above.
> The attached patch separates refreshNodes() call into its own thread.
> Thanks to Enis and Devaraj for offline discussion.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)