hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15155) Show All RPC handler tasks stops working after cluster is under heavy load for a while
Date Wed, 03 Feb 2016 22:14:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131245#comment-15131245
] 

Ted Yu commented on HBASE-15155:
--------------------------------

Tianying:
Can you submit patch for QA run ?

> Show All RPC handler tasks stops working after cluster is under heavy load for a while
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-15155
>                 URL: https://issues.apache.org/jira/browse/HBASE-15155
>             Project: HBase
>          Issue Type: Bug
>          Components: monitoring
>    Affects Versions: 0.98.0, 1.0.0, 0.94.19
>            Reporter: Tianying Chang
>            Assignee: Tianying Chang
>         Attachments: hbase-15155.patch
>
>
> After we upgrade from 94.7 to 94.26 and 1.0, we found that "Show All RPC handler status"
link on RS webUI stops working after running in production cluster with relatively high load
for several days.  
> Turn out to be it is a bug introduced by https://issues.apache.org/jira/browse/HBASE-10312
The BoundedFIFOBuffer cause RPCHandler Status overriden/removed permanently when there is
a spike of non-RPC tasks status that is over the MAX_SIZE (1000).  So as long as the RS experienced
"high" load once, the RPC status monitoring is gone forever, until RS is restarted. 
>  We added a unit test that can repro this. And the fix can pass the test.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message