hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11747) ClusterStatus is too bulky
Date Wed, 01 Jul 2015 22:54:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611125#comment-14611125
] 

stack commented on HBASE-11747:
-------------------------------

bq. Did you mean rpc, not jmx?

JMX. Idea is to help shrink ClusterStatus by moving metrics out.

bq. I'm missing something, we don't really use it in any heardbeats

Right (I thought we did but it is just a sub-element, the ServerLoad, that is passed on the
heartbeat -- pardon me).

bq. "check if monolithic cluster status is looking too big (over defined limit) on server
side, and return it with empty load in this case, setting some flag indicating that message
is partially constructed to not fail as transport level, and that client should use separate
call to request server/region load for the list of RSs it's interested to know about?"

Rather than have a protocol that cuts in only when we are too big, could we not slim ClusterStatus
so vitals only and always require client use a separate call for detail (or go to metrics
system if it is counts, etc., that it is interested in)

I like your suggestion of adding a new call for doing new "protocol" That'd be best.

bq. Hmm, that's something different, drawing metrics in the UI?

Yes. Pardon my conflation. Will restrain myself in future.







> ClusterStatus is too bulky 
> ---------------------------
>
>                 Key: HBASE-11747
>                 URL: https://issues.apache.org/jira/browse/HBASE-11747
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Virag Kothari
>         Attachments: exceptiontrace
>
>
> Following exception on 0.98 with 1M regions on cluster with 160 region servers
> {code}
> Caused by: java.io.IOException: Call to regionserverhost:port failed on local exception:
com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large.  May be
malicious.  Use CodedInputStream.setSizeLimit() to increase the size limit.
> 	at org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
> 	at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
> 	at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
> 	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
> 	at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
> 	... 43 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
large.  May be malicious.  Use CodedInputStream.setSizeLimit() to increase the size limit.
> 	at com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message