hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-20303) RS RPC server should not allow the response queue size to be too large
Date Thu, 29 Mar 2018 08:25:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-20303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418595#comment-16418595

Duo Zhang commented on HBASE-20303:

How about SimpleRpcServer?  Do I need to do it? 

Can be another issue I think.

> RS RPC server should not allow the response queue size to be too large
> ----------------------------------------------------------------------
>                 Key: HBASE-20303
>                 URL: https://issues.apache.org/jira/browse/HBASE-20303
>             Project: HBase
>          Issue Type: Improvement
>          Components: rpc
>    Affects Versions: 3.0.0
>         Environment: 2000+ Region Servers
>            Reporter: Chance Li
>            Assignee: Chance Li
>            Priority: Major
>             Fix For: 3.0.0
>         Attachments: HBASE-20303.master.v1.patch
> With async clients, in some scenarios RS RPC server will occur Full GC because of the
large reponse queue. 
>  Netty provides a WriteBufferHighWaterMark on channel, But this doesn't meet RS's needs(Consider
5k ~ 10k sockets in one RS server, it will need 50G ~ 100G heap for 10M per channel).
> RS rpc server will add a gloabl response buffer water mark(2G by default). when reaching
the throttle, RS will not serve any request.
>  And RS rpc server will add a water mark for channel (100M by default), because it's
mostly possible that this client is abnomal.
> We created a unit test to simulate abnormal case: a client that has only 1 socket can
lead RS server to occupy heap up to 100M.
>  Notes:
>  1. For client compability, we still use existed exception(CALL_QUEUE_TOO_BIG_EXCEPTION)
but error message is different.
>  2. Not for SimpleRpcServer, because It's rarely used.

This message was sent by Atlassian JIRA

View raw message