hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8820) Enable RPC Congestion control by default
Date Tue, 03 Nov 2015 18:50:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987848#comment-14987848

Arpit Agarwal commented on HDFS-8820:

Thanks for the feedback [~mingma]. I didn't get a chance to look at this again until now.

bq. Thanks Arpit Agarwal. Should we enable this for communication between DN and NN?
Good point, this should not be enabled for the service RPC port. I'll fix that part of the

bq. For the configuration part, I wonder if we should use the pattern similar to RPC's setProtocolEngine,or
ipc.server.read.threadpool.size where NN or other services can call RPC.Builder#setnumReaders
to override the value. In that way, the NN doesn't need to know the format of the configuration
key name.
Agreed that would be best. However I am not sure changing the RPCEngine interface will be
a compatible change. 

> Enable RPC Congestion control by default
> ----------------------------------------
>                 Key: HDFS-8820
>                 URL: https://issues.apache.org/jira/browse/HDFS-8820
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch
> We propose enabling RPC congestion control introduced by HADOOP-10597 by default.
> We enabled it on a couple of large clusters a few weeks ago and it has helped keep the
namenodes responsive under load.

This message was sent by Atlassian JIRA

View raw message