hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10391) Always enable NameNode service RPC port
Date Tue, 26 Sep 2017 19:36:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181408#comment-16181408

Arpit Agarwal commented on HDFS-10391:

bq. So let me ask: What scenario leads you to believe a service port should be mandatory?
Here's one we see couple of times a month. Transient disk/network blips cause logSync to block
for a few seconds. The client RPC handlers are quickly exhausted waiting for logSync calls
to complete, and DNs start going stale. A more serious problem is that ZooKeeper FailoverControllers
cannot connect for service checks, so we see spurious NameNode failovers (most of our clusters
run with JN HA + automatic failover). Enabling the dedicated port allows the system to recover
without an unnecessary failover.

The incompatibility is minor and does not break rolling upgrade.

bq.  reconsidered after I added the async IBR processing which queues and batches IBRs.
I started looking through your patches and noticed HDFS-7964 which looks like it will address
the problem. I will look into that change more closely. It sounds like you no longer consider
the service RPC port necessary to keep the NameNode responsive.

> Always enable NameNode service RPC port
> ---------------------------------------
>                 Key: HDFS-10391
>                 URL: https://issues.apache.org/jira/browse/HDFS-10391
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, namenode
>            Reporter: Arpit Agarwal
>            Assignee: Gergely Novák
>              Labels: Incompatible
>         Attachments: HDFS-10391.001.patch, HDFS-10391.002.patch, HDFS-10391.003.patch,
HDFS-10391.004.patch, HDFS-10391.005.patch, HDFS-10391.006.patch, HDFS-10391.007.patch, HDFS-10391.008.patch,
HDFS-10391.009.patch, HDFS-10391.010.patch, HDFS-10391.v5-v6-delta.patch
> The NameNode should always be setup with a service RPC port so that it does not have
to be explicitly enabled by an administrator.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message