hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Surendra Singh Lilhore (Jira)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-15067) Optimize heartbeat for large cluster
Date Thu, 09 Jan 2020 12:56:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-15067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Surendra Singh Lilhore updated HDFS-15067:
------------------------------------------
    Status: Patch Available  (was: Open)

> Optimize heartbeat for large cluster
> ------------------------------------
>
>                 Key: HDFS-15067
>                 URL: https://issues.apache.org/jira/browse/HDFS-15067
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 3.1.1
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>            Priority: Major
>         Attachments: HDFS-15067.01.patch, image-2020-01-09-18-00-49-556.png
>
>
> In a large cluster Namenode spend some time in processing heartbeats. For example, in
10K node cluster namenode process 10K RPC's for heartbeat in each 3sec. This will impact the
client response time. This heart beat can be optimized. DN can start skipping one heart beat
if no work(Write/replication/Delete) is allocated from long time. DN can start sending heart
beat in 6 sec. Once the DN stating getting work from NN , it can start sending heart beat
normally.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message