From hdfs-issues-return-294384-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Jan 9 12:56:05 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id DC08618063F for ; Thu, 9 Jan 2020 13:56:04 +0100 (CET) Received: (qmail 13616 invoked by uid 500); 9 Jan 2020 12:56:04 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 13601 invoked by uid 99); 9 Jan 2020 12:56:04 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jan 2020 12:56:04 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 315A5E3160 for ; Thu, 9 Jan 2020 12:56:02 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 9C372780B1F for ; Thu, 9 Jan 2020 12:56:00 +0000 (UTC) Date: Thu, 9 Jan 2020 12:56:00 +0000 (UTC) From: "Surendra Singh Lilhore (Jira)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-15067) Optimize heartbeat for large cluster MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-15067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-15067: ------------------------------------------ Status: Patch Available (was: Open) > Optimize heartbeat for large cluster > ------------------------------------ > > Key: HDFS-15067 > URL: https://issues.apache.org/jira/browse/HDFS-15067 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode > Affects Versions: 3.1.1 > Reporter: Surendra Singh Lilhore > Assignee: Surendra Singh Lilhore > Priority: Major > Attachments: HDFS-15067.01.patch, image-2020-01-09-18-00-49-556.png > > > In a large cluster Namenode spend some time in processing heartbeats. For example, in 10K node cluster namenode process 10K RPC's for heartbeat in each 3sec. This will impact the client response time. This heart beat can be optimized. DN can start skipping one heart beat if no work(Write/replication/Delete) is allocated from long time. DN can start sending heart beat in 6 sec. Once the DN stating getting work from NN , it can start sending heart beat normally. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org