From hdfs-issues-return-268027-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Tue Jun 18 00:00:52 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 410A818062B for ; Tue, 18 Jun 2019 02:00:52 +0200 (CEST) Received: (qmail 95267 invoked by uid 500); 18 Jun 2019 00:00:51 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 95252 invoked by uid 99); 18 Jun 2019 00:00:51 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Jun 2019 00:00:51 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id DB03EE02E3 for ; Tue, 18 Jun 2019 00:00:50 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 9682A24617 for ; Tue, 18 Jun 2019 00:00:50 +0000 (UTC) Date: Tue, 18 Jun 2019 00:00:50 +0000 (UTC) From: "Hudson (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866082#comment-16866082 ] Hudson commented on HDFS-12914: ------------------------------- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16762 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16762/]) Revert "HDFS-12914. Addendum patch. Block report leases cause missing (weichiu: rev a50c35bb81105936dc0129b81f913e7307e306fc) * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java Revert "HDFS-12914. Block report leases cause missing blocks until next (weichiu: rev 7314185c4a313842115e18b5f42d118392cee929) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java HDFS-12914. Block report leases cause missing blocks until next report. (weichiu: rev 6822193ee6d6ac8b08822fa76c89e1dd61c5ddca) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java > Block report leases cause missing blocks until next report > ---------------------------------------------------------- > > Key: HDFS-12914 > URL: https://issues.apache.org/jira/browse/HDFS-12914 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.8.0, 2.9.2 > Reporter: Daryn Sharp > Assignee: Santosh Marella > Priority: Critical > Fix For: 3.3.0, 3.2.1 > > Attachments: HDFS-12914-branch-2.001.patch, HDFS-12914-trunk.00.patch, HDFS-12914-trunk.01.patch, HDFS-12914.005.patch, HDFS-12914.006.patch, HDFS-12914.007.patch, HDFS-12914.008.patch, HDFS-12914.009.patch, HDFS-12914.branch-3.2.patch, HDFS-12914.utfix.patch > > > {{BlockReportLeaseManager#checkLease}} will reject FBRs from DNs for conditions such as "unknown datanode", "not in pending set", "lease has expired", wrong lease id, etc. Lease rejection does not throw an exception. It returns false which bubbles up to {{NameNodeRpcServer#blockReport}} and interpreted as {{noStaleStorages}}. > A re-registering node whose FBR is rejected from an invalid lease becomes active with _no blocks_. A replication storm ensues possibly causing DNs to temporarily go dead (HDFS-12645), leading to more FBR lease rejections on re-registration. The cluster will have many "missing blocks" until the DNs next FBR is sent and/or forced. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org