Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EE51118486 for ; Sat, 29 Aug 2015 01:06:46 +0000 (UTC) Received: (qmail 98564 invoked by uid 500); 29 Aug 2015 01:06:46 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 98510 invoked by uid 500); 29 Aug 2015 01:06:46 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 98497 invoked by uid 99); 29 Aug 2015 01:06:46 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 29 Aug 2015 01:06:46 +0000 Date: Sat, 29 Aug 2015 01:06:46 +0000 (UTC) From: "Hudson (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-8950) NameNode refresh doesn't remove DataNodes that are no longer in the allowed list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720866#comment-14720866 ] Hudson commented on HDFS-8950: ------------------------------ FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #319 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/319/]) HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list (Daniel Templeton) (cmccabe: rev b94b56806d3d6e04984e229b479f7ac15b62bbfa) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestHostFileManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommission.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java > NameNode refresh doesn't remove DataNodes that are no longer in the allowed list > -------------------------------------------------------------------------------- > > Key: HDFS-8950 > URL: https://issues.apache.org/jira/browse/HDFS-8950 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, namenode > Affects Versions: 2.6.0 > Reporter: Daniel Templeton > Assignee: Daniel Templeton > Fix For: 2.8.0 > > Attachments: HDFS-8950.001.patch, HDFS-8950.002.patch, HDFS-8950.003.patch, HDFS-8950.004.patch, HDFS-8950.005.patch > > > If you remove a DN from NN's allowed host list (HDFS was HA) and then do NN refresh, it doesn't remove it actually and the NN UI keeps showing that node. It may try to allocate some blocks to that DN as well during an MR job. This issue is independent from DN decommission. > To reproduce: > 1. Add a DN to dfs_hosts_allow > 2. Refresh NN > 3. Start DN. Now NN starts seeing DN. > 4. Stop DN > 5. Remove DN from dfs_hosts_allow > 6. Refresh NN -> NN is still reporting DN as being used by HDFS. > This is different from decom because there DN is added to exclude list in addition to being removed from allowed list, and in that case everything works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)