Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3672917AE0 for ; Tue, 3 Nov 2015 07:30:56 +0000 (UTC) Received: (qmail 78268 invoked by uid 500); 3 Nov 2015 07:12:28 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 78222 invoked by uid 500); 3 Nov 2015 07:12:28 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 78137 invoked by uid 99); 3 Nov 2015 07:12:27 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Nov 2015 07:12:27 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id BB8B72C1F6A for ; Tue, 3 Nov 2015 07:12:27 +0000 (UTC) Date: Tue, 3 Nov 2015 07:12:27 +0000 (UTC) From: "Hudson (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-9313) Possible NullPointerException in BlockManager if no excess replica can be chosen MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-9313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986819#comment-14986819 ] Hudson commented on HDFS-9313: ------------------------------ FAILURE: Integrated in Hadoop-Yarn-trunk #1354 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1354/]) HDFS-9313. Possible NullPointerException in BlockManager if no excess (mingma: rev d565480da2f646b40c3180e1ccb2935c9863dfef) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Possible NullPointerException in BlockManager if no excess replica can be chosen > -------------------------------------------------------------------------------- > > Key: HDFS-9313 > URL: https://issues.apache.org/jira/browse/HDFS-9313 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Ming Ma > Assignee: Ming Ma > Fix For: 2.8.0 > > Attachments: HDFS-9313-2.patch, HDFS-9313.patch > > > HDFS-8647 makes it easier to reason about various block placement scenarios. Here is one possible case where BlockManager won't be able to find the excess replica to delete: when storage policy changes around the same time balancer moves the block. When this happens, it will cause NullPointerException. > {noformat} > java.lang.NullPointerException > at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.adjustSetsWithChosenReplica(BlockPlacementPolicy.java:156) > at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseReplicasToDelete(BlockPlacementPolicyDefault.java:978) > {noformat} > Note that it isn't found in any production clusters. Instead, it is found from new unit tests. In addition, the issue has been there before HDFS-8647. -- This message was sent by Atlassian JIRA (v6.3.4#6332)