Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 6 Sep 2016 18:06:20 +0000 (UTC)
From: "Manoj Govindassamy (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12937730.1454976242000.499513.1473185180800@Atlassian.JIRA>
In-Reply-To: <JIRA.12937730.1454976242000@Atlassian.JIRA>
References: <JIRA.12937730.1454976242000@Atlassian.JIRA> <JIRA.12937730.1454976242349@arcas>
Subject: [jira] [Commented] (HDFS-9781) FsDatasetImpl#getBlockReports can
 occasionally throw NullPointerException
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 06 Sep 2016 18:06:22 -0000


    [ https://issues.apache.org/jira/browse/HDFS-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15468073#comment-15468073 ] 

Manoj Govindassamy commented on HDFS-9781:
------------------------------------------

[~arpitagarwal], Thanks for looking at test failures. would like to understand more on these test failure signatures. 

The fix in this bug HDFS-9781 does the following
1. fixes NPE issue in getBlockReports()
2. test case TestFsDatasetImpl#testRemoveVolumeBeingWritten() updated to verify both HDFS-9781 (NPE) and HDFS-10830 (IllegalMonitorStateException) issues 

So I believe, even if you back out HDFS-9781 NPE fix, you will continue to see the other problem exposed from the updated testcase, until the wait in removeVolumes() is fixed. 

Can you please tell me how the test case fails when you backed out all other issues. Is that only by IllegalMonitorStateException or by other errors.

> FsDatasetImpl#getBlockReports can occasionally throw NullPointerException
> -------------------------------------------------------------------------
>
>                 Key: HDFS-9781
>                 URL: https://issues.apache.org/jira/browse/HDFS-9781
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 3.0.0-alpha1
>         Environment: Jenkins
>            Reporter: Wei-Chiu Chuang
>            Assignee: Manoj Govindassamy
>             Fix For: 3.0.0-alpha2
>
>         Attachments: HDFS-9781-branch-2.001.patch, HDFS-9781.002.patch, HDFS-9781.003.patch, HDFS-9781.01.patch, HDFS-9781.branch-2.001.patch
>
>
> FsDatasetImpl#getBlockReports occasionally throws NPE. The NPE caused TestFsDatasetImpl#testRemoveVolumeBeingWritten to time out, because the test waits for the call to FsDatasetImpl#getBlockReports to complete without exceptions.
> Additionally, the test should be updated to identify an expected exception, using {{GenericTestUtils.assertExceptionContains()}}
> {noformat}
> Exception in thread "Thread-20" java.lang.NullPointerException
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.getBlockReports(FsDatasetImpl.java:1709)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl$1BlockReportThread.run(TestFsDatasetImpl.java:587)
> 2016-02-08 15:47:30,379 [Thread-21] WARN  impl.TestFsDatasetImpl (TestFsDatasetImpl.java:run(606)) - Exception caught. This should not affect the test
> java.io.IOException: Failed to move meta file for ReplicaBeingWritten, blk_0_0, RBW
>   getNumBytes()     = 0
>   getBytesOnDisk()  = 0
>   getVisibleLength()= 0
>   getVolume()       = /home/weichiu/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/Nmi6rYndvr/data0/current
>   getBlockFile()    = /home/weichiu/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/Nmi6rYndvr/data0/current/bpid-0/current/rbw/blk_0
>   bytesAcked=0
>   bytesOnDisk=0 from /home/weichiu/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/Nmi6rYndvr/data0/current/bpid-0/current/rbw/blk_0_0.meta to /home/weichiu/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/Nmi6rYndvr/data0/current/bpid-0/current/finalized/subdir0/subdir0/blk_0_0.meta
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:857)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addFinalizedBlock(BlockPoolSlice.java:295)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addFinalizedBlock(FsVolumeImpl.java:819)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeReplica(FsDatasetImpl.java:1620)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeBlock(FsDatasetImpl.java:1601)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl$1ResponderThread.run(TestFsDatasetImpl.java:603)
> Caused by: java.io.IOException: renameTo(src=/home/weichiu/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/Nmi6rYndvr/data0/current/bpid-0/current/rbw/blk_0_0.meta, dst=/home/weichiu/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/Nmi6rYndvr/data0/current/bpid-0/current/finalized/subdir0/subdir0/blk_0_0.meta) failed.
>         at org.apache.hadoop.io.nativeio.NativeIO.renameTo(NativeIO.java:873)
>         at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:855)
>         ... 5 more
> 2016-02-08 15:47:34,381 [Thread-19] INFO  impl.FsDatasetImpl (FsVolumeList.java:waitVolumeRemoved(287)) - Volume reference is released.
> 2016-02-08 15:47:34,384 [Thread-19] INFO  impl.TestFsDatasetImpl (TestFsDatasetImpl.java:testRemoveVolumeBeingWritten(622)) - Volumes removed
> {noformat}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org