hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2
Date Tue, 17 May 2016 21:48:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287671#comment-15287671
] 

Mingliang Liu commented on HDFS-10415:
--------------------------------------

# If we call the {{initialize()}} method in the test, it will pass. Of course we have to do
this before mocking the {{dfsclient}} object.
{code:java|title=Solution 2 - calling initialize() explicitly}
   private static class MyDistributedFileSystem extends DistributedFileSystem {
     MyDistributedFileSystem() {
-      statistics = new FileSystem.Statistics("myhdfs"); // can't mock finals
+      initialize(new URI("hdfs://localhost"), new HdfsConfiguration()); // exception may
be thrown
       dfs = mock(DFSClient.class);
     }
{code}
# {{DistributedFileSystem#close()}} per se is not resilient to invocation prior to {{initialize()}}
being called. Meanwhile, the {{MyDistributedFileSystem}} also has to create a {{Statistics}}
object explicitly which is partial of what the {{initialize()}} method does. To me, this is
not ideal. It also has to trick out the {{deleteOnExit()}} by returning true for any path.
I'm more comfortable by simply using a real DFS object, and validating the order of implicit
operations when closing.
# Anyway, we can avoid calling {{initialize()}} method by constructing the {{storageStatistics}}
object in the {{MyDistributedFileSystem}} as following:
{code:title=Solution 3 - constructing the storageStatistics}
     MyDistributedFileSystem() {
       statistics = new FileSystem.Statistics("myhdfs"); // can't mock finals
+      storageStatistics = new DFSOpsCountStatistics(); // field needs to be protected
       dfs = mock(DFSClient.class);
     }
{code}


> TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2
> ------------------------------------------------------------------
>
>                 Key: HDFS-10415
>                 URL: https://issues.apache.org/jira/browse/HDFS-10415
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 2.9.0
>         Environment: jenkins
>            Reporter: Sangjin Lee
>            Assignee: Mingliang Liu
>         Attachments: HDFS-10415-branch-2.000.patch
>
>
> {noformat}
> Tests run: 24, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 51.096 sec <<<
FAILURE! - in org.apache.hadoop.hdfs.TestDistributedFileSystem
> testDFSCloseOrdering(org.apache.hadoop.hdfs.TestDistributedFileSystem)  Time elapsed:
0.045 sec  <<< ERROR!
> java.lang.NullPointerException: null
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:790)
> 	at org.apache.hadoop.fs.FileSystem.processDeleteOnExit(FileSystem.java:1417)
> 	at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:2084)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1187)
> 	at org.apache.hadoop.hdfs.TestDistributedFileSystem.testDFSCloseOrdering(TestDistributedFileSystem.java:217)
> {noformat}
> This is with Java 8 on Mac. It passes fine on trunk. I haven't tried other combinations.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message