incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BLUR-40) Test cases failing in BlurClusterTest due to invalid directory permissions
Date Fri, 26 Oct 2012 13:15:12 GMT

    [ https://issues.apache.org/jira/browse/BLUR-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484903#comment-13484903
] 

Aaron McCurry commented on BLUR-40:
-----------------------------------

I have run into an issue.  Running the default setup in BlurClusterTest, however changing
it to 755 works as expected.  I feel like we should come up with a different solution (or
enhance this one) so that devs like me don't have to modify code to run the tests.

Starting DataNode 0 with dfs.data.dir: /tmp/cluster/dfs/data/data1,/tmp/cluster/dfs/data/data2
ERROR 20121026_09:01:21:021_EDT [main] datanode.DataNode: All directories in dfs.data.dir
are invalid.
ERROR 20121026_09:01:21:021_EDT [main] datanode.DataNode: All directories in dfs.data.dir
are invalid.
ERROR 20121026_09:01:21:021_EDT [main] blur.MiniCluster: error opening file system
java.lang.NullPointerException
	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
	at org.apache.blur.MiniCluster.startDfs(MiniCluster.java:435)
	at org.apache.blur.MiniCluster.startDfs(MiniCluster.java:420)
	at org.apache.blur.MiniCluster.startDfs(MiniCluster.java:416)
	at org.apache.blur.MiniCluster.startBlurCluster(MiniCluster.java:129)
	at org.apache.blur.thrift.BlurClusterTest.startCluster(BlurClusterTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)


It appears to me that the MiniDfsCluster does not obey the "dfs.datanode.data.dir.perm" permission,
in MiniDfsCluster method startDataNodes, this is how it creates those sub dirs:

    for (int i = curDatanodesNum; i < curDatanodesNum+numDataNodes; i++) {
      Configuration dnConf = new Configuration(conf);
      if (manageDfsDirs) {
        File dir1 = new File(data_dir, "data"+(2*i+1));
        File dir2 = new File(data_dir, "data"+(2*i+2));
        dir1.mkdirs();
        dir2.mkdirs();
        if (!dir1.isDirectory() || !dir2.isDirectory()) { 
          throw new IOException("Mkdirs failed to create directory for DataNode "
                                + i + ": " + dir1 + " or " + dir2);
        }
        dnConf.set(DataNode.DATA_DIR_KEY, dir1.getPath() + "," + dir2.getPath());
      }

No permissions are set, but they checked during startup.  If no valid directories are found
the datanode object return as null, hence the NPE.
                
> Test cases failing in BlurClusterTest due to invalid directory permissions
> --------------------------------------------------------------------------
>
>                 Key: BLUR-40
>                 URL: https://issues.apache.org/jira/browse/BLUR-40
>             Project: Apache Blur
>          Issue Type: Bug
>            Reporter: Gagan Deep Juneja
>         Attachments: 0001-BLUR-ID-40-fix-failing-tests.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message