hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3682) MiniDFSCluster#init should provide more info when it fails
Date Sat, 06 Oct 2012 00:46:03 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Lipcon updated HDFS-3682:
------------------------------

    Attachment: hdfs-3682.txt

Here's a patch which adds some error logging when the cluster fails to init. In order to test,
I chmodded my build directory to 000, and then ran one of the test cases. I got the following
in the test log:

2012-10-05 17:44:00,203 ERROR hdfs.MiniDFSCluster (MiniDFSCluster.java:initMiniDFSCluster(636))
- IOE creating namenodes. Permissions dump:
path '/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test/data/cluster5/data':

	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test/data/cluster5/data
	permissions: ----
path '/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test/data/cluster5':

	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test/data/cluster5
	permissions: ----
path '/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test/data': 
	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test/data
	permissions: ----
path '/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test': 
	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build/test
	permissions: ----
path '/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build': 
	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/build
	permissions: d---
path '/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs': 
	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs
	permissions: drwx
path '/home/todd/git/hadoop-common/hadoop-hdfs-project': 
	absolute:/home/todd/git/hadoop-common/hadoop-hdfs-project
	permissions: drwx
path '/home/todd/git/hadoop-common': 
	absolute:/home/todd/git/hadoop-common
	permissions: drwx
path '/home/todd/git': 
	absolute:/home/todd/git
	permissions: drwx
path '/home/todd': 
	absolute:/home/todd
	permissions: drwx
path '/home': 
	absolute:/home
	permissions: dr-x
path '/': 
	absolute:/
	permissions: dr-x

which would probably help diagnose.
                
> MiniDFSCluster#init should provide more info when it fails
> ----------------------------------------------------------
>
>                 Key: HDFS-3682
>                 URL: https://issues.apache.org/jira/browse/HDFS-3682
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 2.0.0-alpha
>            Reporter: Eli Collins
>            Assignee: Todd Lipcon
>            Priority: Minor
>              Labels: test-fail
>         Attachments: hdfs-3682.txt
>
>
> I've seen the occasional test failure due to the MiniDFSCluster init failing due to "java.io.IOException:
Cannot remove data directory" for the target/test/data/dfs/data directory. We should improve
the error handling here so we can better understand the failure, eg perhaps one of the tests
that chmods a directory fails and causes this failure.
> {noformat}
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:596)
> at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:297)
> at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:116)
> at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:279)
> at org.apache.hadoop.hdfs.security.TestDelegationToken.setUp(TestDelegationToken.java:77
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message