hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4839) add NativeIO#mkdirs, that provides an error message on failure
Date Mon, 16 Dec 2013 01:49:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848747#comment-13848747

Colin Patrick McCabe commented on HDFS-4839:

Hi [~steve_l@iseran.com].  I agree with you that we can't change the behavior of {{FileSystem#mkdir}},
in {{LocalFileSystem}} or any other subclass.  I was not proposing that.  Sorry if it was
unclear.  I was proposing that we use a version of mkdirs which provides error information
internally in the HDFS {{NameNode}}, {{DataNode}}, etc.  Currently, we get very little information
about why something failed, and that makes debugging challenging.

For example, if the NameNode's {{Storage#clearDirectory}} fails with "Cannot create directory",
what can we tell users now?  Not very much.  It could be permissions, an I/O error, disk full,
or something else.  "I don't know, could be anything" is not something users like to hear
on a mailing list.

bq. If this can be done in Java7 then its not worth doing here, unless a goal is to backport
behaviour to java6 runtimes.

Well, as of now (December 2013), all versions of Hadoop must build on Java6.  It is a supported
platform.  In the past, we've added JNI or other workarounds to overcome deficiencies in Java6.
 I don't see why this time is any different, especially given the small amount of code we're
talking about here.

> add NativeIO#mkdirs, that provides an error message on failure
> --------------------------------------------------------------
>                 Key: HDFS-4839
>                 URL: https://issues.apache.org/jira/browse/HDFS-4839
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.1.0-beta
>            Reporter: Colin Patrick McCabe
>            Priority: Minor
> It would be nice to have a variant of mkdirs that provided an error message explaining
why it failed.  This would make it easier to debug certain failing unit tests that rely on
mkdir / mkdirs-- the ChecksumFilesystem tests, for example.

This message was sent by Atlassian JIRA

View raw message