hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-890) Have a way of creating datanodes that throws an meaningful exception on failure
Date Mon, 11 Jan 2010 22:56:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798912#action_12798912

Steve Loughran commented on HDFS-890:

Looking at the code, I don't think we need to change the method at all. Every other reason
for startup to fail, -bad configuration arguments, ports in use- cause {{makeInstance()}}
to fail with an exception, one that translates to a -1 exit code. 

It's only the no-valid-data dirs that causes a silent "no exception, no error code" failure,
one that 3 of 5 callers in the Hadoop codebase assume does not happen, callers who will NPE
if it does. 

The simplest solution here would just go: throw an exception in this situation, the main method
exits with -1; all the code that would have NPEd will now fail with meaningful errors. This
wouldn't need any new/deprecated interfaces, just a change to note "hdfs datanode fails with
-1 exit code if there is no directory"

We could also add a new Exception: NoDataDirsException extends IOException, includes a list
of data dirs and their individual exceptions. This could be used by callers to handle this
specific problem, if they were really interested in handling it differently. 

> Have a way of creating datanodes that throws an meaningful exception on failure
> -------------------------------------------------------------------------------
>                 Key: HDFS-890
>                 URL: https://issues.apache.org/jira/browse/HDFS-890
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>    Affects Versions: 0.22.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
> In HDFS-884, I proposed printing out more details on why things fail. This is hard to
test, because you need to subvert the log4j back end that your test harness will itself have
> There is a way to make it testable, and to make it easier for anyone creating datanodes
in process to recognise and handle failure: have a static CreateDatanode() method that throws
exceptions when directories cannot be created or other problems arise. Right now some problems
trigger failure, others just return a null reference saying "something went wrong but we won't
tell you what -hope you know where the logs go". 
> The HDFS-884 patch would be replaced by something that threw an exception; the existing
methods would catch this, log it and return null. The new method would pass it straight up.

> This is easier to test, better for others. If people think this is good, I will code
it up and mark the old API as deprecated. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message