hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2430) The number of failed or low-resource volumes the NN can tolerate should be configurable
Date Wed, 30 Nov 2011 22:37:40 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160409#comment-13160409

Aaron T. Myers commented on HDFS-2430:

Thanks a lot for the review, Eli. Comments below.

The api for NNResourcePolicy#areResourcesAvailable is a little strange, if a required resource
is not available we return false, but if there are not enough redundant resources we throw
a RTE. Do callers need to distinguish between these two cases? If not, we should return false
in both cases.
If so, there should be two separate apis. I'd make the method name better reflect the purpose
in the comment (whether we can log edits), the method is generic but the particular policy
is edits-specific.

It only throws an RTE if the *configured* number of redundant resources is less than the configured
minimum number of redundant resources. i.e. it throws only in the case of misconfiguration.
If indeed the number of configured redundant resources is greater than the minimum, but the
number of available redundant resources is below the minimum, the method will return false.
Does that seem reasonable?

bq. Does the dir checking use canonical paths so we're sure the checks pass in case of symlinks?

Yes indeed. {{FSNS#getStorageDirs}} ends up calling {{o.a.h.hdfs.server.common.Util#fileAsURI}}
which calls {{File#getCanonicalFile}}.

I'll upload a patch in a moment which addresses the nits.
> The number of failed or low-resource volumes the NN can tolerate should be configurable
> ---------------------------------------------------------------------------------------
>                 Key: HDFS-2430
>                 URL: https://issues.apache.org/jira/browse/HDFS-2430
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: name-node
>    Affects Versions: 0.24.0
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>         Attachments: HDFS-2430.patch, HDFS-2430.patch, HDFS-2430.patch
> Currently the number of failed or low-resource volumes the NN can tolerate is effectively
hard-coded at 1. It would be nice if this were configurable.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message