hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Chansler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1751) Intrinsic limits for HDFS files, directories
Date Tue, 22 Mar 2011 05:03:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009552#comment-13009552
] 

Robert Chansler commented on HDFS-1751:
---------------------------------------

First, I must ‘fess up that “intrinsic” was my suggestion. I understand Jakob’s point,
but defend my choice as well representing the intention that the limit be independent of any
particular directory or user. The whole point is avoid exhausting resources by foolish/mistaken
activity. And quota is a resource, which is why this facility needs to be independent. Testing
the limits (“intrinsic” or otherwise) at the same place as the quotas are tested make
sense as the same logic applies if the test fails. (The information returned to the client
should distinguish the faults.) But the intent would be defeated if there were anything like
per-directory configuration. Not to mention that the implementation would be more complex,
extra space would be consumed, and the complexity presented to users and administrators would
be increased. 

> Intrinsic limits for HDFS files, directories
> --------------------------------------------
>
>                 Key: HDFS-1751
>                 URL: https://issues.apache.org/jira/browse/HDFS-1751
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node
>    Affects Versions: 0.22.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>             Fix For: 0.23.0
>
>         Attachments: HDFS-1751-2.patch, HDFS-1751-3.patch, HDFS-1751-4.patch, HDFS-1751.patch
>
>
> Enforce a configurable limit on:
>   the length of a path component
>   the number of names in a directory
> The intention is to prevent a too-long name or a too-full directory. This is not about
RPC buffers, the length of command lines, etc. There may be good reasons for those kinds of
limits, but that is not the intended scope of this feature. Consequently, a reasonable implementation
might be to extend the existing quota checker so that it faults the creation of a name that
violates the limits. This strategy of faulting new creation evades the problem of existing
names or directories that violate the limits.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message