hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1751) Intrinsic limits for HDFS files, directories
Date Fri, 01 Apr 2011 01:24:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014339#comment-13014339
] 

Jakob Homan commented on HDFS-1751:
-----------------------------------

Nicholas - I had sent you, Rob and Daryn an email letting you know I'd be out of town this
last week and apologizing for disappearing in the middle of the discussion.  I do hope you
got it.  Still catching up.

I think the disagreement at this point is between per-dir or globally defined.  Per-dir would
be more expensive, but would provide more utility than globally configured (I also think this
functionality may be subject to feature creep that per-dir would preempt).  A compromise would
be to go with globally configured now and if this does creep, to move its implementation to
per-dir later on (shoehorning in backwards compatibility).  I would be fine with that.

However, no one has responded to my concern about code duplication and we're-really-not-a-quota-even-though-we-are
concerns.  If globally configured can be done with a correct design of no code duplication
and cleanly integrated with the regular quota code (to avoid duplication and get better code
coverage), I'll be happy.  This is also more future proof if the need for per-dir arises.

> Intrinsic limits for HDFS files, directories
> --------------------------------------------
>
>                 Key: HDFS-1751
>                 URL: https://issues.apache.org/jira/browse/HDFS-1751
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node
>    Affects Versions: 0.22.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>             Fix For: 0.23.0
>
>         Attachments: HDFS-1751-2.patch, HDFS-1751-3.patch, HDFS-1751-4.patch, HDFS-1751.patch
>
>
> Enforce a configurable limit on:
>   the length of a path component
>   the number of names in a directory
> The intention is to prevent a too-long name or a too-full directory. This is not about
RPC buffers, the length of command lines, etc. There may be good reasons for those kinds of
limits, but that is not the intended scope of this feature. Consequently, a reasonable implementation
might be to extend the existing quota checker so that it faults the creation of a name that
violates the limits. This strategy of faulting new creation evades the problem of existing
names or directories that violate the limits.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message