hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wendy Chien (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-438) DFS pathname limitation.
Date Mon, 11 Sep 2006 21:59:25 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-438?page=comments#action_12433992 ] 
Wendy Chien commented on HADOOP-438:

We've changed our plan.  Instead of enforcing the length in the Path constructor, we are going
to enforce it in mkdirs, addFile, and renameTo.  The reason for this is that the limitation
is not actually in Path, but in the filesystem, so we should not enforce it in Path.  An exception
will still be passed back to the client. 

If you object to this plan or prefer the original, please comment. 

> DFS pathname limitation.
> ------------------------
>                 Key: HADOOP-438
>                 URL: http://issues.apache.org/jira/browse/HADOOP-438
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.5.0, 0.4.0, 0.3.2, 0.3.1, 0.3.0, 0.2.1, 0.2.0, 0.1.1, 0.1.0
>            Reporter: Konstantin Shvachko
> I was trying to create a deep hierarchy of directories using DFS mkdirs().
> When the path to the leaf directory became long (~20000) DFS was still able to create
> directories with these names, but UTF8 started truncating long strings resulting in
> incorrect logging of namespace edits. That later crashed the namenode during restart,
> when it was trying to reproduce file creation logged in the edits file with truncated
> UTF8 is deprecated now so we will have to replace it with Text.
> With UTF8  we should enforce a pathname limit of 0xffff/3 = 21845
> With Text it is going to be larger. Not sure what the exact number is.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message