hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wendy Chien (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-438) DFS pathname limitation.
Date Thu, 07 Sep 2006 01:26:23 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-438?page=comments#action_12433010 ] 
Wendy Chien commented on HADOOP-438:

To fix this we're going to enforce a pathname limit in the Path class constructor. .  (8K
in length and 1K in depth.)   The constructor will throw an exception which will be passed
back to the client. 

> DFS pathname limitation.
> ------------------------
>                 Key: HADOOP-438
>                 URL: http://issues.apache.org/jira/browse/HADOOP-438
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.5.0, 0.4.0, 0.3.2, 0.3.1, 0.3.0, 0.2.1, 0.2.0, 0.1.1, 0.1.0
>            Reporter: Konstantin Shvachko
>             Fix For: 0.6.0
> I was trying to create a deep hierarchy of directories using DFS mkdirs().
> When the path to the leaf directory became long (~20000) DFS was still able to create
> directories with these names, but UTF8 started truncating long strings resulting in
> incorrect logging of namespace edits. That later crashed the namenode during restart,
> when it was trying to reproduce file creation logged in the edits file with truncated
> UTF8 is deprecated now so we will have to replace it with Text.
> With UTF8  we should enforce a pathname limit of 0xffff/3 = 21845
> With Text it is going to be larger. Not sure what the exact number is.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message