Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <6351019.1158011965675.JavaMail.jira@brutus>
Date: Mon, 11 Sep 2006 14:59:25 -0700 (PDT)
From: "Wendy Chien (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-438) DFS pathname limitation.
In-Reply-To: <28546816.1155172213960.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/HADOOP-438?page=comments#action_12433992 ] 
            
Wendy Chien commented on HADOOP-438:
------------------------------------

We've changed our plan.  Instead of enforcing the length in the Path constructor, we are going to enforce it in mkdirs, addFile, and renameTo.  The reason for this is that the limitation is not actually in Path, but in the filesystem, so we should not enforce it in Path.  An exception will still be passed back to the client. 

If you object to this plan or prefer the original, please comment. 


> DFS pathname limitation.
> ------------------------
>
>                 Key: HADOOP-438
>                 URL: http://issues.apache.org/jira/browse/HADOOP-438
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.5.0, 0.4.0, 0.3.2, 0.3.1, 0.3.0, 0.2.1, 0.2.0, 0.1.1, 0.1.0
>            Reporter: Konstantin Shvachko
>
> I was trying to create a deep hierarchy of directories using DFS mkdirs().
> When the path to the leaf directory became long (~20000) DFS was still able to create
> directories with these names, but UTF8 started truncating long strings resulting in
> incorrect logging of namespace edits. That later crashed the namenode during restart,
> when it was trying to reproduce file creation logged in the edits file with truncated names.
> UTF8 is deprecated now so we will have to replace it with Text.
> With UTF8  we should enforce a pathname limit of 0xffff/3 = 21845
> With Text it is going to be larger. Not sure what the exact number is.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira