Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <28546816.1155172213960.JavaMail.jira@brutus>
Date: Wed, 9 Aug 2006 18:10:13 -0700 (PDT)
From: "Konstantin Shvachko (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Created: (HADOOP-438) DFS pathname limitation.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

DFS pathname limitation.
------------------------

                 Key: HADOOP-438
                 URL: http://issues.apache.org/jira/browse/HADOOP-438
             Project: Hadoop
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.5.0, 0.4.0, 0.3.2, 0.3.1, 0.3.0, 0.2.1, 0.2.0, 0.1.1, 0.1.0
            Reporter: Konstantin Shvachko
             Fix For: 0.6.0


I was trying to create a deep hierarchy of directories using DFS mkdirs().
When the path to the leaf directory became long (~20000) DFS was still able to create
directories with these names, but UTF8 started truncating long strings resulting in
incorrect logging of namespace edits. That later crashed the namenode during restart,
when it was trying to reproduce file creation logged in the edits file with truncated names.
UTF8 is deprecated now so we will have to replace it with Text.
With UTF8  we should enforce a pathname limit of 0xffff/3 = 21845
With Text it is going to be larger. Not sure what the exact number is.


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira