Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <32814299.35411273099695803.JavaMail.jira@thor>
Date: Wed, 5 May 2010 18:48:15 -0400 (EDT)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-1028) INode.getPathNames could split more
 efficiently
In-Reply-To: <1543593839.115381267859247283.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864568#action_12864568 ] 

Todd Lipcon commented on HDFS-1028:
-----------------------------------

+1 to the body of the patch. Regarding the test change, how long does the test case take now? Can we make a better benchmark that is independent of the unit tests (I assume this change was to show a speed improvement)? I don't think it makes sense to overload the unit tests for the purposes of benchmarking.

Personally I'd be satisfied to just have simple timings of loading one of your production fsimages with/without the change.

> INode.getPathNames could split more efficiently
> -----------------------------------------------
>
>                 Key: HDFS-1028
>                 URL: https://issues.apache.org/jira/browse/HDFS-1028
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: Todd Lipcon
>            Assignee: Dmytro Molkov
>            Priority: Minor
>         Attachments: HDFS-split.patch
>
>
> INode.getPathnames uses String.split(String) which actually uses the full Java regex implementation. Since we're always splitting on a single char, we could implement a faster one like StringUtils.split() (except without the escape character). This takes a significant amount of CPU during FSImage loading so should be a worthwhile speedup.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.