Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Message-ID: <29856531.149311292460663505.JavaMail.jira@thor>
Date: Wed, 15 Dec 2010 19:51:03 -0500 (EST)
From: "Hairong Kuang (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Subject: [jira] Commented: (HIVE-1852) Reduce unnecessary DFSClient.rename()
 calls
In-Reply-To: <6663913.145761292448841012.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971908#action_12971908 ] 

Hairong Kuang commented on HIVE-1852:
-------------------------------------

Joydeep is right. I did not realize that you try to merge multiple source directories into one.

> Reduce unnecessary DFSClient.rename() calls
> -------------------------------------------
>
>                 Key: HIVE-1852
>                 URL: https://issues.apache.org/jira/browse/HIVE-1852
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Ning Zhang
>            Assignee: Ning Zhang
>         Attachments: HIVE-1852.patch
>
>
> In Hive client side (MoveTask etc), DFSCleint.rename() is called for every file inside a directory. It is very expensive for a large directory in a busy DFS namenode. We should replace it with a single rename() call on the whole directory. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.