hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arup Malakar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4142) hadoop fs -mv command creates nested directory instead of overwriting when a same named directory as source already exists
Date Fri, 02 Nov 2012 00:48:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arup Malakar updated HDFS-4142:
-------------------------------

    Description: 
Using the hadoop cli when I try to move a directory to another directory, if the target directory
contains a directory with the same name as the source directory, it would create nested directories
instead of overwriting it. This seems counter intuitive as this is not the behavior with unix
mv command. 

Here is an example to explain the bug:

{code}
~ $ hadoop fs -lsr /tmp/root
lsr: DEPRECATED: Please use 'ls -R' instead.
drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent
drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent/child
-rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent/child/passwd
drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2/child
~ $ hadoop fs -mv /tmp/root/parent/child /tmp/root/parent2
~ $ hadoop fs -lsr /tmp/root
lsr: DEPRECATED: Please use 'ls -R' instead.
drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent
drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent2/child
drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent2/child/child
-rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent2/child/child/passwd
{code}

The same operation seems to fail when using the [FileSystem|http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#rename(org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.Path)] rename api  though.

Using the java api:
{code}
~ $ hadoop jar test.jar RenameTest
Before:
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
-rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
After moving: /tmp/renametest/parent/child to /tmp/renametest/targetparent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
-rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child/file
Before:
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
-rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
Could not rename directory: /tmp/renametest/parent/child to /tmp/renametest/targetparent
{code}


  was:
Using the hadoop cli when I try to move a directory to another directory, if the target directory
contains a directory with the same name as the source directory, it would create nested directories
instead of overwriting it. This seems counter intuitive as this is not the behavior with unix
mv command. 

Here is an example to explain the bug:

{code}
~ $ hadoop fs -lsr /tmp/root
lsr: DEPRECATED: Please use 'ls -R' instead.
drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent
drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent/child
-rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent/child/passwd
drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2/child
~ $ hadoop fs -mv /tmp/root/parent/child /tmp/root/parent2
~ $ hadoop fs -lsr /tmp/root
lsr: DEPRECATED: Please use 'ls -R' instead.
drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent
drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent2/child
drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent2/child/child
-rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent2/child/child/passwd
{code}

The same operation seems to fail when using the [FileSystem|http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#rename(org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.Path)] rename api  though.

Using the java api:
~ $ hadoop jar test.jar RenameTest
Before:
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
-rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
After moving: /tmp/renametest/parent/child to /tmp/renametest/targetparent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
-rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child/file
Before:
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
-rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
Could not rename directory: /tmp/renametest/parent/child to /tmp/renametest/targetparent



    
> hadoop fs -mv command creates nested directory instead of overwriting when a same named
directory as source already exists
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4142
>                 URL: https://issues.apache.org/jira/browse/HDFS-4142
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: hadoop 0.23.4
>            Reporter: Arup Malakar
>
> Using the hadoop cli when I try to move a directory to another directory, if the target
directory contains a directory with the same name as the source directory, it would create
nested directories instead of overwriting it. This seems counter intuitive as this is not
the behavior with unix mv command. 
> Here is an example to explain the bug:
> {code}
> ~ $ hadoop fs -lsr /tmp/root
> lsr: DEPRECATED: Please use 'ls -R' instead.
> drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent
> drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent/child
> -rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent/child/passwd
> drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
> drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2/child
> ~ $ hadoop fs -mv /tmp/root/parent/child /tmp/root/parent2
> ~ $ hadoop fs -lsr /tmp/root
> lsr: DEPRECATED: Please use 'ls -R' instead.
> drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent
> drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
> drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent2/child
> drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent2/child/child
> -rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent2/child/child/passwd
> {code}
> The same operation seems to fail when using the [FileSystem|http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#rename(org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.Path)] rename api  though.
> Using the java api:
> {code}
> ~ $ hadoop jar test.jar RenameTest
> Before:
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
> -rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
> About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
> After moving: /tmp/renametest/parent/child to /tmp/renametest/targetparent
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
> -rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child/file
> Before:
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
> -rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
> drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
> About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
> Could not rename directory: /tmp/renametest/parent/child to /tmp/renametest/targetparent
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message