hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Le Dem (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8842) local file system behavior of mv into an empty directory is inconsistent with HDFS
Date Tue, 25 Sep 2012 17:34:08 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Julien Le Dem updated HADOOP-8842:
----------------------------------

    Description: 
moving into an empty directory replaces the directory instead.
See output of attached script to reproduce :

repro.sh
{noformat}
rm -rf local_fs_bug
mkdir local_fs_bug
hdfs -rmr local_fs_bug
hdfs -mkdir local_fs_bug

echo ">>> HDFS: normal behavior"
touch part-0000
hdfs -mkdir local_fs_bug/a
hdfs -copyFromLocal part-0000 local_fs_bug/a
hdfs -mkdir local_fs_bug/b
hdfs -mkdir local_fs_bug/b/c
echo "content of a: 1 part"
hdfs -ls local_fs_bug/a
echo "content of b/c: empty"
hdfs -ls local_fs_bug/b/c
echo "mv a b/c"
hdfs -mv local_fs_bug/a local_fs_bug/b/c
echo "resulting content of b/c"
hdfs -ls local_fs_bug/b/c
echo "a is moved inside of c"
echo
echo ">>> local fs: bug"
mkdir -p local_fs_bug/a
touch local_fs_bug/a/part-0000
mkdir -p local_fs_bug/b/c
echo "content of a: 1 part"
hdfs -fs local -ls local_fs_bug/a
echo "content of b/c: empty"
hdfs -fs local -ls local_fs_bug/b/c
echo "mv a b/c"
hdfs -fs local -mv local_fs_bug/a local_fs_bug/b/c
echo "resulting content of b/c"
hdfs -fs local -ls local_fs_bug/b/c
echo "bug: a replaces c"
echo
echo ">>> but it works if the destination is not empty"
mkdir local_fs_bug/a2
touch local_fs_bug/a2/part-0000
mkdir -p local_fs_bug/b2/c2
touch local_fs_bug/b2/c2/dummy
echo "content of a2: 1 part"
hdfs -fs local -ls local_fs_bug/a2
echo "content of b2/c2: 1 dummy file"
hdfs -fs local -ls local_fs_bug/b2/c2
echo "mv a2 b2/c2"
hdfs -fs local -mv local_fs_bug/a2 local_fs_bug/b2/c2
echo "resulting content of b/c"
hdfs -fs local -ls local_fs_bug/b2/c2
echo "a2 is moved inside of c2"
{noformat}

Output:
{noformat}
>>> HDFS: normal behavior
content of a: 1 part
Found 1 items
-rw-r--r--   3 julien g          0 2012-09-25 17:16 /user/julien/local_fs_bug/a/part-0000
content of b/c: empty
mv a b/c
resulting content of b/c
Found 1 items
drwxr-xr-x   - julien g          0 2012-09-25 17:16 /user/julien/local_fs_bug/b/c/a
a is moved inside of c

>>> local fs: bug
content of a: 1 part
12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/a/part-0000
content of b/c: empty
12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
mv a b/c
12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
resulting content of b/c
12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b/c/part-0000
bug: a replaces c

>>> but it works if the destination is not empty
content of a2: 1 part
12/09/25 17:16:36 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/a2/part-0000
content of b2/c2: 1 dummy file
12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/dummy
mv a2 b2/c2
12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
resulting content of b/c
12/09/25 17:16:38 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 2 items
drwxr-xr-x   - julien g       4096 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/a2
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/dummy
a2 is moved inside of c2
{noformat}

  was:
moving into an empty directory replaces the directory instead.
See output of attached script to reproduce :

>>> HDFS: normal behavior
content of a: 1 part
Found 1 items
-rw-r--r--   3 julien g          0 2012-09-25 17:16 /user/julien/local_fs_bug/a/part-0000
content of b/c: empty
mv a b/c
resulting content of b/c
Found 1 items
drwxr-xr-x   - julien g          0 2012-09-25 17:16 /user/julien/local_fs_bug/b/c/a
a is moved inside of c

>>> local fs: bug
content of a: 1 part
12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/a/part-0000
content of b/c: empty
12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
mv a b/c
12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
resulting content of b/c
12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b/c/part-0000
bug: a replaces c

>>> but it works if the destination is not empty
content of a2: 1 part
12/09/25 17:16:36 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/a2/part-0000
content of b2/c2: 1 dummy file
12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 1 items
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/dummy
mv a2 b2/c2
12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
resulting content of b/c
12/09/25 17:16:38 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
Found 2 items
drwxr-xr-x   - julien g       4096 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/a2
-rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/dummy
a2 is moved inside of c2

    
> local file system behavior of mv into an empty directory is inconsistent with HDFS
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-8842
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8842
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: Julien Le Dem
>
> moving into an empty directory replaces the directory instead.
> See output of attached script to reproduce :
> repro.sh
> {noformat}
> rm -rf local_fs_bug
> mkdir local_fs_bug
> hdfs -rmr local_fs_bug
> hdfs -mkdir local_fs_bug
> echo ">>> HDFS: normal behavior"
> touch part-0000
> hdfs -mkdir local_fs_bug/a
> hdfs -copyFromLocal part-0000 local_fs_bug/a
> hdfs -mkdir local_fs_bug/b
> hdfs -mkdir local_fs_bug/b/c
> echo "content of a: 1 part"
> hdfs -ls local_fs_bug/a
> echo "content of b/c: empty"
> hdfs -ls local_fs_bug/b/c
> echo "mv a b/c"
> hdfs -mv local_fs_bug/a local_fs_bug/b/c
> echo "resulting content of b/c"
> hdfs -ls local_fs_bug/b/c
> echo "a is moved inside of c"
> echo
> echo ">>> local fs: bug"
> mkdir -p local_fs_bug/a
> touch local_fs_bug/a/part-0000
> mkdir -p local_fs_bug/b/c
> echo "content of a: 1 part"
> hdfs -fs local -ls local_fs_bug/a
> echo "content of b/c: empty"
> hdfs -fs local -ls local_fs_bug/b/c
> echo "mv a b/c"
> hdfs -fs local -mv local_fs_bug/a local_fs_bug/b/c
> echo "resulting content of b/c"
> hdfs -fs local -ls local_fs_bug/b/c
> echo "bug: a replaces c"
> echo
> echo ">>> but it works if the destination is not empty"
> mkdir local_fs_bug/a2
> touch local_fs_bug/a2/part-0000
> mkdir -p local_fs_bug/b2/c2
> touch local_fs_bug/b2/c2/dummy
> echo "content of a2: 1 part"
> hdfs -fs local -ls local_fs_bug/a2
> echo "content of b2/c2: 1 dummy file"
> hdfs -fs local -ls local_fs_bug/b2/c2
> echo "mv a2 b2/c2"
> hdfs -fs local -mv local_fs_bug/a2 local_fs_bug/b2/c2
> echo "resulting content of b/c"
> hdfs -fs local -ls local_fs_bug/b2/c2
> echo "a2 is moved inside of c2"
> {noformat}
> Output:
> {noformat}
> >>> HDFS: normal behavior
> content of a: 1 part
> Found 1 items
> -rw-r--r--   3 julien g          0 2012-09-25 17:16 /user/julien/local_fs_bug/a/part-0000
> content of b/c: empty
> mv a b/c
> resulting content of b/c
> Found 1 items
> drwxr-xr-x   - julien g          0 2012-09-25 17:16 /user/julien/local_fs_bug/b/c/a
> a is moved inside of c
> >>> local fs: bug
> content of a: 1 part
> 12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> Found 1 items
> -rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/a/part-0000
> content of b/c: empty
> 12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> mv a b/c
> 12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> resulting content of b/c
> 12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> Found 1 items
> -rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b/c/part-0000
> bug: a replaces c
> >>> but it works if the destination is not empty
> content of a2: 1 part
> 12/09/25 17:16:36 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> Found 1 items
> -rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/a2/part-0000
> content of b2/c2: 1 dummy file
> 12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> Found 1 items
> -rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/dummy
> mv a2 b2/c2
> 12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> resulting content of b/c
> 12/09/25 17:16:38 WARN fs.FileSystem: "local" is a deprecated filesystem name. Use "file:///"
instead.
> Found 2 items
> drwxr-xr-x   - julien g       4096 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/a2
> -rw-r--r--   1 julien g          0 2012-09-25 17:16 /home/julien/local_fs_bug/b2/c2/dummy
> a2 is moved inside of c2
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message