hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Maier <christopher.ma...@gm.com>
Subject Hadoop fs mv pushes folder under subfolder of same name
Date Wed, 17 Feb 2016 18:20:51 GMT
Hi,

I've run into an odd condition in the Hadoop move command. When moving a directory 'a' under
a directory 'b' which already has a directory 'a' under it, it moves under that subdirectory
rather than failing.

> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a/a

Why is that happening? I've tried looking through the classes on grepcode, but I can't find
a condition where this behavior will occur.

I've also tried directly running the 'rename' command for org.apache.hadoop.hdfs.DistributedFileSystem
from Java, but it behaves as expected.

Every other condition I've been able to come up with (eg, 'b' contains directory 'c' which
contains 'a') behaves as expected.

> hadoop version
Hadoop 2.2.0
Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
Compiled by jenkins on 2015-01-15T00:39Z
Compiled with protoc 2.5.0
>From source with checksum ad91691b86dceaaa5875773852f328d
This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> hdfs -mkdir abc
> hdfs -ls -R abc
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:28 abc/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:28 abc/b
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:28 abc/b
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a/a
> hdfs -mkdir abc/a
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:30 abc/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a/a
> hdfs -mv abc/a abc/b
mv: `abc/a': Input/output error
> hdfs -mv abc/b/a abc/b/c
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:30 abc/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:44 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c/a
> hdfs -mv abc/a abc/b
> hdfs -ls -R abc
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:44 abc/b
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:30 abc/b/a
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c
drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c/a

Thanks,
Kit



Nothing in this message is intended to constitute an electronic signature unless a specific
statement to the contrary is included in this message.

Confidentiality Note: This message is intended only for the person or entity to which it is
addressed. It may contain confidential and/or privileged material. Any review, transmission,
dissemination or other use, or taking of any action in reliance upon this message by persons
or entities other than the intended recipient is prohibited and may be unlawful. If you received
this message in error, please contact the sender and delete it from your computer.

Mime
View raw message