hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandeep Singh ...@chandeep.com>
Subject Re: Hadoop fs mv pushes folder under subfolder of same name
Date Wed, 17 Feb 2016 18:29:18 GMT
This is what I get on CDH 5.4.8:

[csingh]$ hadoop fs -ls -R abc
drwxr-xr-x   - hdfs supergroup          0 2016-02-17 18:25 abc/a
drwxr-xr-x   - hdfs supergroup          0 2016-02-17 18:26 abc/b
drwxr-xr-x   - hdfs supergroup          0 2016-02-17 18:26 abc/b/a

[csingh]$ hadoop fs -mv abc/a abc/b
mv: `abc/b/a': File exists

> On Feb 17, 2016, at 6:20 PM, Christopher Maier <christopher.maier@gm.com> wrote:
> 
> Hi, 
>  
> I’ve run into an odd condition in the Hadoop move command. When moving a directory
‘a’ under a directory ‘b’ which already has a directory ‘a’ under it, it moves
under that subdirectory rather than failing.
>  
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a/a
>  
> Why is that happening? I’ve tried looking through the classes on grepcode, but I can’t
find a condition where this behavior will occur.
>  
> I’ve also tried directly running the ‘rename’ command for org.apache.hadoop.hdfs.DistributedFileSystem
from Java, but it behaves as expected.
>  
> Every other condition I’ve been able to come up with (eg, ‘b’ contains directory
‘c’ which contains ‘a’) behaves as expected.
>  
> > hadoop version
> Hadoop 2.2.0
> Subversion git@bioss:hadoop -r ad26232c97ed70ea7df58f92ab5e52a617d6f3dc
> Compiled by jenkins on 2015-01-15T00:39Z
> Compiled with protoc 2.5.0
> From source with checksum ad91691b86dceaaa5875773852f328d
> This command was run using /opt/ibm/BigInsights_v3.0.0.2_b20150120_1502/IHC/share/hadoop/common/hadoop-common-2.2.0.jar
> > hdfs -mkdir abc
> > hdfs -ls -R abc
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:28 abc/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:28 abc/b
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:28 abc/b
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mkdir abc/a
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:30 abc/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/a/a
> > hdfs -mv abc/a abc/b
> mv: `abc/a': Input/output error
> > hdfs -mv abc/b/a abc/b/c
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:30 abc/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:44 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c/a
> > hdfs -mv abc/a abc/b
> > hdfs -ls -R abc
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:44 abc/b
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:30 abc/b/a
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c
> drwxr-x--x   - dedwmktmedia supergroup          0 2016-02-17 12:29 abc/b/c/a
>  
> Thanks,
> Kit
>  
> 
> 
> Nothing in this message is intended to constitute an electronic signature unless a specific
statement to the contrary is included in this message. 
> 
> Confidentiality Note: This message is intended only for the person or entity to which
it is addressed. It may contain confidential and/or privileged material. Any review, transmission,
dissemination or other use, or taking of any action in reliance upon this message by persons
or entities other than the intended recipient is prohibited and may be unlawful. If you received
this message in error, please contact the sender and delete it from your computer.


Mime
View raw message