hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Meza <j_meza...@hotmail.com>
Subject RE: does "fs -put " create subdirectories?
Date Wed, 16 Jan 2013 21:39:41 GMT
Modifying the script to precede the put with a mkdir (which creates the missing subdirectories)
fixes the issue.thanksJohn

Date: Wed, 16 Jan 2013 12:36:20 -0500
Subject: Re: does "fs -put " create subdirectories?
From: cembree@gmail.com
To: user@hadoop.apache.org

Good point Harsh.  As a Linux Admin, I prefer the behavior of 2.x.  It allows me to see if
I've made a mistake in my planned placement of files instead of blindly writing.

On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <harsh@cloudera.com> wrote:

On 1.x, -put does mkdir the parent directories if they are non existent (akin to mkdir -p).

On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd before -put.


➜  ~  hadoop fs -put foo a/bc/dfe/fsdf.tput: `a/bc/dfe/fsdf.t': No such file or directory


On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org> wrote:


Yes it does, you can just try ;)



hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/

Found 1 items

drwxr-xr-x   - hbase supergroup          0 2013-01-03 09:54 /user/hbase



hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt

/user/test/CHANGES.txt



hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/

Found 2 items

drwxr-xr-x   - hbase  supergroup          0 2013-01-03 09:54 /user/hbase

drwxr-xr-x   - hadoop supergroup          0 2013-01-16 11:56 /user/test



hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/

Found 1 items

-rw-r--r--   3 hadoop supergroup     446615 2013-01-16 11:56

/user/test/CHANGES.txt



hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt

/user/test2/subdir/anotherone/CHANGES.txt



hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/

Found 1 items

-rw-r--r--   3 hadoop supergroup     446615 2013-01-16 11:58

/user/test2/subdir/anotherone/CHANGES.txt





2013/1/16, John Meza <j_mezazap@hotmail.com>:

> Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the

> necassary subdirectories in hdfs?thanksJohn



-- 
Harsh J


 		 	   		  
Mime
View raw message