hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: FileSystem API - Moving files in HDFS
Date Mon, 16 May 2011 17:06:46 GMT
If they are lots of large files, and you need to copy them quickly, i.e. Not have all the data
go through a single machine, you can use hadoop distcp too.


On 5/14/11 12:49 AM, "Mahadev Konar" <mahadev@apache.org> wrote:

 you can use FileUtil.copy() methods to copy files.

Hope that helps.


On Fri, May 13, 2011 at 2:00 PM, lohit <lohit_bv@yahoo.com> wrote:
> There is no FileSystem API to copy.
> You could try
> hadoop dfs -cp <src> <dest>
> which basically reads the file and writes to new file.
> The code for this is in FsShell.java
> ----- Original Message ----
> From: Jim Twensky <jim.twensky@gmail.com>
> To: core-user@hadoop.apache.org
> Sent: Fri, May 13, 2011 1:21:09 PM
> Subject: FileSystem API - Moving files in HDFS
> Hi,
> I'd like to move and copy files from one directory in HDFS to another
> one. I know there are methods in the Filesystem API that enable
> copying files between the local disk and HDFS, but I couldn't figure
> out how to do this between two paths both in HDFS. I think rename(Path
> src, Path dest) can be used to move files, but copying still remains a
> challenge to me. Any ideas?
> Thanks,
> Jim

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message