hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <stutiawas...@hcl.com>
Subject Copying directories using distcp
Date Tue, 25 Sep 2012 05:58:59 GMT
Hi all,

I have data in S3 bucket in different folders and want to copy to HDFS directly . I used Distcp
to copy files to HDFS by specifying complete s3 path of the file.
I have 2 queries :

1. Can I copy the entire folder from S3 to HDFS using distcp.
2. Can I generate the exact input folder replica on HDFS as destination folder.

I tried using -f <uri_list> but in that also, I need to provide the complete path of
the S3 files.
Please suggest



The contents of this e-mail and any attachment(s) are confidential and intended for the named
recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e
mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator
or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may
not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying,
disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized
representative of
HCL is strictly prohibited. If you have received this email in error please delete it and
notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.


View raw message