hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Pivovarov <apivova...@gmail.com>
Subject Re: how to copy data between two hdfs cluster fastly?
Date Fri, 17 Oct 2014 18:50:28 GMT
try to run on dest cluster datanode
$ hadoop fs -cp hdfs://from_cluster/....    hdfs://to_cluster/....



On Fri, Oct 17, 2014 at 11:26 AM, Shivram Mani <smani@pivotal.io> wrote:

> What is your approx input size ?
> Do you have multiple files or is this one large file ?
> What is your block size (source and destination cluster) ?
>
> On Fri, Oct 17, 2014 at 4:19 AM, ch huang <justlooks@gmail.com> wrote:
>
>> no ,all default
>>
>> On Fri, Oct 17, 2014 at 5:46 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>
>>> Did you specified how many map tasks?
>>>
>>>
>>> On Fri, Oct 17, 2014 at 4:58 PM, ch huang <justlooks@gmail.com> wrote:
>>>
>>>> hi,maillist:
>>>>              i now use distcp to migrate data from CDH4.4 to CDH5.1 , i
>>>> find when copy small file,it very good, but when transfer big data ,it very
>>>> slow ,any good method recommand? thanks
>>>>
>>>
>>>
>>
>
>
> --
> Thanks
> Shivram
>

Mime
View raw message