hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ch huang <justlo...@gmail.com>
Subject Re: how to copy data between two hdfs cluster fastly?
Date Sat, 18 Oct 2014 03:18:19 GMT
some file , total size  is 2T ,and block size  is 128M

On Sat, Oct 18, 2014 at 2:26 AM, Shivram Mani <smani@pivotal.io> wrote:

> What is your approx input size ?
> Do you have multiple files or is this one large file ?
> What is your block size (source and destination cluster) ?
>
> On Fri, Oct 17, 2014 at 4:19 AM, ch huang <justlooks@gmail.com> wrote:
>
>> no ,all default
>>
>> On Fri, Oct 17, 2014 at 5:46 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>
>>> Did you specified how many map tasks?
>>>
>>>
>>> On Fri, Oct 17, 2014 at 4:58 PM, ch huang <justlooks@gmail.com> wrote:
>>>
>>>> hi,maillist:
>>>>              i now use distcp to migrate data from CDH4.4 to CDH5.1 , i
>>>> find when copy small file,it very good, but when transfer big data ,it very
>>>> slow ,any good method recommand? thanks
>>>>
>>>
>>>
>>
>
>
> --
> Thanks
> Shivram
>

Mime
View raw message