hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Divya Gehlot <divya.htco...@gmail.com>
Subject Re: copy data from one hadoop cluster to another hadoop cluster + cant use distcp
Date Tue, 23 Jun 2015 14:45:54 GMT
Can you please elaborate it more.
On 20 Jun 2015 2:46 pm, "SF Hadoop" <sfhadoop@gmail.com> wrote:

> Really depends on your requirements for the format of the data.
>
> The easiest way I can think of is to "stream" batches of data into a pub
> sub system that the target system can access and then consume.
>
> Verify each batch and then ditch them.
>
> You can throttle the size of the intermediary infrastructure based on your
> batches.
>
> Seems the most efficient approach.
>
> On Thursday, June 18, 2015, Divya Gehlot <divya.htconex@gmail.com> wrote:
>
>> Hi,
>> I need to copy data from first hadoop cluster to second hadoop cluster.
>> I cant access second hadoop cluster from first hadoop cluster due to some
>> security issue.
>> Can any point me how can I do apart from distcp command.
>> For instance
>> Cluster 1 secured zone -> copy hdfs data  to -> cluster 2 in non secured
>> zone
>>
>>
>>
>> Thanks,
>> Divya
>>
>>
>>

Mime
View raw message