hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hadoop hive <hadooph...@gmail.com>
Subject Re: copy data from one hadoop cluster to another hadoop cluster + cant use distcp
Date Wed, 24 Jun 2015 15:31:26 GMT
You can use node.js for this.

On Tue, Jun 23, 2015 at 8:15 PM, Divya Gehlot <divya.htconex@gmail.com>
wrote:

> Can you please elaborate it more.
> On 20 Jun 2015 2:46 pm, "SF Hadoop" <sfhadoop@gmail.com> wrote:
>
>> Really depends on your requirements for the format of the data.
>>
>> The easiest way I can think of is to "stream" batches of data into a pub
>> sub system that the target system can access and then consume.
>>
>> Verify each batch and then ditch them.
>>
>> You can throttle the size of the intermediary infrastructure based on
>> your batches.
>>
>> Seems the most efficient approach.
>>
>> On Thursday, June 18, 2015, Divya Gehlot <divya.htconex@gmail.com> wrote:
>>
>>> Hi,
>>> I need to copy data from first hadoop cluster to second hadoop cluster.
>>> I cant access second hadoop cluster from first hadoop cluster due to
>>> some security issue.
>>> Can any point me how can I do apart from distcp command.
>>> For instance
>>> Cluster 1 secured zone -> copy hdfs data  to -> cluster 2 in non secured
>>> zone
>>>
>>>
>>>
>>> Thanks,
>>> Divya
>>>
>>>
>>>

Mime
View raw message