hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gmail <asim...@gmail.com>
Subject Re: Teradata into hadoop Migration
Date Mon, 01 Aug 2016 12:46:58 GMT
Hi Bhagaban

I have seen more efficient way of data transfer by fast exporting the data and put them directly
in HDFS. And then create structure as per the data in HIVE or PIG. We have developed DDL transformation
script using shell script which can convert the TERADATA DDL to Hive DDL. This automation
effort will not take much time . We have faced some challenges while converting the timestamp
and date data type in parquet format. But later those resolved with higher version of hive.
I will also suggest to check TERADATA connector for Hadoop the tool developed by TERADATA
which is self efficient to choose different data transfer strategies.

Thanks
Asim

Sent from my iPhone

> On Aug 1, 2016, at 7:37 AM, Rakesh Radhakrishnan <rakeshr@apache.org> wrote:
> 
> Hi Bhagaban,
> 
> Perhaps, you can try "Apache Sqoop" to transfer data to Hadoop from Teradata. Apache
Sqoop provides an efficient approach for transferring large data between Hadoop related systems
and structured data stores. It allows support for a data store to be added as a so-called
connector and can connect to various databases including Oracle etc.
> 
> I hope the below links will be helpful to you,
> http://sqoop.apache.org/
> http://blog.cloudera.com/blog/2012/01/cloudera-connector-for-teradata-1-0-0/
> http://hortonworks.com/blog/round-trip-data-enrichment-teradata-hadoop/
> http://dataconomy.com/wp-content/uploads/2014/06/Syncsort-A-123ApproachtoTeradataOffloadwithHadoop.pdf
> 
> Below are few data ingestion tools, probably you can dig more into it,
> https://www.datatorrent.com/product/datatorrent-ingestion/
> https://www.datatorrent.com/dtingest-unified-streaming-batch-data-ingestion-hadoop/
> 
> Thanks,
> Rakesh
> 
>> On Mon, Aug 1, 2016 at 4:54 PM, Bhagaban Khatai <email.bhagaban@gmail.com>
wrote:
>> Hi Guys-
>> 
>> I need a quick help if anybody done any migration project in TD into hadoop.
>> We have very tight deadline and I am trying to find any tool (online or paid) for
quick development.
>> 
>> Please help us here and guide me if any other way is available to do the development
fast.
>> 
>> Bhagaban
> 

Mime
View raw message