hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nagaraj Chandrashekar <nchandrashe...@innominds.com>
Subject Re: ETL/DW to Hadoop migrations
Date Wed, 09 Sep 2015 01:55:10 GMT
Hello Abhishek,

I think you may find this white paper useful.  This document talks about offloading Teradata
with Hadoop. It also talks about capacity and savings costs using Hadoop solutions.
http://dataconomy.com/wp-content/uploads/2014/06/Syncsort-A-123ApproachtoTeradataOffloadwithHadoop.pdf

Cheers
Nagaraj C

Learn And Share! It's Big Data.


From: Sandesh Hegde <sandesh.hegde@gmail.com<mailto:sandesh.hegde@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Wednesday, September 9, 2015 at 1:24 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: ETL/DW to Hadoop migrations

Hello Abhishek,

Below is a link to Free data ingestion tool, dtIngest, this runs on Hadoop as Yarn app. Support
various data sources.
Currently it doesn't have a support for Databases, future versions may have it. For database
you can try Apache Sqoop.

https://www.datatorrent.com/product/datatorrent-dtingest/
https://www.datatorrent.com/dtingest-unified-streaming-batch-data-ingestion-hadoop/

Thanks
Sandesh

PS: I work for DataTorrent.

On Tue, Sep 8, 2015 at 9:57 AM, Abhishek Singh <23singhabhishek@gmail.com<mailto:23singhabhishek@gmail.com>>
wrote:
Hi Kishore,

Thanks for reverting. We are planning to do a POC in such a manner that we can replace Datastage.
Datastage and Teradata are costly tools which is making a big hole in pocket. So, have you
come across anything where ETL pipeline could be replaced with Hadoop? I understand about
connectors which you are saying, but how about replacing an ETL tool?

Any links would do more than good.

Thanks once again.

Abhishek

On Tue, Sep 8, 2015 at 9:28 AM, Krishna Kishore Bonagiri <write2kishore@gmail.com<mailto:write2kishore@gmail.com>>
wrote:
Abhishek,

   Are you looking for loading your data into Hadoop? if yes, IBM DataStage has a stage called
BDFS that loads/writes your data into Hadoop.

Thanks,
Kishore

On Tue, Sep 8, 2015 at 1:29 AM, <23singhabhishek@gmail.com<mailto:23singhabhishek@gmail.com>>
wrote:

Hi guys,

I am looking for pointers on migrating existing data warehouse to Hadoop.  Currently,  we
are using IBM Data stage an ETL tool and loading into Teradata staging/maintain tables.  Please
suggest an architecture which reduces cost without much degrade in performance.  Has anyone
of you been a part of such migration before? If yes then please provide some inputs,  especially
on what aspects should we be taking care of.  Talking about source data,  it is mainly in
the form of flat files and database.

Thanks in advance.

Regards,

Abhishek Singh




Mime
View raw message