hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandesh Hegde <sandesh.he...@gmail.com>
Subject Re: ETL/DW to Hadoop migrations
Date Tue, 08 Sep 2015 19:54:48 GMT
Hello Abhishek,

Below is a link to Free data ingestion tool, dtIngest, this runs on Hadoop
as Yarn app. Support various data sources.
Currently it doesn't have a support for Databases, future versions may have
it. For database you can try Apache Sqoop.



PS: I work for DataTorrent.

On Tue, Sep 8, 2015 at 9:57 AM, Abhishek Singh <23singhabhishek@gmail.com>

> Hi Kishore,
> Thanks for reverting. We are planning to do a POC in such a manner that we
> can replace Datastage. Datastage and Teradata are costly tools which is
> making a big hole in pocket. So, have you come across anything where ETL
> pipeline could be replaced with Hadoop? I understand about connectors which
> you are saying, but how about replacing an ETL tool?
> Any links would do more than good.
> Thanks once again.
> Abhishek
> On Tue, Sep 8, 2015 at 9:28 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>> Abhishek,
>>    Are you looking for loading your data into Hadoop? if yes, IBM
>> DataStage has a stage called BDFS that loads/writes your data into Hadoop.
>> Thanks,
>> Kishore
>> On Tue, Sep 8, 2015 at 1:29 AM, <23singhabhishek@gmail.com> wrote:
>>> Hi guys,
>>> I am looking for pointers on migrating existing data warehouse to
>>> Hadoop.  Currently,  we are using IBM Data stage an ETL tool and loading
>>> into Teradata staging/maintain tables.  Please suggest an architecture
>>> which reduces cost without much degrade in performance.  Has anyone of you
>>> been a part of such migration before? If yes then please provide some
>>> inputs,  especially on what aspects should we be taking care of.  Talking
>>> about source data,  it is mainly in the form of flat files and database.
>>> Thanks in advance.
>>> Regards,
>>> Abhishek Singh

View raw message