hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank <fhfo...@comcast.net>
Subject Re: Migrating from Legacy to Hadoop.
Date Wed, 09 Oct 2013 02:35:50 GMT
Low cost alternative ETL is syncsort DMX-h ETL which extends hadoop MapReduce

Sent from my iPad

On Oct 8, 2013, at 10:16 PM, Peyman Mohajerian <mohajeri@gmail.com> wrote:

> I wonder if JDBC driver over Hive could help you. If you legacy ETL job can talk to a
jdbc driver, it is a slow way of writing to HDFS and I don't have any experience doing it,
e.g.:
> http://doc.cloveretl.com/documentation/UserGuide/index.jsp?topic=/com.cloveretl.gui.docs/docs/hive-connection.html
> 
> 
> On Tue, Oct 8, 2013 at 10:07 AM, Jitendra Yadav <jeetuyadav200890@gmail.com> wrote:
>> Hi Bertrand,
>> 
>> Thanks for your reply.
>> 
>> As per my understanding mentioned open source does not support
>> procedure language(PL) flexibility. Right?
>> 
>> I was looking for some other alternatives so that we can migrate our
>> existing code rather then creating java UDF etc. So handling complex
>> ETL business logic is still very difficult on Hadoop in terms of
>> coding, QA and performance?
>> 
>> 
>> Regards
>> Jitendra
>> 
>> On 10/8/13, Bertrand Dechoux <dechouxb@gmail.com> wrote:
>> > open source : Pig, Hive, Cascading ...
>> > other : Talend ...
>> >
>> > Is that the answer you are expecting or are you looking for something more
>> > specific?
>> >
>> > Regards
>> >
>> > Bertrand
>> >
>> >
>> > On Tue, Oct 8, 2013 at 5:47 PM, Jitendra Yadav
>> > <jeetuyadav200890@gmail.com>wrote:
>> >
>> >> Hi All,
>> >>
>> >> We are planning to consolidate our 3 existing warehouse databases to
>> >> Hadoop cluster, In our testing phase we have designed the target
>> >> environment and transferred the data from source to target (not in
>> >> sync but almost completed ). These legacy systems were using
>> >> traditional ETL/replication  mechanism like Golden gate, Loaders,
>> >> PL/SQL language etc., FYI we are using 80%  PL/SQL code and SQL server
>> >> packages in current environment, however we have re-writing some of
>> >> ETL jobs through java and python MR but looking for some more and easy
>> >> alternatives.
>> >>
>> >> What is best approach we should follow to complete this process? while
>> >> suggesting please take effort and timing in consideration( if
>> >> possible).
>> >>
>> >> Please guide.
>> >>
>> >> Regards
>> >> Jitendra
>> >>
>> >
> 

Mime
View raw message