hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankam Venkateshwarlu <ank...@gmail.com>
Subject Re: Hadoop and MainFrame integration
Date Tue, 28 Aug 2012 16:51:05 GMT
Can you please explain how to automate the process of sending files back
and forth from Mainframe?  Is it done using NDM?

I have a requirement to migrate Mainframe to Hadoop.  I am looking for more
information in this area about the economics, process and tools that
support migration etc.  Any information related to this is highly
appreciated.  Thanks!

Venkat Ankam

On Tue, Aug 28, 2012 at 10:00 PM, modemide <modemide@gmail.com> wrote:

> At some point in the work flow you're going to have to transfer the file
> from the mainframe to the Hadoop cluster for processing, and then send it
> back for storage on the mainframe.
> You should be able to automate the process of sending the files back and
> forth.
> It's been my experience that it's often faster to process and sort large
> files on a Hadoop cluster even while factoring in the cost to transfer
> to/from the mainframe.
> Hopefully that answers your question.  If not, are you looking to actually
> use Hadoop to process files in place on the mainframe?  That concept
> conflicts with my understanding of Hadoop.
> On Tue, Aug 28, 2012 at 12:24 PM, Siddharth Tiwari <
> siddharth.tiwari@live.com> wrote:
>>  Hi Users.
>> We have flat files on mainframes with around a billion records. We need
>> to sort them and then use them with different jobs on mainframe for report
>> generation. I was wondering was there any way I could integrate the
>> mainframe with hadoop do the sorting and keep the file on the sever itself
>> ( I do not want to ftp the file to a hadoop cluster and then ftp back the
>> sorted file to Mainframe as it would waste MIPS and nullify the advantage
>> ). This way I could save on MIPS and ultimately improve profitability.
>> Thank you in advance
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*

View raw message