hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From modemide <modem...@gmail.com>
Subject Re: Hadoop and MainFrame integration
Date Tue, 28 Aug 2012 16:30:10 GMT
At some point in the work flow you're going to have to transfer the file
from the mainframe to the Hadoop cluster for processing, and then send it
back for storage on the mainframe.

You should be able to automate the process of sending the files back and

It's been my experience that it's often faster to process and sort large
files on a Hadoop cluster even while factoring in the cost to transfer
to/from the mainframe.

Hopefully that answers your question.  If not, are you looking to actually
use Hadoop to process files in place on the mainframe?  That concept
conflicts with my understanding of Hadoop.

On Tue, Aug 28, 2012 at 12:24 PM, Siddharth Tiwari <
siddharth.tiwari@live.com> wrote:

>  Hi Users.
> We have flat files on mainframes with around a billion records. We need to
> sort them and then use them with different jobs on mainframe for report
> generation. I was wondering was there any way I could integrate the
> mainframe with hadoop do the sorting and keep the file on the sever itself
> ( I do not want to ftp the file to a hadoop cluster and then ftp back the
> sorted file to Mainframe as it would waste MIPS and nullify the advantage
> ). This way I could save on MIPS and ultimately improve profitability.
> Thank you in advance
> **------------------------**
> *Cheers !!!*
> *Siddharth Tiwari*
> Have a refreshing day !!!
> *"Every duty is holy, and devotion to duty is the highest form of worship
> of God.” *
> *"Maybe other people will try to limit me but I don't limit myself"*

View raw message