Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of modemide@gmail.com designates
 209.85.214.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <SNT142-W50AC715BB76270AD9FB383E0A10@phx.gbl>
References: <SNT142-W50AC715BB76270AD9FB383E0A10@phx.gbl>
From: modemide <modemide@gmail.com>
Date: Tue, 28 Aug 2012 12:30:10 -0400
Message-ID: 
 <CAFNH2sYM0V_3ynL0sFY_CichbOuR=uKeRPcyRLLRp_=sZzg6cA@mail.gmail.com>
Subject: Re: Hadoop and MainFrame integration
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=f46d04016c23f00f7304c855f551

--f46d04016c23f00f7304c855f551
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

At some point in the work flow you're going to have to transfer the file
from the mainframe to the Hadoop cluster for processing, and then send it
back for storage on the mainframe.

You should be able to automate the process of sending the files back and
forth.

It's been my experience that it's often faster to process and sort large
files on a Hadoop cluster even while factoring in the cost to transfer
to/from the mainframe.

Hopefully that answers your question.  If not, are you looking to actually
use Hadoop to process files in place on the mainframe?  That concept
conflicts with my understanding of Hadoop.

On Tue, Aug 28, 2012 at 12:24 PM, Siddharth Tiwari <
siddharth.tiwari@live.com> wrote:

>  Hi Users.
>
> We have flat files on mainframes with around a billion records. We need t=
o
> sort them and then use them with different jobs on mainframe for report
> generation. I was wondering was there any way I could integrate the
> mainframe with hadoop do the sorting and keep the file on the sever itsel=
f
> ( I do not want to ftp the file to a hadoop cluster and then ftp back the
> sorted file to Mainframe as it would waste MIPS and nullify the advantage
> ). This way I could save on MIPS and ultimately improve profitability.
>
> Thank you in advance
>
>
> **------------------------**
> *Cheers !!!*
> *Siddharth Tiwari*
> Have a refreshing day !!!
> *"Every duty is holy, and devotion to duty is the highest form of worship
> of God.=94 *
> *"Maybe other people will try to limit me but I don't limit myself"*
>

--f46d04016c23f00f7304c855f551
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

At some point in the work flow you&#39;re going to have to transfer the fil=
e from the mainframe to the Hadoop cluster for processing, and then send it=
 back for storage on the mainframe.<div><br></div><div>You should be able t=
o automate the process of sending the files back and forth. =A0</div>

<div><br></div><div>It&#39;s been my experience that it&#39;s often faster =
to process and sort large files on a Hadoop cluster even while factoring in=
 the cost to transfer to/from the mainframe.</div><div><br></div><div>
Hopefully that answers your question. =A0If not, are you looking to actuall=
y use Hadoop to process files in place on the mainframe? =A0That concept co=
nflicts with my understanding of Hadoop.</div>
<div><br><div class=3D"gmail_quote">On Tue, Aug 28, 2012 at 12:24 PM, Siddh=
arth Tiwari <span dir=3D"ltr">&lt;<a href=3D"mailto:siddharth.tiwari@live.c=
om" target=3D"_blank">siddharth.tiwari@live.com</a>&gt;</span> wrote:<br><b=
lockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px =
#ccc solid;padding-left:1ex">


<div><div dir=3D"ltr">
<font color=3D"#17365D" face=3D"Tahoma">Hi Users.<br><br>We have flat files=
 on mainframes with around a billion records. We need to sort them and then=
 use them with different jobs on mainframe for report generation. I was won=
dering was there any way I could integrate the mainframe with hadoop do the=
 sorting and keep the file on the sever itself ( I do not want to ftp the f=
ile to a hadoop cluster and then ftp back the sorted file to Mainframe as i=
t would waste MIPS and nullify the advantage ). This way I could save on MI=
PS and ultimately improve profitability. <br>

<br>Thank you in advance<br></font><br><br><strong><font color=3D"#00b050">=
*------------------------*</font></strong><br>
<font color=3D"#17365D" face=3D"Franklin Gothic Medium"><strong><u>Cheers !=
!!</u></strong></font><span class=3D"HOEnZb"><font color=3D"#888888"><font =
color=3D"#17365D" face=3D"Franklin Gothic Medium"><br></font>
<font color=3D"#17365D" face=3D"Franklin Gothic Medium"><strong><font>Siddh=
arth</font> <font>Tiwari</font></strong></font><font color=3D"#17365D" face=
=3D"Franklin Gothic Medium"><br></font>
<font color=3D"#17365D" face=3D"Franklin Gothic Medium">Have a refreshing d=
ay !!!</font><font face=3D"Franklin Gothic Medium"><br></font><font color=
=3D"#974806" face=3D"Franklin Gothic Medium"><b>&quot;Every duty is holy, a=
nd devotion to duty is the highest form of worship of God.=94 </b></font><b=
r>


<b><font color=3D"#002060">&quot;</font><span><font color=3D"#C00000">Maybe=
 other people will try to limit me but I don&#39;t limit myself</font><font=
 color=3D"#002060">&quot;</font></span></b><br> 		 	   		  </font></span></=
div>

</div>
</blockquote></div><br></div>

--f46d04016c23f00f7304c855f551--