Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
Content-Type: multipart/alternative; boundary=Apple-Mail-C0E8202F-DA03-4FBF-BAA6-3E2C51234441
Content-Transfer-Encoding: 7bit
From: =?utf-8?Q?J=C3=B6rn_Franke?= <jornfranke@gmail.com>
Mime-Version: 1.0 (1.0)
Subject: Re: migration from Teradata to Spark SQL
Date: Wed, 4 May 2016 09:22:12 +0200
Message-Id: <3DAB59E5-F6CD-4DF3-8F92-3737D9213266@gmail.com>
References: <CAP_GdYvQpS65+R8j5Fw9mFbsdxvsHNiTSn69_X_KDAaNpd7qjA@mail.gmail.com>
Cc: user@spark.apache.org
In-Reply-To: <CAP_GdYvQpS65+R8j5Fw9mFbsdxvsHNiTSn69_X_KDAaNpd7qjA@mail.gmail.com>
To: Tapan Upadhyay <tapanu@gmail.com>
archived-at: Wed, 04 May 2016 07:28:24 -0000

--Apple-Mail-C0E8202F-DA03-4FBF-BAA6-3E2C51234441
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

Look at lambda architecture.

What is the motivation of your migration?

> On 04 May 2016, at 03:29, Tapan Upadhyay <tapanu@gmail.com> wrote:
>=20
> Hi,
>=20
> We are planning to move our adhoc queries from teradata to spark. We have h=
uge volume of queries during the day. What is best way to go about it -=20
>=20
> 1) Read data directly from teradata db using spark jdbc
>=20
> 2) Import data using sqoop by EOD jobs into hive tables stored as parquet a=
nd then run queries on hive tables using spark sql or spark hive context.
>=20
> any other ways through which we can do it in a better/efficiently?
>=20
> Please guide.
>=20
> Regards,
> Tapan
>=20

--Apple-Mail-C0E8202F-DA03-4FBF-BAA6-3E2C51234441
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div>Look at lambda architecture.</div><div=
 id=3D"AppleMailSignature"><br></div><div id=3D"AppleMailSignature">What is t=
he motivation of your migration?</div><div><br>On 04 May 2016, at 03:29, Tap=
an Upadhyay &lt;<a href=3D"mailto:tapanu@gmail.com">tapanu@gmail.com</a>&gt;=
 wrote:<br><br></div><blockquote type=3D"cite"><div><div dir=3D"ltr">Hi,<div=
><br></div><div>We are planning to move our adhoc queries from teradata to s=
park. We have huge volume of queries during the day. What is best way to go a=
bout it -&nbsp;</div><div><br></div><div>1) Read data directly from teradata=
 db using spark jdbc</div><div><br></div><div>2) Import data using sqoop by E=
OD jobs into hive tables stored as parquet and then run queries on hive tabl=
es using spark sql or spark hive context.</div><div><br></div><div>any other=
 ways through which we can do it in a better/efficiently?</div><div><br></di=
v><div>Please guide.</div><div><br></div><div><div><div class=3D"gmail_signa=
ture"><div dir=3D"ltr"><div dir=3D"ltr"><div style=3D"font-size:small">Regar=
ds,</div><div style=3D"font-size:small">Tapan</div><div style=3D"font-size:s=
mall"><br></div></div></div></div></div>
</div></div>
</div></blockquote></body></html>=

--Apple-Mail-C0E8202F-DA03-4FBF-BAA6-3E2C51234441--