Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAC3gpCaZj8W=2LaHE7FkK2QXkv1fqRnxpqkR-S29_fBz+ZNHtA@mail.gmail.com>
References: <3038E6F0-C311-459F-90A3-2B101A75EA2C@gmail.com>
	<CAC3gpCaZj8W=2LaHE7FkK2QXkv1fqRnxpqkR-S29_fBz+ZNHtA@mail.gmail.com>
Date: Tue, 16 Feb 2016 16:50:58 +0530
Message-ID: 
 <CAGd0WxKtOcsv9D_t_yTYbqx9GXX5UHfVi+=fEZBesgD1h-2ygw@mail.gmail.com>
Subject: Re: Is it ok to build an entire ETL/ELT data flow using HIVE queries?
From: Devopam Mittra <devopam@gmail.com>
To: "user@hive.apache.org" <user@hive.apache.org>
Content-Type: multipart/alternative; boundary=001a11401e28de5e09052be1531a

--001a11401e28de5e09052be1531a
Content-Type: text/plain; charset=UTF-8

+1 for all suggestions provided already.

I have personally use Talend Big Data Studio in conjunction with Hive +
Cron/Autosys to build and manage small DW.
Found it easy to rapidly build and deploy. Helps with email integration etc
which was my custom requirement (spool few reports and share via email at
routine intervals).

regards
Dev

On Tue, Feb 16, 2016 at 4:10 PM, Elliot West <teabot@gmail.com> wrote:

> I'd say that so long as you can achieve a similar quality of engineering
> as is possible with other software development domains, then 'yes, it is
> ok'.
>
> Specifically, our Hive projects are packaged as RPMs, built and released
> with Maven, covered by suites of unit tests developed with HiveRunner, and
> part of the same Jenkins CI process as other Java based projects.
> Decomposing large processes into sensible units is not as easy as with
> other frameworks so this may require more thought and care.
>
> More information here:
> https://cwiki.apache.org/confluence/display/Hive/Unit+testing+HQL
>
> You have many potential alternatives depending on which languages you are
> comfortable using: Pig, Flink, Cascading, Spark, Crunch, Scrunch, Scalding,
> etc.
>
> Elliot.
>
>
> On Tuesday, 16 February 2016, Ramasubramanian <
> ramasubramanian.narayanan@gmail.com> wrote:
>
>> Hi,
>>
>> Is it ok to build an entire ETL/ELT data flow using HIVE queries?
>>
>> Data is stored in HIVE. We have transactional and reference data. We need
>> to build a small warehouse.
>>
>> Need suggestion on alternatives too.
>>
>> Regards,
>> Rams
>
>


-- 
Devopam Mittra
Life and Relations are not binary

--001a11401e28de5e09052be1531a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">+1 for all suggestions provided already.<div><br><div>I ha=
ve personally use Talend Big Data Studio in conjunction with Hive + Cron/Au=
tosys to build and manage small DW.=C2=A0</div><div>Found it easy to rapidl=
y build and deploy. Helps with email integration etc which was my custom re=
quirement (spool few reports and share via email at routine intervals).</di=
v><div><br></div><div>regards</div><div>Dev</div></div></div><div class=3D"=
gmail_extra"><br><div class=3D"gmail_quote">On Tue, Feb 16, 2016 at 4:10 PM=
, Elliot West <span dir=3D"ltr">&lt;<a href=3D"mailto:teabot@gmail.com" tar=
get=3D"_blank">teabot@gmail.com</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">I&#39;d say that so long as you can achieve a=C2=A0similar qu=
ality=C2=A0of engineering as is possible with other software development do=
mains, then &#39;yes, it is ok&#39;.=C2=A0<div><br></div><div>Specifically,=
 our Hive projects are packaged as RPMs, built and released with Maven, cov=
ered by suites of unit tests developed with HiveRunner, and part of the sam=
e Jenkins CI process as other Java based projects. Decomposing large proces=
ses into sensible units=C2=A0is not as easy as with other frameworks so thi=
s may require=C2=A0more thought and care.</div><div><br></div><div>More inf=
ormation here:</div><div><a href=3D"https://cwiki.apache.org/confluence/dis=
play/Hive/Unit+testing+HQL" target=3D"_blank">https://cwiki.apache.org/conf=
luence/display/Hive/Unit+testing+HQL</a><br></div><div><br></div><div>You h=
ave many potential=C2=A0alternatives depending on which languages you are c=
omfortable using: Pig, Flink,=C2=A0Cascading, Spark, Crunch, Scrunch, Scald=
ing, etc.</div><span class=3D"HOEnZb"><font color=3D"#888888"><div><br></di=
v></font></span><div><span class=3D"HOEnZb"><font color=3D"#888888">Elliot.=
</font></span><div><div class=3D"h5"><br><br>On Tuesday, 16 February 2016, =
Ramasubramanian &lt;<a href=3D"mailto:ramasubramanian.narayanan@gmail.com" =
target=3D"_blank">ramasubramanian.narayanan@gmail.com</a>&gt; wrote:<br><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #=
ccc solid;padding-left:1ex">Hi,<br>
<br>
Is it ok to build an entire ETL/ELT data flow using HIVE queries?<br>
<br>
Data is stored in HIVE. We have transactional and reference data. We need t=
o build a small warehouse.<br>
<br>
Need suggestion on alternatives too.<br>
<br>
Regards,<br>
Rams</blockquote></div></div></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature">Devopam Mittra<br>Life and Relations are not binary</d=
iv>
</div>

--001a11401e28de5e09052be1531a--