hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuefu Zhang <xzh...@cloudera.com>
Subject Re: Pros and cons -Saving spark data in hive
Date Wed, 16 Dec 2015 04:00:58 GMT
You might want to consider Hive on Spark where you can work directly with
Hive and your query execution is powered by Spark as an engine.

--Xuefu

On Tue, Dec 15, 2015 at 6:04 PM, Divya Gehlot <divya.htconex@gmail.com>
wrote:

> Hi,
> I am new bee to Spark and  I am exploring option and pros and cons which
> one will work best in spark and hive context.My  dataset  inputs are CSV
> files, using spark to process the my data and saving it in hive using
> hivecontext
>
> 1) Process the CSV file using spark-csv package and create temptable and
> store the data in hive using hive context.
> 2) Process the file as normal text file in sqlcontext  ,register its as
> temptable in sqlcontext and store it as ORC file and read that ORC file in
> hive context and store it in hive.
>
> Is there any other best options apart from mentioned above.
> Would really appreciate the inputs.
> Thanks in advance.
>
> Thanks,
> Regards,
> Divya
>

Mime
View raw message