hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergio Peña (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HIVE-14029) Update Spark version to 2.0.0
Date Tue, 20 Sep 2016 14:44:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506772#comment-15506772
] 

Sergio Peña commented on HIVE-14029:
------------------------------------

Sure. [~stakiar] Let me know if the below statements are correct, and feel free to correct
me.

- Spark2 uses a fork of Hive 1.2 due to issues with Apache Hive. They called this project
{{spark-hive}}. Spark only uses Hive 1.2 metastore/serde/udf jars form this forked project.
  They download this from https://mvnrepository.com/artifact/org.apache.spark/spark-hive_2.10


- Spark2 assembly without hive will be built without any of the above dependencies.

- Hive2 itests will use Spark2 assembly to run Hive2 tests. This means Hive2 might not test
Spark2 correctly due to the lack of Hive 1.2 libraries in it.

> Update Spark version to 2.0.0
> -----------------------------
>
>                 Key: HIVE-14029
>                 URL: https://issues.apache.org/jira/browse/HIVE-14029
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Ferdinand Xu
>            Assignee: Ferdinand Xu
>         Attachments: HIVE-14029.1.patch, HIVE-14029.patch
>
>
> There are quite some new optimizations in Spark 2.0.0. We need to bump up Spark to 2.0.0
to benefit those performance improvements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message