hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0
Date Mon, 07 Nov 2016 07:13:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643352#comment-15643352
] 

Rui Li commented on HIVE-14919:
-------------------------------

One thing I noted is the Xms flag was removed from the executor's options via SPARK-12384.
We may want to set it the same as Xmx to achieve better performance.

> Improve the performance of Hive on Spark 2.0.0
> ----------------------------------------------
>
>                 Key: HIVE-14919
>                 URL: https://issues.apache.org/jira/browse/HIVE-14919
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Ferdinand Xu
>            Assignee: Ferdinand Xu
>         Attachments: benchmark.xlsx
>
>
> In HIVE-14029, we have updated Spark dependency to 2.0.0. We use Intel BigBench[1] to
run benchmark with Spark 2.0 over 10 GB data set comparing with Spark 1.6. We can see quite
some performance degradation for most of the queries for BigBench. For detailed information,
please see the attached file for detailed information. This JIRA is the umbrella ticket addressing
those performance issues.
> [1] https://github.com/intel-hadoop/Big-Data-Benchmark-for-Big-Bench



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message