hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17545) Make HoS RDD Cacheing Optimization Configurable
Date Thu, 28 Sep 2017 00:17:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183483#comment-16183483
] 

Rui Li commented on HIVE-17545:
-------------------------------

[~stakiar] I think so. We actually have {{SplitSparkWorkResolver}} to clone the works if they
have multiple children. Besides, RDD caching is also used in other places like parallel order
by. If we want to control the behaviour, guess we need to consolidate the usage a little bit.

> Make HoS RDD Cacheing Optimization Configurable
> -----------------------------------------------
>
>                 Key: HIVE-17545
>                 URL: https://issues.apache.org/jira/browse/HIVE-17545
>             Project: Hive
>          Issue Type: Improvement
>          Components: Physical Optimizer, Spark
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>         Attachments: HIVE-17545.1.patch, HIVE-17545.2.patch
>
>
> The RDD cacheing optimization add in HIVE-10550 is enabled by default. We should make
it configurable in case users want to disable it. We can leave it on by default to preserve
backwards compatibility.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message