hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark [Spark Branch]
Date Wed, 12 Apr 2017 21:31:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966675#comment-15966675
] 

Sahil Takiar commented on HIVE-11133:
-------------------------------------

RB: https://reviews.apache.org/r/58402/

* Added a {{DagJsonParser}} which is an abstract class and has two implementations: {{TezJsonParser}}
and {{SparkJsonParser}}
** Almost all of the logic in {{TezJsonParser}} was moved to {{DagJsonParser}}
* Had to disable the {{CorrelationOptimizer}} for Spark, seems that it doesn't work - should
be ok since the {{CorrelationOptimizer}} is disabled by default anyways
* Had to add a few explicit checks for JSON objects to avoid some NullPointerExceptions
* Most of the other changes are just re-factoring
* Turned {{hive.explain.user}} off for all the Spark qtests so that we don't have to updated
all the qtests
* {{hive.explain.user}} is on by default, but we could create a separate config for Spark
and turn it off by default?

> Support hive.explain.user for Spark [Spark Branch]
> --------------------------------------------------
>
>                 Key: HIVE-11133
>                 URL: https://issues.apache.org/jira/browse/HIVE-11133
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Mohit Sabharwal
>            Assignee: Sahil Takiar
>         Attachments: HIVE-11133.1.patch, HIVE-11133.2.patch, HIVE-11133.3.patch, HIVE-11133.4.patch
>
>
> User friendly explain output ({{set hive.explain.user=true}}) should support Spark as
well. 
> Once supported, we should also enable related q-tests like {{explainuser_1.q}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message