hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <>
Subject [jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark [Spark Branch]
Date Wed, 12 Apr 2017 21:31:41 GMT


Sahil Takiar commented on HIVE-11133:


* Added a {{DagJsonParser}} which is an abstract class and has two implementations: {{TezJsonParser}}
and {{SparkJsonParser}}
** Almost all of the logic in {{TezJsonParser}} was moved to {{DagJsonParser}}
* Had to disable the {{CorrelationOptimizer}} for Spark, seems that it doesn't work - should
be ok since the {{CorrelationOptimizer}} is disabled by default anyways
* Had to add a few explicit checks for JSON objects to avoid some NullPointerExceptions
* Most of the other changes are just re-factoring
* Turned {{hive.explain.user}} off for all the Spark qtests so that we don't have to updated
all the qtests
* {{hive.explain.user}} is on by default, but we could create a separate config for Spark
and turn it off by default?

> Support hive.explain.user for Spark [Spark Branch]
> --------------------------------------------------
>                 Key: HIVE-11133
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Mohit Sabharwal
>            Assignee: Sahil Takiar
>         Attachments: HIVE-11133.1.patch, HIVE-11133.2.patch, HIVE-11133.3.patch, HIVE-11133.4.patch
> User friendly explain output ({{set hive.explain.user=true}}) should support Spark as
> Once supported, we should also enable related q-tests like {{explainuser_1.q}}

This message was sent by Atlassian JIRA

View raw message