hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-10058) Log the information of cached RDD [Spark Branch]
Date Mon, 23 Mar 2015 20:16:53 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376552#comment-14376552
] 

Xuefu Zhang edited comment on HIVE-10058 at 3/23/15 8:16 PM:
-------------------------------------------------------------

[~chinnalalam], Sorry that I wasn't clear. I was thinking more on HIVE-8858, where I'd like
to have visual representation of SparkPlan. As you can see in the class definition, A SparkPlan
consists of a graph of SparkTrans. SparkTran has a few subclasses. Some subclass, such as
MapInput has property such as "toCache". What is desirable is that we log a SparkPlan in a
graphical way similar to what's show for work graph in explain plan, such as:
{code} 
MapInput (cache off) -> Shuffle (cache on) -> Reduce
                                           \-> Reduce
{code}
This is will give us some idea about SparkPlan that we are executing. Let me know if you have
any questions.


was (Author: xuefuz):
[~chinnalalam], Sorry that I wasn't clear. I was thinking more on HIVE-8858, where I'd like
to have visual representation of SparkPlan. As you can see in the class definition, A SparkPlan
consists of a graph of SparkTrans. SparkTran has a few subclasses. Some subclass, such as
MapInput has property such as "toCache". What is desirable is that we log a SparkPlan in a
graphical way similar to what's show for work graph in explain plan, such as:
{code} 
MapInput (cache off) -> Shuffle (cache on) -> Reduce
                                        \-> Reduce
{code}
This is will give us some idea about SparkPlan that we are executing. Let me know if you have
any questions.

> Log the information of cached RDD [Spark Branch]
> ------------------------------------------------
>
>                 Key: HIVE-10058
>                 URL: https://issues.apache.org/jira/browse/HIVE-10058
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Chinna Rao Lalam
>            Assignee: Chinna Rao Lalam
>             Fix For: spark-branch
>
>         Attachments: HIVE-10058.1-spark.patch, HIVE-10058.2-spark.patch
>
>
> Log the cached RDD Id's at info level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message