spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-26139) Support passing shuffle metrics to exchange operator
Date Fri, 23 Nov 2018 21:25:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-26139:
------------------------------------

    Assignee: Reynold Xin  (was: Apache Spark)

> Support passing shuffle metrics to exchange operator
> ----------------------------------------------------
>
>                 Key: SPARK-26139
>                 URL: https://issues.apache.org/jira/browse/SPARK-26139
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Reynold Xin
>            Assignee: Reynold Xin
>            Priority: Major
>
> Due to the way Spark's architected (SQL is defined on top of the RDD API), there are
two separate metrics system used in core vs SQL. Ideally, we'd want to be able to get the
shuffle metrics for each of the exchange operator independently, e.g. blocks read, number
of records.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message