spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] gatorsmile commented on a change in pull request #26127: [SPARK-29348][SQL] Add observable Metrics for Streaming queries
Date Mon, 30 Dec 2019 00:19:35 GMT
gatorsmile commented on a change in pull request #26127: [SPARK-29348][SQL] Add observable
Metrics for Streaming queries
URL: https://github.com/apache/spark/pull/26127#discussion_r361880867
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
 ##########
 @@ -106,6 +106,9 @@ class QueryExecution(
   lazy val toRdd: RDD[InternalRow] = new SQLExecutionRDD(
     executedPlan.execute(), sparkSession.sessionState.conf)
 
+  /** Get the metrics observed during the execution of the query plan. */
+  def observedMetrics: Map[String, Row] = CollectMetricsExec.collect(executedPlan)
 
 Review comment:
   Based on this PR description, `observedMetrics ` sounds a public API for end users to use.

   
   However, it conflicts with what we commented above [in toRDD]. QueryExecution is not a
public class and we discourage users to call the internal functions. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message