spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xdcjie (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-19269) Scheduler Delay in spark ui
Date Wed, 18 Jan 2017 06:14:26 GMT
xdcjie created SPARK-19269:
------------------------------

             Summary: Scheduler Delay in spark ui
                 Key: SPARK-19269
                 URL: https://issues.apache.org/jira/browse/SPARK-19269
             Project: Spark
          Issue Type: Question
          Components: Web UI
    Affects Versions: 2.0.2
            Reporter: xdcjie


In sparkUI, SchedulerDelay is launchtime(in driver) - starttime (in executor), but in the
code, that it consists of launchtime - starttime and statupdate (executor update status to
driver time).
  
private[ui] def getSchedulerDelay(
      info: TaskInfo, metrics: TaskMetricsUIData, currentTime: Long): Long = {
    if (info.finished) {
      val totalExecutionTime = info.finishTime - info.launchTime
      val executorOverhead = (metrics.executorDeserializeTime +
        metrics.resultSerializationTime)
      math.max(
        0,
        totalExecutionTime - metrics.executorRunTime - executorOverhead -
          getGettingResultTime(info, currentTime))
    } else {
      // The task is still running and the metrics like executorRunTime are not available.
      0L
    }
  }

Could we split "scheduler delay" into "scheduler delay time (launchtime-starttime) and "status
update time".  in my workspace, scheduler delay time is 60s,   but launchtime - starttime
is very fast, <1s.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message