spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Or (JIRA)" <>
Subject [jira] [Resolved] (SPARK-13269) Expose more executor stats in stable status API
Date Wed, 04 May 2016 03:20:13 GMT


Andrew Or resolved SPARK-13269.
       Resolution: Fixed
         Assignee: Wenchen Fan
    Fix Version/s: 2.0.0

> Expose more executor stats in stable status API
> -----------------------------------------------
>                 Key: SPARK-13269
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>            Reporter: Andrew Or
>            Assignee: Wenchen Fan
>             Fix For: 2.0.0
> Currently the stable status API is quite limited; it exposes only a small subset of the
things exposed by JobProgressListener. It is useful for very high level querying but falls
short when the developer wants to build an application on top of Spark with more integration.
> In this issue I propose that we expose at least two things:
> - Which executors are running tasks, and
> - Which executors cached how much in memory and on disk
> The goal is not to expose exactly these two things, but to expose something that would
allow the developer to learn about them. These concepts are very much fundamental in Spark's
design so there's almost no chance that they will go away in the future.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message