spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <>
Subject [jira] [Commented] (SPARK-6418) Add simple per-stage visualization to the UI
Date Fri, 20 Mar 2015 21:23:38 GMT


Sandy Ryza commented on SPARK-6418:

I think this would be a great addition.  One note is that I think it might be easier to interpret
if the results were sorted by start time.  If possible it would also be really helpful to
somehow give an indication of which tasks ran on which nodes.  And to have task metrics show
when you hover over or something. And to have filtering.  And pagination.  Ok I'm getting
carried away, but would be great to see something like this in.

> Add simple per-stage visualization to the UI
> --------------------------------------------
>                 Key: SPARK-6418
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Web UI
>            Reporter: Kay Ousterhout
>            Assignee: Pradyumn Shroff
>         Attachments: Screen Shot 2015-03-18 at 6.13.04 PM.png
> Visualizing how tasks in a stage spend their time can be very helpful to understanding
performance.  Many folks have started using the visualization tools here:
(see the README at the bottom) to analyze their jobs after they've finished running, but it
would be great if this functionality were natively integrated into Spark's UI.
> I'd propose adding a relatively simple visualization to the stage detail page, that's
hidden by default but that users can view by clicking on a drop-down menu.  The plan is to
implement this using D3; a mock up of how this would look (that uses D3) is attached.
> This is intended to be a much simpler and more limited version of SPARK-3468

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message