spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-7889) Jobs progress of apps on complete page of HistoryServer shows uncompleted
Date Tue, 26 Jan 2016 14:33:40 GMT

    [ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15117288#comment-15117288
] 

Steve Loughran commented on SPARK-7889:
---------------------------------------

I'm going to note something something problematic about probing HDFS for changes, HDFS-5478
reports that flushing fs data doesn't immediately update fs length. So detecting changes in
log files saved to HDFS isn't likely to be that immediate. the SPARK-1537 timeline binding
won't have this problem, nor will logs saved to other filesystems (object stores have different
issues)

> Jobs progress of apps on complete page of HistoryServer shows uncompleted
> -------------------------------------------------------------------------
>
>                 Key: SPARK-7889
>                 URL: https://issues.apache.org/jira/browse/SPARK-7889
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>            Reporter: meiyoula
>            Priority: Minor
>
> When running a SparkPi with 2000 tasks, cliking into the app on incomplete page, the
job progress shows 400/2000. After the app is completed, the app goes to complete page from
incomplete, and now cliking into the app, the  job progress still shows 400/2000.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message