spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Yu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-16859) History Server storage information is missing
Date Sun, 11 Mar 2018 17:26:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394580#comment-16394580
] 

Richard Yu edited comment on SPARK-16859 at 3/11/18 5:25 PM:
-------------------------------------------------------------

Is this issue not already resolved?

When looking into {{JsonProtocol}} , I found that {{SparkListenerBlockUpdated}} was already
included as a case in Serialization.

This was in Scala-2.11.

 


was (Author: yohan123):
Is this issue not already resolved?

When looking into {{JsonProtocol}} , I found that {{SparkListenerBlockUpdated}} was already
included as a case in Serialization.

 

> History Server storage information is missing
> ---------------------------------------------
>
>                 Key: SPARK-16859
>                 URL: https://issues.apache.org/jira/browse/SPARK-16859
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.6.2, 2.0.0
>            Reporter: Andrei Ivanov
>            Priority: Major
>              Labels: historyserver, newbie
>
> It looks like job history storage tab in history server is broken for completed jobs
since *1.6.2*. 
> More specifically it's broken since [SPARK-13845|https://issues.apache.org/jira/browse/SPARK-13845].
> I've fixed for my installation by effectively reverting the above patch ([see|https://github.com/EinsamHauer/spark/commit/3af62ea09af8bb350c8c8a9117149c09b8feba08]).
> IMHO, the most straightforward fix would be to implement _SparkListenerBlockUpdated_
serialization to JSON in _JsonProtocol_ making sure it works from _ReplayListenerBus_.
> The downside will be that it will still work incorrectly with pre patch job histories.
But then, it doesn't work since *1.6.2* anyhow.
> PS: I'd really love to have this fixed eventually. But I'm pretty new to Apache Spark
and missing hands on Scala experience. So  I'd prefer that it be fixed by someone experienced
with roadmap vision. If nobody volunteers I'll try to patch myself.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message