spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-18298) HistoryServer use GMT time all time
Date Mon, 07 Nov 2016 20:22:58 GMT

    [ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645307#comment-15645307
] 

Sean Owen commented on SPARK-18298:
-----------------------------------

If the dates are stored as longs they don't have any timezone issue to begin with, not at
that level. 
What do you mean here by parsed as CST? I see GMT here everywhere.

> HistoryServer use GMT time all time
> -----------------------------------
>
>                 Key: SPARK-18298
>                 URL: https://issues.apache.org/jira/browse/SPARK-18298
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 2.0.0, 2.0.1
>         Environment: suse 11.3 with CST time
>            Reporter: Tao Wang
>
> When I started HistoryServer for reading event logs, the timestamp readed will be parsed
using local timezone like "CST"(confirmed via debug).
> But the time related columns like "Started"/"Completed"/"Last Updated" in History Server
UI using "GMT" time, which is 8 hours earlier than "CST".
> {quote}
> App ID	App Name	Started	Completed	Duration	Spark User	Last Updated	Event Log
> local-1478225166651	Spark shell	2016-11-04 02:06:06	2016-11-07 01:33:30	71.5 h	root	2016-11-07
01:33:30
> {quote}
> I've checked the REST api and found the result like:
> {color:red}
> [ {
>   "id" : "local-1478225166651",
>   "name" : "Spark shell",
>   "attempts" : [ {
>     "startTime" : "2016-11-04T02:06:06.020GMT",  
>     "endTime" : "2016-11-07T01:33:30.265GMT",  
>     "lastUpdated" : "2016-11-07T01:33:30.000GMT",
>     "duration" : 257244245,
>     "sparkUser" : "root",
>     "completed" : true,
>     "lastUpdatedEpoch" : 1478482410000,
>     "endTimeEpoch" : 1478482410265,
>     "startTimeEpoch" : 1478225166020
>   } ]
> }, {
>   "id" : "local-1478224925869",
>   "name" : "Spark Pi",
>   "attempts" : [ {
>     "startTime" : "2016-11-04T02:02:02.133GMT",
>     "endTime" : "2016-11-04T02:02:07.468GMT",
>     "lastUpdated" : "2016-11-04T02:02:07.000GMT",
>     "duration" : 5335,
>     "sparkUser" : "root",
>     "completed" : true,
> ...
> {color}
> So maybe the change happened in transferring between server and browser? I have no idea
where to go from this point.
> Hope guys can offer some help, or just fix it if it's easy? :)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message