spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bozarth (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-13241) add long--formatted timestamps to org.apache.spark.status.api.v1.ApplicationAttemptInfo
Date Thu, 18 Feb 2016 20:38:18 GMT

    [ https://issues.apache.org/jira/browse/SPARK-13241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153040#comment-15153040
] 

Alex Bozarth edited comment on SPARK-13241 at 2/18/16 8:38 PM:
---------------------------------------------------------------

Looked into this and I could update the api to include this, but before I go through the leg
work, do we actually want it?
[~joshrosen] [~rxin] since you're in charge of the API, is adding the date params as {{long}}
s in {{ApplicationAttemptInfo}} something we actually want to do?
It's definitely useful, but the duplicate data clusters up the api a bit.


was (Author: ajbozarth):
Looked into this and I could update the api to include this, but before I go through the leg
work, do we actually want it?
[~joshrosen] [~rxin] since you're in charge of the API, is adding the date params as {{long}}s
in {{ApplicationAttemptInfo}} something we actually want to do?
It's definitely useful, but the duplicate data clusters up the api a bit.

> add long--formatted timestamps to org.apache.spark.status.api.v1.ApplicationAttemptInfo
> ---------------------------------------------------------------------------------------
>
>                 Key: SPARK-13241
>                 URL: https://issues.apache.org/jira/browse/SPARK-13241
>             Project: Spark
>          Issue Type: Improvement
>          Components: Web UI
>    Affects Versions: 1.6.0
>            Reporter: Steve Loughran
>
> While writing tests to parse {{org.apache.spark.status.api.v1.ApplicationAttemptInfo}}
coming off the history rest server, I can see that I can't actually unmarshall the timestamps
without parsing the strings —and I fear that may be local specific.
> The problem here is that jersey is marshalling {{Date}} classes by calling {{Date.toString()}},
leaving an entry like
> {code}
>   {
>     "id": "application_1111_0000",
>     "name": "spark-demo",
>     "attempts": [
>       {
>         "attemptId": "1",
>         "startTime": "2016-02-08T20:12:20.825GMT",
>         "endTime": "1970-01-01T00:00:00.000GMT",
>         "lastUpdated": "2016-02-08T20:12:20.848GMT",
>         "duration": 0,
>         "sparkUser": "hdfs",
>         "completed": false
>       }
>     ]
>   }
> {code}
> This is good for humans and the web UI, bad for any code trying to use the API for its
own purposes.
> I propose adding, alongside the existing values, a simple {{long}} value for each time,
representing the time in millis from the start of the epoch. This is trivial to marshall/unmarshall.
> This is easy to add, provided everyone is happy that adding a new field to the returned
JSON is considered compatible with the existing REST API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message