hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4179) [reader implementation] support flow activity queries based on time
Date Thu, 15 Oct 2015 19:08:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959439#comment-14959439
] 

Varun Saxena commented on YARN-4179:
------------------------------------

[~sjlee0]
bq. As I mentioned in an earlier comment, can we simply use JDK's date formatter for this?
It would be good to avoid adding a new dependency unless it is absolutely necessary.
JAVAs' {{SimpleDateFormat}} is not thread safe. Apache commons-lang's {{FastDateFormat}} claims
to be 25% faster too. Used commons-lang3 because commons-lang's {{FastDateFormat}} does not
provide date parsing capability.
If we you want to avoid adding a dependency and the we will use a single date format, I can
write a custom method to do date parsing as well and convert it into seconds since epoch.

bq. now the question of a more "standard" date format can be highly subjective, but how about
using "yyyyMMdd"?
Ok. We can go with this format.

bq. instead of duplicating the date format here, we should simply reuse what's defined in
TimelineReaderWebServices
Ok.

bq. Why is this conversion from long to Date needed?
When Date is converted to JSON, it is represented as a long. Hence when JSON parsing is done
at the client side, {{getInfo().get(DATE_INFO_KEY)}} returns a long. That is why the conversion.

bq. let's use a normal java predicate style date != null
Ok.

bq.  nit: parseDate can return primitive long
Ok.

Regarding your other comments, I will add a javadoc. 
And I have chosen a single query param daterange(delimited by "-") i.e. a specific date or
a range. 
If we want to specify a startdate and enddate we will need 2 query params. If startdate is
not specified, every date starting from 1970 till enddate can be taken(constrained by limit)
and if enddate isnt specified every date from startdate till today can be taken. Do you want
this approach ?

> [reader implementation] support flow activity queries based on time
> -------------------------------------------------------------------
>
>                 Key: YARN-4179
>                 URL: https://issues.apache.org/jira/browse/YARN-4179
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>            Priority: Minor
>         Attachments: YARN-4179-YARN-2928.01.patch
>
>
> This came up as part of YARN-4074 and YARN-4075.
> Currently the only query pattern that's supported on the flow activity table is by cluster
only. But it might be useful to support queries by cluster and certain date or dates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message