hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4700) ATS storage has one extra record each time the RM got restarted
Date Wed, 02 Mar 2016 01:05:18 GMT

     [ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Naganarasimha G R updated YARN-4700:
    Attachment: YARN-4700-YARN-2928.v1.002.patch

Thanks for the comments [~varun_saxena] and [~vrushalic],
bq. I believe the constructor for FlowActivityRowKey should change to correctly calculate
top of the day timestamp given the input timestamp. 
As [~varun_saxena] mentioned ??FlowActivityRowKey constructor is used while parsing row key
so I don't think we should be changing that?? and without correcting it, tests passed .
bq.  It might be more explicit to fetch the exact created (or finished) event from the TimelineEntity
and use the timestamp that belong to either ApplicationMetricsConstants.CREATED_EVENT_TYPE
I have refactored quiet a bit for this to avoid the looping of events at multiple places.
Please check.
I have corrected all the other comments by correcting the time stamps

> ATS storage has one extra record each time the RM got restarted
> ---------------------------------------------------------------
>                 Key: YARN-4700
>                 URL: https://issues.apache.org/jira/browse/YARN-4700
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Li Lu
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4700-YARN-2928.v1.001.patch, YARN-4700-YARN-2928.v1.002.patch,
> When testing the new web UI for ATS v2, I noticed that we're creating one extra record
for each finished application (but still hold in the RM state store) each time the RM got
restarted. It's quite possible that we add the cluster start timestamp into the default cluster
id, thus each time we're creating a new record for one application (cluster id is a part of
the row key). We need to fix this behavior, probably by having a better default cluster id.

This message was sent by Atlassian JIRA

View raw message