hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4700) ATS storage has one extra record each time the RM got restarted
Date Mon, 29 Feb 2016 18:40:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172335#comment-15172335
] 

Vrushali C commented on YARN-4700:
----------------------------------

Hi [~sjlee0]
Yes, the flow activity table's row key always needs to  belong to the top of the day timestamp.
But the event timestamp should be used to find out the top of that day.

bq.  If they meant that we would use the actual event timestamps as is for the row key, I'm
not as sure.
No, we can't use the event timestamp as is. It needs to be top of the day of that timestamp.
 Which is what I said in the previous comment, " the entry for that flow should go into THAT
older day's row, hence we should use the event timestamp." 

You are right, the code in FlowActivityRowKey#getRowKey() needs to change to take the event
timestamp, not the current time. I thought we were sending in null for the timestamp and hence
using current time, but looks like it's directly using current time here. 

{code}
  long dayTs = TimelineStorageUtils.getTopOfTheDayTimestamp(System
        .currentTimeMillis());
{code}



> ATS storage has one extra record each time the RM got restarted
> ---------------------------------------------------------------
>
>                 Key: YARN-4700
>                 URL: https://issues.apache.org/jira/browse/YARN-4700
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Li Lu
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
>
> When testing the new web UI for ATS v2, I noticed that we're creating one extra record
for each finished application (but still hold in the RM state store) each time the RM got
restarted. It's quite possible that we add the cluster start timestamp into the default cluster
id, thus each time we're creating a new record for one application (cluster id is a part of
the row key). We need to fix this behavior, probably by having a better default cluster id.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message