hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6850) Ensure that supplemented timestamp is stored only for flow run metrics
Date Fri, 21 Jul 2017 11:59:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16096166#comment-16096166

Varun Saxena commented on YARN-6850:

Thanks [~vrushalic] for the review.

bq. In case someone wants all versions for this metric, how would they do it without knowing
the boundary? 
We have another parameter, namely metricslimit which controls this. This is analogous to created
time range and limit query params which restrics the number of entities to return. 
You can give a high value to fetch all the records (if not limited by metricstimebegin and
metricstimeend). The check displayed above is primarily for the case where metricsTimeBegin/End
are not specified as query param. If both these params are null we change them to 0 and Long#MAX_VALUE

bq. If we don't want to update the documentation here, could you add in a note in the documentation
jira so that we don't miss this.
Ok. Will do so.

> Ensure that supplemented timestamp is stored only for flow run metrics
> ----------------------------------------------------------------------
>                 Key: YARN-6850
>                 URL: https://issues.apache.org/jira/browse/YARN-6850
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Varun Saxena
>              Labels: yarn-5355-merge-blocker
>         Attachments: YARN-6850-YARN-5355.01.patch
> In timeline service v2,  ColumnHelper#getPutTimestamp supplements the timestamp and is
called by ColumnHelper#store. This is not conditional and called for every put.
> We need to ensure that the cell timestamps for metrics in entity and application (and
sub application) tables are "correct" timestamps since we will be enabling TTLs for these
> The supplemented timestamp is to be used only in the flow run table by the coprocessor
which intercepts all reads & writes to cells in this table. It looks at the supplemented
timestamp to figure out which app id this particular cell belongs to. This is done in order
to ensure no collision occurs when two apps belonging to same flow run write the same metric
at the same timestamp. 
> Discovered in the discussion in YARN-4455 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message