hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4455) Support fetching metrics by time range
Date Tue, 18 Jul 2017 17:28:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091847#comment-16091847

Vrushali C commented on YARN-4455:

Yes, I know we have functions to read the supplemented timestamp... This modified timestamp
was put in for the flow run table for the coprocessor to be distinguish between metrics from
different apps for the same timestamp, as you have mentioned in some comment above. This is
to be used only for the cells in the flow run table.  

I see that you have mentioned "multiplying it by a factor ensures that code path is common
while writing". Did you see any issues while trying to write regular timestamps? I am looking
but do you know where this is a common code path for flowrun table and the application/entity
tables. Is it during reading? 

The reading of metrics for flow run will not be a timeseries since the flow run table will
only keep the latest value of the metric. The reason is that we want only one cell per metric
for the flow run across all apps. Having a timeseries for the flow run is a bit more complicated
since we have to decide what timestamps we want to keep. The coprocessor will intercept any
scan/read (as well as writes) to the table. 

We should not be using the supplemented timestamp for other tables, since it will change the
meaning of the cell timestamp for entity and application tables. The HBase TTL will not work
if we modify the cell timestamp. We would like clean up of old metrics to happen after a certain
expiration period. 

Also note that the TS_MULTIPLIER is 1000000L, not 1000. 

Do you want to have a call to discuss this? I am good with having a call during IST daytime

> Support fetching metrics by time range
> --------------------------------------
>                 Key: YARN-4455
>                 URL: https://issues.apache.org/jira/browse/YARN-4455
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-5355
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: YARN-5355, yarn-5355-merge-blocker
>         Attachments: YARN-4455-YARN-5355.01.patch, YARN-4455-YARN-5355.02.patch, YARN-4455-YARN-5355.03.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message