hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3551) Consolidate data model change according to the backend implementation
Date Wed, 29 Apr 2015 02:07:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14518569#comment-14518569
] 

Vrushali C commented on YARN-3551:
----------------------------------


Thanks [~zjshen] for the jira and the patch. 

I was wondering about the TimelineMetric class. Do you think it might sound better if we call
it latestData or mostRecentData instead of singleData? 

Also, think along with singleData, we also need a timestamp member variable, else we would
not know which timestamp this singleData value belongs to.

Another way of doing this would be to keep adding to the time series member of TimelineMetric
and modify the write API to write only the latest metric value. I am wondering if this may
actually be better than creating a timeline entity object with a single metric value each
time. Metrics are updated very frequently and  for long running apps, we will end up with
a lot of object creation churn. 

> Consolidate data model change according to the backend implementation
> ---------------------------------------------------------------------
>
>                 Key: YARN-3551
>                 URL: https://issues.apache.org/jira/browse/YARN-3551
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-3551.1.patch, YARN-3551.2.patch, YARN-3551.3.patch
>
>
> Based on the comments on [YARN-3134|https://issues.apache.org/jira/browse/YARN-3134?focusedCommentId=14512080&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14512080]
and [YARN-3411|https://issues.apache.org/jira/browse/YARN-3411?focusedCommentId=14512098&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14512098],
we need to change the data model to restrict the data type of info/config/metric section.
> 1. Info: the value could be all kinds object that is able to be serialized/deserialized
by jackson.
> 2. Config: the value will always be assumed as String.
> 3. Metric: single data or time series value have to be number for aggregation.
> Other than that, info/start time/finish time of metric seem not to be necessary for storage.
They should be removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message