hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Fri, 14 Aug 2015 19:02:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697556#comment-14697556

Varun Saxena commented on YARN-4053:

bq. What kind of metrics do you have in mind that will have floating point numbers ?
There was some plan for reporting some cluster level metrics in future too, few of them would
be floating point as well. Refer to json in YARN-3881
Also I remember some discussion during aggregation design regarding storing averages. Are
we planning to calculate them on the fly instead ?
Moreover,TimelineMetric stores metric value as a {{java.lang.Number}}. This means we are saying
metric can store a floating point value as well. As we have no control over systems outside
YARN(say Tez), if they use ATS and publish a metric of floating type, I guess we should be
able to handle it.

Thoughts ?

If it has been decided that metrics can only be integral values, then its fine. Wont have
to take care of it then. Let me know.
Also, another key point we need to decide is that do we only support values till signed longs(8
bytes) ? 

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.

This message was sent by Atlassian JIRA

View raw message