Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Wed, 2 Sep 2015 17:03:45 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12856230.1439574222000.237363.1441213425759@Atlassian.JIRA>
In-Reply-To: <JIRA.12856230.1439574222000@Atlassian.JIRA>
References: <JIRA.12856230.1439574222000@Atlassian.JIRA>
 <JIRA.12856230.1439574222728@arcas>
Subject: [jira] [Commented] (YARN-4053) Change the way metric values are
 stored in HBase Storage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727641#comment-14727641 ] 

Varun Saxena commented on YARN-4053:
------------------------------------

bq. to use cell tag identify metrics value type
In YARN-3816, by metric value type I meant whether its aggregated metric or not. I guess that is what you were referring to. Cell tags can be useful that way. But identifying the metric's data type by cell tag wont work in our case because we have to apply metric filters.

IMHO, I think forcing user to send consistent values is a fair enough expectation. 
If user doesnt send, they will get inconsistent results.

If we go by this premise, I think we can go ahead with Longs and Doubles.
Otherwise, we can assume all values as long.

The point which Vrushali raised above regarding floating point precision not being that important. Well, as I point out in last Wednesday's meeting and the example which [~djp] gives, floating points may not matter for very large values but they would matter for metrics whose range is towards the lower end(say, 0 to 5).
Now the real question is do we expect such metrics ? 

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
>
>
> Currently HBase implementation uses GenericObjectMapper to convert and store values in backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them in HBase.
>  


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)