hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Fri, 14 Aug 2015 19:20:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697583#comment-14697583

Vrushali C commented on YARN-4053:

Hmm good points. 

I think all metrics should be stored of the same type else we have to deal with knowing which
metric is of which type and would need to store metadata to know how to read it back. Storing
it as an ascii value is not good, we need to be able to query for things like less than greater
than etc. 

My vote is for going with Longs for all metrics right now unless there is a very strong use
case where only decimals will do. We truncate (cast down) decimals to long if we receive any,
so 99.9 means 99. I realize this is restrictive but my thinking is that instead of trying
to do everything for this current ATS release, let's go with Longs and see if we really need
decimal precision. If we do, we can revisit and modify to accept more data types. cc [~jrottinghuis]

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.

This message was sent by Atlassian JIRA

View raw message