hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Fri, 28 Aug 2015 01:12:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717863#comment-14717863
] 

Sangjin Lee commented on YARN-4053:
-----------------------------------

Thanks [~varun_saxena] for the discussion. As you said, one thing that really causes issues
is when inconsistent values are used for the same metric. At a high level, I think we need
to ask these questions:

- How important is it to support this scenario?
- If we don't really support this scenario, then what is the minimally acceptable behavior
if that were to happen?

The gist of the problem is that one cannot really write/read consistent values without knowing
the "right" type of the metric. The user will likely not know that either for the write or
read path. In the face of this, the main difference between approach #1 (encoding it into
the value) and approach #2 (adding it to the column qualifier) is that approach #1 will mix
different-type values into a single time series (column), and approach #2 will effectively
create two separate time series (columns). The rest is the fallout.


> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
>
>
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message