hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Wed, 02 Sep 2015 18:38:47 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727816#comment-14727816
] 

Joep Rottinghuis commented on YARN-4053:
----------------------------------------

Part of the issue is that there are many moving components that all need to work together
around properly handling longs and doubles. This is aggregation, store, read, column value
queries, filters, etc.
Tags have challenges pointed out before, and in addition, tags are filtered out on read (you
will not get all the tag values, unless you do voodoo in a coprocessor).

Since we're still figuring out how all these pieces work together in a consistent manner,
our suggestion is to first make this all work together with longs only, and then after that
figure out how to add support for double.
Trying to make things work for both will bog us down right now and we'll likely not get things
working consistently, we simply can't oversee all the consequences and impacts on aggregation,
querying etc.

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
>
>
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message