hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Sun, 16 Aug 2015 09:28:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698602#comment-14698602

Varun Saxena commented on YARN-4053:

This patch demonstrates the approach mentioned above and works for both integral and floating
point values. But for floating point values, the restriction on part of the client is that
it should send values in decimal format always otherwise when I add metric filters, matching
will fail. I guess its a fair enough restriction to place.

In the patch, we can indicate that numerical values have to be stored per column/column prefix.
We can however extend this logic for all values and indicate if values to be stored are ASCII
encoded as well, so that different kind of values can be stored differently in same column.

But there is no use case for this as of now, so haven't done so. 

I will remove the part about floating point numbers from patch, if we dont want it now.

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.

This message was sent by Atlassian JIRA

View raw message