hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Fri, 13 Nov 2015 21:12:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004727#comment-15004727
] 

Varun Saxena commented on YARN-4053:
------------------------------------

bq. Would it be safe to always assume a long converter?
I think while processing it would be safe to assume with current columns and converters. The
issue comes if somebody changes the converter corresponding to column/column prefix in future.
And possibly forgets to change here. Arguably though its difficult to miss such a change after
testing.

The reason this iterating over all columns/column prefixes was added was to handle future
column additions or change in converter for existing columns.

Getting the converter on every iteration was a concern even for me. That's why I am doing
it only when column qualifier changes while iterating cells.
Not sure if we can get hold of all possible column qualifiers(and hence the converter) for
this scan in the constructor.

Another option would be to use cell tags to identify converters.




> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4053-YARN-2928.01.patch, YARN-4053-YARN-2928.02.patch, YARN-4053-feature-YARN-2928.03.patch
>
>
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message