hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage
Date Mon, 17 Aug 2015 17:25:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699872#comment-14699872

Li Lu commented on YARN-4053:

Hi [~varun_saxena], thanks for the patch! With regard to the POC, I thought we agreed on the
general plan of the POC on a web UI connected to the timeline reader during our weekly standup
discussion? If this is the case, I would personally give the RESTful API patch slightly higher
priority since that is critical to the whole workflow of the reader/webUI interface? 

About this patch, I totally agree that we should directly store the byte representation of
the numbers instead of using generic object mapper. Having looked at the patch, I have some
general comments here. Maybe what we want here is a way to model the types of the timeline
metrics, so that the type information can be carried over from TimelineMetric objects to the
storage layer? We may have something like TimelineData.FLOAT, TimelineData.LONG, TimelineData.GENERIC_OBJECT,
etc., so that we can easily transfer those messages? My main concern is on the ColumnPrefix
descriptions, where we now use a boolean flag to indicate if the column is numeric or not.
This will also help us to better organize the serialization and deserialization helper methods
and all related tests. Let me know if this idea works here, thanks! 

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
> Currently HBase implementation uses GenericObjectMapper to convert and store values in
backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded
byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them
in HBase.

This message was sent by Atlassian JIRA

View raw message