Return-Path: 
 <yarn-issues-return-73746-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 603CE18066
	for <apmail-hadoop-yarn-issues-archive@minotaur.apache.org>;
 Thu, 19 Nov 2015 02:04:11 +0000 (UTC)
Received: (qmail 85580 invoked by uid 500); 19 Nov 2015 02:04:11 -0000
Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org
Received: (qmail 85532 invoked by uid 500); 19 Nov 2015 02:04:11 -0000
Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:yarn-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:yarn-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:yarn-issues@hadoop.apache.org>
List-Id: <yarn-issues.hadoop.apache.org>
Reply-To: yarn-issues@hadoop.apache.org
Delivered-To: mailing list yarn-issues@hadoop.apache.org
Received: (qmail 85512 invoked by uid 99); 19 Nov 2015 02:04:11 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Nov 2015 02:04:11 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 075F32C1F6B
	for <yarn-issues@hadoop.apache.org>; Thu, 19 Nov 2015 02:04:11 +0000 (UTC)
Date: Thu, 19 Nov 2015 02:04:11 +0000 (UTC)
From: "Sangjin Lee (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12856230.1439574222000.116289.1447898651028@Atlassian.JIRA>
In-Reply-To: <JIRA.12856230.1439574222000@Atlassian.JIRA>
References: <JIRA.12856230.1439574222000@Atlassian.JIRA>
 <JIRA.12856230.1439574222728@arcas>
Subject: [jira] [Commented] (YARN-4053) Change the way metric values are
 stored in HBase Storage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012620#comment-15012620 ] 

Sangjin Lee commented on YARN-4053:
-----------------------------------

Yes, I spoke with [~jrottinghuis] regarding the stronger type safety suggestion. I'm now more or less comfortable with the current version. The additional type safety you get is probably minimal, and can cause bit more headache as a side effect as Joep mentioned.

Just for the record, I tinkered with this a little bit, and there seems to be a way to do this safely. The assumption is {{add()}} and {{compare()}} are invoked only on the return values of {{decodeValue()}}. That is currently the case, but it might not always be true. In any case, if that were always true, we could do

{code}
public interface ValueConverter<T> {
  byte[] encodeValue(Object value) throws IOException; // this value is still object
  T decodeValue(byte[] bytes) throws IOException;
}

public interface NumericValueConverter<T extends Number>
    extends ValueConverter<T>, Comparator<T> {
  T add(T num1, T num2);
}

public class LongConverter implements NumericValueConverter<Long> {
  ...
}
{code}

This is basically tying the loop from {{decodeValue()}} and {{add()}} and {{compare()}}. Again, this is a mild suggestion at this point, and it works only if the assumption that {{add()}} and {{compare()}} work only on the result of {{decodeValue()}} is true.

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4053-YARN-2928.01.patch, YARN-4053-YARN-2928.02.patch, YARN-4053-feature-YARN-2928.03.patch, YARN-4053-feature-YARN-2928.04.patch, YARN-4053-feature-YARN-2928.05.patch
>
>
> Currently HBase implementation uses GenericObjectMapper to convert and store values in backend HBase storage. This converts everything into a string representation(ASCII/UTF-8 encoded byte array).
> While this is fine in most cases, it does not quite serve our use case for metrics. 
> So we need to decide how are we going to encode and decode metric values and store them in HBase.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)