hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3411) [Storage implementation] explore the native HBase write schema for storage
Date Tue, 28 Apr 2015 16:26:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517331#comment-14517331

Vrushali C commented on YARN-3411:

Hi [~zjshen] 

>> For metrics, shall we be more generalized to support all kinds of numeric value,
boolean and so on?
Hmm. So, hbase stores values in bytes. Reading back a value written as a Long using Bytes.toInteger
does not work. Hence, there needs to be a way to know what the data type is going to be. If
metrics will be numbers and booleans, could we say we will always write them as longs? Boolean
values can be represented as 0 and 1 for false and true and all numbers can be written out
as longs, do you think would that be fine? I have to check what [~gtCarrera9]'s patch writes
them as. 

For the config, I went with the background of hadoop configs where in everything is like a
string in the xml, even a number or a ip address etc. 

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, YARN-3411.poc.2.txt, YARN-3411.poc.txt
> There is work that's in progress to implement the storage based on a Phoenix schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native HBase schema
for the write path. Such a schema does not exclude using Phoenix, especially for reads and
offline queries.
> Once we have basic implementations of both options, we could evaluate them in terms of
performance, scalability, usability, etc. and make a call.

This message was sent by Atlassian JIRA

View raw message