hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3411) [Storage implementation] explore the native HBase write schema for storage
Date Fri, 24 Apr 2015 16:46:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511323#comment-14511323
] 

Junping Du commented on YARN-3411:
----------------------------------

Hi [~vrushalic], thanks for updating the patch! Just a quick go through for the latest patch,
sounds like we have ruby code to create schema:
{code}
+create 'ats.entity', 
+  {NAME => 'i', COMPRESSION => 'LZO', BLOOMFILTER => 'ROWCOL'},
+  {NAME => 'm', VERSIONS => 2147483647, MIN_VERSIONS => 1, COMPRESSION => 'LZO',
BLOCKCACHE => false, TTL => '2592000'},
+  {NAME => 'c', COMPRESSION => 'LZO', BLOCKCACHE => false, BLOOMFILTER => 'ROWCOL'
}
{code} 
Can we turn this into java code? As we haven't add any ruby code before, it could bring extra
complexity/dependency on ruby.

BTW, I meet the problem to build locally with applying the patch. I think the reason seems
like my local use JDK 1.8, but HBase version we are using here (1.0.0) depends on jdk tools
1.7. I assume we will move to Java 8 in short term (may be in 2.8 release cycle? - need confirm
later). If so, we may need solution for this problem.

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, YARN-3411.poc.2.txt, YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native HBase schema
for the write path. Such a schema does not exclude using Phoenix, especially for reads and
offline queries.
> Once we have basic implementations of both options, we could evaluate them in terms of
performance, scalability, usability, etc. and make a call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message