hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3411) [Storage implementation] explore the native HBase write schema for storage
Date Wed, 13 May 2015 23:30:01 GMT

     [ https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vrushali C updated YARN-3411:
-----------------------------
    Attachment: YARN-3411.poc.7.txt


Attaching v7 patch. Addresses the review suggestions and includes the latest updates on the
branch to fix the build errors by YARN-3529 .

I ran [~sjlee0]'s driver code in his latest patch in MAPREDUCE-6337 on my pseudo distributed
cluster setup. It seems to be working fine. 

Ran 3 commands: 
{code}
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT-tests.jar
timelineperformance -m 10 -t 1000
{code}

{code}
 bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT-tests.jar
timelineperformance -m 10 -mtype 2 -d /u/tmp/loadtest -r 2
{code}

and 
{code}
 bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT-tests.jar
timelineperformance -m 10 -mtype 2 -d /u/tmp/loadtest -r 1
{code}

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, YARN-3411.poc.2.txt, YARN-3411.poc.3.txt,
YARN-3411.poc.4.txt, YARN-3411.poc.5.txt, YARN-3411.poc.6.txt, YARN-3411.poc.7.txt, YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native HBase schema
for the write path. Such a schema does not exclude using Phoenix, especially for reads and
offline queries.
> Once we have basic implementations of both options, we could evaluate them in terms of
performance, scalability, usability, etc. and make a call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message