hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3411) [Storage implementation] explore the native HBase write schema for storage
Date Tue, 19 May 2015 21:14:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551238#comment-14551238
] 

Li Lu commented on YARN-3411:
-----------------------------

Hi [~vrushalic], thanks for working on this! I just got back and looked at the latest (v6)
patch. In general I think it's quite close. I just have a few quick comments. 

# I notice you made {{EntityColumnFamilyDetails}} and {{EntityInfoColumnDetails}} to be public.
Even though we're adding @Private tags to them, we may still want to use the strictest access
modifier possible. If they're fine to be visible to package only, maybe we can use default,
rather than public for them? (I tried with my local version and seems like they worked fine,
but not sure if I missed anything. )
# I share the same concern as [~zjshen] on schema creation. Having a separate step to set
up the HBase data schema appears to add some workload to Hadoop deployments. So, although
it's totally fine to have this step for our performance benchmark, it will be great to keep
this in mind when we consolidate the storage part. 
# Maybe it's helpful to move the entity creation steps of TestHBaseTimelineWriterImpl to TestTimelineWriterImpl?
In this way we can improve test coverage for both writer implementations after this work.


> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, YARN-3411-YARN-2928.001.patch,
YARN-3411-YARN-2928.002.patch, YARN-3411-YARN-2928.003.patch, YARN-3411-YARN-2928.004.patch,
YARN-3411-YARN-2928.005.patch, YARN-3411-YARN-2928.006.patch, YARN-3411.poc.2.txt, YARN-3411.poc.3.txt,
YARN-3411.poc.4.txt, YARN-3411.poc.5.txt, YARN-3411.poc.6.txt, YARN-3411.poc.7.txt, YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native HBase schema
for the write path. Such a schema does not exclude using Phoenix, especially for reads and
offline queries.
> Once we have basic implementations of both options, we could evaluate them in terms of
performance, scalability, usability, etc. and make a call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message