hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2928) YARN Timeline Service: Next generation
Date Tue, 09 Jun 2015 17:50:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14579305#comment-14579305

James Taylor commented on YARN-2928:

bq. I'm wondering that, when adding the dynamic columns into a view, do I still need to explicitly
claim those dynamic columns (I assume yes but would like to double check)?
Yes - instead of building up the SQL string with the dynamic column name, you'd execute the
ALTER VIEW my_view ADD IF NOT EXISTS <dynamic column name> <dynamic column type>

Then, when you query, you no longer need to use dynamic columns, but instead can select all
of them:
SELECT * FROM my_view;

As far as APIs, we'll be happy to give you the ones you need, [~gtCarrera9]. The higher up
in the stack you hook in, the easier it'll be. For reading from HBase, you can always fallback
to creating a read-only view over your HBase table. We should work through a couple of examples,
though, as if you're storing multiple pieces of information in your row key, we'll want to
make sure it's compatible with the way Phoenix expects it to be structured.

For writing to HBase, I think it'd be good to re-test the Phoenix write path once I finish
PHOENIX-2028 (with your KeyPrefixRegionSplitPolicy installed on the table). If it's still
not fast enough, then there are a number of options:
- Use PDataType.toBytes() to get the KeyValue value bytes
- Use PhoenixRuntime APIs to create the row key if they encapsulate multiple pieces of information
- Create new APIs as needed

> YARN Timeline Service: Next generation
> --------------------------------------
>                 Key: YARN-2928
>                 URL: https://issues.apache.org/jira/browse/YARN-2928
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>            Priority: Critical
>         Attachments: ATSv2.rev1.pdf, ATSv2.rev2.pdf, Data model proposal v1.pdf, Timeline
Service Next Gen - Planning - ppt.pptx, TimelineServiceStoragePerformanceTestSummaryYARN-2928.pdf
> We have the application timeline server implemented in yarn per YARN-1530 and YARN-321.
Although it is a great feature, we have recognized several critical issues and features that
need to be addressed.
> This JIRA proposes the design and implementation changes to address those. This is phase
1 of this effort.

This message was sent by Atlassian JIRA

View raw message