hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Wed, 27 May 2015 20:21:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561641#comment-14561641

Varun Saxena commented on YARN-3051:

In the API designed in the patch, there are few things I wanted to discuss.

#  We can either return a single timeline entity for a flow ID(having aggregated metric values)
 or multiple entities indicating multiple flows runs for a flow ID. I have included an API
for the former as of now. I think there can be uses cases for both though. [~vrushalic], 
did hRaven have the facility for both kinds of queries ? I mean, is there a known use case
# Do we plan to include additional info in the user table which can be used for filtering
user level entites ? Could not think of any use case but just for flexibility I have added
filters in the API {{getUserEntities}}.
# I have included an API to query flow information based on the appid. As of now I return
the flow to which app belongs to(includes multiple runs) instead of flow run it belongs to.
Which is a more viable scenario ? Or we need to support both ?
# In the HBase schema design, there are 2 flow summary tables aggregated daily and weekly
respectively. So to limit the number of metric records or to see metrics in a specific time
window, I have added metric start and metric end timestamps in the API design. But if  metrics
are aggregated daily and weekly, we wont be able to get something like value of specific metric
for a flow from say Thursday 4 pm to Friday 9 am. [~vrushalic], can you confirm ? If this
is so, a timestamp doesnt make much sense. Dates can be specified instead.
# Will there be queue table(s) in addition to user table(s) ? If yes, how will queue data
be aggregated ? Based on entity type ? I may need an additional API for queues then.
# The doubt I have regarding flow version will anyways be addressed by YARN-3699

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, YARN-3051-YARN-2928.03.patch, YARN-3051.wip.02.YARN-2928.patch,
YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.

This message was sent by Atlassian JIRA

View raw message