hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Add a new filter fromId in REST endpoints
Date Wed, 07 Sep 2016 04:39:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469513#comment-15469513
] 

Li Lu commented on YARN-5585:
-----------------------------

Thanks [~rohithsharma]! The approach is generally fine, however, I have some confusions:
1. I'm a little bit confused by the usage of Hbase filters here. When performing fromId, we
create a filter like this:
{code}
    Filter singleColValFilterStart = createHBaseSingleColValueFilter(
        column.getColumnFamilyBytes(), column.getColumnQualifierBytes(),
        column.getValueConverter().encodeValue(startValue),
        CompareOp.LESS, true);
{code}
Are we selecting entities whose ID is less than start value, or we're filtering them out?
According to your description fromId = app-5 should return something like app-6 to 10, right?
I think it's very important to clearly define the exact meaning of "fromId"? 

2. Because we're selecting entities starting from a given ID, can we directly pass in the
fromID's key when creating the scan? In this way seems like we saved one filter? For example,
if fromId is not provided, we may want to scan from cluster!user!flow!flowrun!appId!type,
but if fromId is provided, we can start from cluster!user!flow!flowrun!appId!type!fromId (or
the next available entity)? 

3. For pagination on containers, why do we need to care about actual creation time when the
entity ids have already been sorted? This said, supporting paginations for generic timeline
entities should not be blocked by YARN-5094? 

> [Atsv2] Add a new filter fromId in REST endpoints
> -------------------------------------------------
>
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>         Attachments: YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-10. But to retrieve next 5 apps, it is difficult.
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> This is very useful for pagination in web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message