hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6027) Improve /flows API for more flexible filters fromid, collapse, userid
Date Tue, 07 Feb 2017 17:50:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856413#comment-15856413
] 

Varun Saxena commented on YARN-6027:
------------------------------------

Thanks [~rohithsharma] for the patch. Few high level comments. Will take a detailed look at
the patch a little later.
# Do we need cluster ID in fromId because we are ignoring it completely?
# If there is a / in cluster ID we may have to escape it to avoid parsing errors.
# If we use collapse, even with fromId, there seems to be a full table scan which will impact
the run time of this query. Maybe we can send the last real ID in info field of last flow
activity entity if previous query was made with collapse field.  UI can then send this ID
and we can use it to make the query from that specific row instead of having a full table
scan.
# In processResults method you have mentioned that fromId validation is happening in getResult
method. Could not find it.
# In processResults we first get the result from backend while applying limit and then process
result for collapse and fromId filters. In this case we may return less records than limit
even if they are available. Because some entities maybe skipped due to fromId check. And even
for collapse we will merge entities with same flow and user which means we will return entities
less than limit. If we want behavior limit in this case to be different, we should document
it.

> Improve /flows API for more flexible filters fromid, collapse, userid
> ---------------------------------------------------------------------
>
>                 Key: YARN-6027
>                 URL: https://issues.apache.org/jira/browse/YARN-6027
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>              Labels: yarn-5355-merge-blocker
>         Attachments: YARN-6027-YARN-5355.0001.patch
>
>
> In YARN-5585 , fromId is supported for retrieving entities. We need similar filter for
flows/flowRun apps and flow run and flow as well. 
> Along with supporting fromId, this JIRA should also discuss following points
> * Should we throw an exception for entities/entity retrieval if duplicates found?
> * TimelieEntity :
> ** Should equals method also check for idPrefix?
> ** Does idPrefix is part of identifiers?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message