hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6027) Improve /flows API for more flexible filters fromid, collapse, userid
Date Fri, 10 Feb 2017 19:41:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861739#comment-15861739

Sangjin Lee commented on YARN-6027:

Absolutely. Thanks for the reminder.

The idea of supporting the "collapse" here is to combine flow activities from different dates
for the same flow and the user so that they are returned as a single flow activity record
by the reader REST server. While we recognized that this may make rendering certain UI easier
on the UI implementation standpoint, we saw a few problems with that as well.

First, this goes against the design intent on flow activities as "daily activities" of flows
as a fair amount of thought went into designing that table to represent daily activities.
We're not saying since the schema is this way the reader has to return the data that way too
to satisfy the schema. We tried to design the schema to model truly useful concepts and that
way we could accomplish things like filter pushdowns. In that sense, we need to see if we
can support various use cases still based on that intent and design.

Another point was that if we were to introduce something like collapse we would ideally need
to do this in a consistent manner across many/most endpoints. And that definitely means a
large scope of introducing collapse if we do. So we agreed that we would recognize that as
a separate discussion and still move forward with implementing pagination.

Let me know if I missed anything major.

> Improve /flows API for more flexible filters fromid, collapse, userid
> ---------------------------------------------------------------------
>                 Key: YARN-6027
>                 URL: https://issues.apache.org/jira/browse/YARN-6027
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>              Labels: yarn-5355-merge-blocker
>         Attachments: YARN-6027-YARN-5355.0001.patch
> In YARN-5585 , fromId is supported for retrieving entities. We need similar filter for
flows/flowRun apps and flow run and flow as well. 
> Along with supporting fromId, this JIRA should also discuss following points
> * Should we throw an exception for entities/entity retrieval if duplicates found?
> * TimelieEntity :
> ** Should equals method also check for idPrefix?
> ** Does idPrefix is part of identifiers?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message