hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6027) Support fromId for flows API
Date Wed, 11 Jan 2017 05:54:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15817288#comment-15817288

Vrushali C commented on YARN-6027:

bq. How many do we expect typically ? Can it run into thousands ? 
So, let's see. Say, something is running every 5 mins on the cluster and let's say it completes
in less than 5 mins, meaning we have at least one hadoop job every 5 mins, then that is 60
* 24 / 5 = 288. So much less than a thousand. I think it's reasonable to think it won't be
a thousand runs in a day, unless someone is being malicious (triggering a hadoop job every
min is 1440 runs). 

That said, what do we think might break or be a problem if we do end up having >1000 runs
in one flow in a day? Over time, we would have thousands of runs for that flow. 

bq. I had raised a JIRA to limit flow runs within a flow. We should probably have that support
Hmm. Is that YARN-4489? In general, it is good idea to limit the number of records being returned,
regardless of what is being returned, be it flows, entities, applications, anything really.
Unless the user explicitly asks for more or disables the limit by setting it to say -1, we
should consider having a limit on payload being returned. 

> Support fromId for flows API 
> -----------------------------
>                 Key: YARN-6027
>                 URL: https://issues.apache.org/jira/browse/YARN-6027
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>              Labels: yarn-5355-merge-blocker
> In YARN-5585 , fromId is supported for retrieving entities. We need similar filter for
flows/flowRun apps and flow run and flow as well. 
> Along with supporting fromId, this JIRA should also discuss following points
> * Should we throw an exception for entities/entity retrieval if duplicates found?
> * TimelieEntity :
> ** Should equals method also check for idPrefix?
> ** Does idPrefix is part of identifiers?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message