hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5627) [Atsv2] Support streaming reader API to fetch entities
Date Wed, 14 Mar 2018 22:35:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399549#comment-16399549
] 

Haibo Chen commented on YARN-5627:
----------------------------------

Thanks [~rohithsharma] for the pointer. Trying to understand FROM_ID better. Say an application
has stored 1000 custom entities of the same entity type in the EntityTable. Hence the table
looks like this:

 
{code:java}
user1/clusterId/flowName/flowRunId/AppId/TYPE_A/e0001

user1/clusterId/flowName/flowRunId/AppId/TYPE_A/e0002

user1/clusterId/flowName/flowRunId/AppId/TYPE_A/e0003

...

user1/clusterId/flowName/flowRunId/AppId/TYPE_A/e1000

{code}
 

How would a REST client make use of FROM_ID to retrieve 100 entities at a time? My current
understanding is that fromId is always used along with limit, and fromid is inclusive. That
should support server side pagination in theory. But the rest client needs to know what it
the first fromid, right?

> [Atsv2] Support streaming reader API to fetch entities
> ------------------------------------------------------
>
>                 Key: YARN-5627
>                 URL: https://issues.apache.org/jira/browse/YARN-5627
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Major
>
> There is no limit for size of TimelineEntitie object. It can be varied from KB's to MB.
While reading entities list, it would be an potential issue that TimeLineReder would go into
OOM situation based on the entity size and limit. 
> Proposal is to support an streaming API to read entities list. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message