hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6245) Add FinalResource object to reduce overhead of Resource class instancing
Date Mon, 03 Apr 2017 22:11:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954232#comment-15954232
] 

Karthik Kambatla commented on YARN-6245:
----------------------------------------

I think I understand some more now. :)

Now, I see how the LightResource helps with reducing the overhead of instantiating Resource
by doing away with protobuf. And, it should address the requirement in the JIRA title and
description? 

Where would we need FinalResource? IMO, we need it in cases where we return a Resource but
don't expect the caller to modify the returned Resource object. Are there other cases? 

The difference between the FinalResource and ObservableResource is: the former holds a snapshot
of whatever value we are returning and the latter is just an immutable pointer to a Resource
object and shows the latest value. Both could be useful, and both could be light. Do we need
either or both of them? 

> Add FinalResource object to reduce overhead of Resource class instancing
> ------------------------------------------------------------------------
>
>                 Key: YARN-6245
>                 URL: https://issues.apache.org/jira/browse/YARN-6245
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>         Attachments: observable-resource.patch, YARN-6245.preliminary-staled.1.patch
>
>
> There're lots of Resource object creation in YARN Scheduler, since Resource object is
backed by protobuf, creation of such objects is expensive and becomes bottleneck.
> To address the problem, we can introduce a FinalResource (Is it better to call it ImmutableResource?)
object, which is not backed by PBImpl. We can use this object in frequent invoke paths in
the scheduler.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message