Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 4 Apr 2017 20:47:41 +0000 (UTC)
From: "Karthik Kambatla (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13046774.1488233023000.204572.1491338861668@Atlassian.JIRA>
In-Reply-To: <JIRA.13046774.1488233023000@Atlassian.JIRA>
References: <JIRA.13046774.1488233023000@Atlassian.JIRA> <JIRA.13046774.1488233023116@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (YARN-6245) Add FinalResource object to
 reduce overhead of Resource class instancing
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 04 Apr 2017 20:47:46 -0000


    [ https://issues.apache.org/jira/browse/YARN-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15955780#comment-15955780 ] 

Karthik Kambatla edited comment on YARN-6245 at 4/4/17 8:47 PM:
----------------------------------------------------------------

bq. It works very well so I think we may not need a huge patch to remove ResourcePBImpl from scheduler entirely.
IMO, the scheduler is becoming increasingly complex. Using both the light and proto versions in the scheduler adds to this complexity. I would really like for us to avoid that.

bq. So my question should I submit both changes in one patch?
It might be nice to have one patch focus on replacing proto-based Resource in the scheduler with LightResource. This would include updating the calculators to use and return LightResource. Another patch could optimize for the number of LightResource objects created. Does that sound reasonable, [~roniburd]?

And, the first patch could be here. YARN-6418 can be closed as a duplicate or re-purposed for the other patch. We might also want to update the title and description here to say LightResource instead of FinalResource. 


was (Author: kasha):
bq. It works very well so I think we may not need a huge patch to remove ResourcePBImpl from scheduler entirely.
IMO, the scheduler is becoming increasingly complex. Using both the light and proto versions in the scheduler adds to this complexity. I would really like for us to avoid that.

bq. So my question should I submit both changes in one patch?
It might be nice to have one patch focus on replacing proto-based Resource in the scheduler with LightResource. This would include updating the calculators to use and return LightResource. Another patch could optimize for the number of LightResource objects created. Does that sound reasonable, [~roniburd]?

> Add FinalResource object to reduce overhead of Resource class instancing
> ------------------------------------------------------------------------
>
>                 Key: YARN-6245
>                 URL: https://issues.apache.org/jira/browse/YARN-6245
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>         Attachments: observable-resource.patch, YARN-6245.preliminary-staled.1.patch
>
>
> There're lots of Resource object creation in YARN Scheduler, since Resource object is backed by protobuf, creation of such objects is expensive and becomes bottleneck.
> To address the problem, we can introduce a FinalResource (Is it better to call it ImmutableResource?) object, which is not backed by PBImpl. We can use this object in frequent invoke paths in the scheduler.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org