Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 1 Jun 2016 05:16:13 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12951947.1458541162000.346096.1464758173155@Atlassian.JIRA>
In-Reply-To: <JIRA.12951947.1458541162000@Atlassian.JIRA>
References: <JIRA.12951947.1458541162000@Atlassian.JIRA> <JIRA.12951947.1458541162401@arcas>
Subject: [jira] [Commented] (YARN-4844) Add
 getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 01 Jun 2016 05:16:15 -0000


    [ https://issues.apache.org/jira/browse/YARN-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309277#comment-15309277 ] 

Wangda Tan commented on YARN-4844:
----------------------------------

Thanks [~vvasudev] for review and commit! However, I found a couple of issues for committed patch, I just came back from vacation, sorry for the late response:
- For branch-2. there're some javac warnings, will fix them in addendum patch.
- For branch-2.8, Jenkins doesn't run for the latest patch, will revert and re-attach patch for branch-2.8.

Some additional notes:
- So far, failed unit tests are not related, have tried to run them locally.
- Not sure if Javadocs warnings are related, there're too many unrelated entries in report them, cannot figure out the problem from the report file. Will monitor this and fix Javadocs warning separately.

> Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource
> ---------------------------------------------------------------------
>
>                 Key: YARN-4844
>                 URL: https://issues.apache.org/jira/browse/YARN-4844
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Blocker
>             Fix For: 2.8.0
>
>         Attachments: YARN-4844.1.patch, YARN-4844.10.patch, YARN-4844.11.patch, YARN-4844.12.patch, YARN-4844.13.patch, YARN-4844.14.patch, YARN-4844.15.patch, YARN-4844.16.branch-2.8.patch, YARN-4844.16.branch-2.patch, YARN-4844.16.patch, YARN-4844.2.patch, YARN-4844.3.patch, YARN-4844.4.patch, YARN-4844.5.patch, YARN-4844.6.patch, YARN-4844.7.patch, YARN-4844.8.branch-2.patch, YARN-4844.8.patch, YARN-4844.9.branch, YARN-4844.9.branch-2.patch
>
>
> We use int32 for memory now, if a cluster has 10k nodes, each node has 210G memory, we will get a negative total cluster memory.
> And another case that easier overflows int32 is: we added all pending resources of running apps to cluster's total pending resources. If a problematic app requires too much resources (let's say 1M+ containers, each of them has 3G containers), int32 will be not enough.
> Even if we can cap each app's pending request, we cannot handle the case that there're many running apps, each of them has capped but still significant numbers of pending resources.
> So we may possibly need to add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org