Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 25 Aug 2016 19:35:23 +0000 (UTC)
From: "Gopal V (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12999107.1471927346000.413212.1472153723190@Atlassian.JIRA>
In-Reply-To: <JIRA.12999107.1471927346000@Atlassian.JIRA>
References: <JIRA.12999107.1471927346000@Atlassian.JIRA> <JIRA.12999107.1471927346274@arcas>
Subject: [jira] [Comment Edited] (YARN-5551) Ignore deleted file mapping
 from memory computation when smaps is enabled
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 25 Aug 2016 19:35:25 -0000


    [ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437506#comment-15437506 ] 

Gopal V edited comment on YARN-5551 at 8/25/16 7:35 PM:
--------------------------------------------------------

bq. purposes of accounting for how much memory the process is using right now.

The crucial distinction is exactly there. YARN can account memory in two different ways - "how much memory is this process using?" vs "how much memory can I retrieve by killing this process?" [to run other containers in that capacity].

The 2nd question is what should motivate a process kill (btw, in the non-smaps case, the kill is motivated by the first, with no concern for the 2nd).


was (Author: gopalv):
bq. purposes of accounting for how much memory the process is using right now.

The crucial distinction is exactly there. YARN can account memory in two different ways - "how much memory is this process using?" vs "how much memory can I retrieve by killing this process?".

The 2nd question is what should motivate a process kill (btw, in the non-smaps case, the kill is motivated by the first, with no concern for the 2nd).

> Ignore deleted file mapping from memory computation when smaps is enabled
> -------------------------------------------------------------------------
>
>                 Key: YARN-5551
>                 URL: https://issues.apache.org/jira/browse/YARN-5551
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>         Attachments: YARN-5551.branch-2.001.patch
>
>
> Currently deleted file mappings are also included in the memory computation when SMAP is enabled. For e.g
> {noformat}
> 7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513                 /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185 (deleted)
> Size:                  8 kB
> Rss:                   4 kB
> Pss:                   2 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          4 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            4 kB
> Anonymous:             0 kB
> AnonHugePages:         0 kB
> Swap:                  0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> 7f6123f99000-7f6163f99000 rw-p 00000000 08:41 211419477                  /grid/4/hadoop/yarn/local/usercache/root/appcache/application_1466700718395_1249/container_e19_1466700718395_1249_01_000003/7389389356021597290.cache (deleted)
> Size:            1048576 kB
> Rss:              637292 kB
> Pss:              637292 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:    637292 kB
> Referenced:       637292 kB
> Anonymous:        637292 kB
> AnonHugePages:         0 kB
> Swap:                  0 kB
> KernelPageSize:        4 kB
> {noformat}
> It would be good to exclude these from getSmapBasedRssMemorySize() computation.  


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org