hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled
Date Thu, 25 Aug 2016 15:21:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437067#comment-15437067
] 

Jason Lowe commented on YARN-5551:
----------------------------------

The "deleted" here refers to the fact that the file path no longer exists, but the mapping
is still valid.  Even though the file path no longer exists the process really is still using
the memory described in that section of the smaps output.  Therefore it is correct to account
for that memory usage against the process.  The storage behind that mapping will _not_ be
freed even though the path has been deleted because this process still has an active mapping
against it.

IMHO this should be closed as invalid.


> Ignore deleted file mapping from memory computation when smaps is enabled
> -------------------------------------------------------------------------
>
>                 Key: YARN-5551
>                 URL: https://issues.apache.org/jira/browse/YARN-5551
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>         Attachments: YARN-5551.branch-2.001.patch
>
>
> Currently deleted file mappings are also included in the memory computation when SMAP
is enabled. For e.g
> {noformat}
> 7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513                 /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185
(deleted)
> Size:                  8 kB
> Rss:                   4 kB
> Pss:                   2 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          4 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            4 kB
> Anonymous:             0 kB
> AnonHugePages:         0 kB
> Swap:                  0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> 7f6123f99000-7f6163f99000 rw-p 00000000 08:41 211419477                  /grid/4/hadoop/yarn/local/usercache/root/appcache/application_1466700718395_1249/container_e19_1466700718395_1249_01_000003/7389389356021597290.cache
(deleted)
> Size:            1048576 kB
> Rss:              637292 kB
> Pss:              637292 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:    637292 kB
> Referenced:       637292 kB
> Anonymous:        637292 kB
> AnonHugePages:         0 kB
> Swap:                  0 kB
> KernelPageSize:        4 kB
> {noformat}
> It would be good to exclude these from getSmapBasedRssMemorySize() computation.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message