hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5551) Ignore deleted file mapping from memory computation when smaps is enabled
Date Thu, 25 Aug 2016 19:33:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437497#comment-15437497
] 

Gopal V edited comment on YARN-5551 at 8/25/16 7:32 PM:
--------------------------------------------------------

bq.  the second has the entire dirty region marked as anonymous

[~nroberts]: good catch - the cache pages were supposed to be private_dirty only - not anon_dirty.


Those allocations were supposed to look the same way, let me fix my cache code and re-run
that on YARN.


was (Author: gopalv):
bq.  the second has the entire dirty region marked as anonymous

[~nroberts]: good catch - the anonymous pages were supposed to be private_dirty only - not
anon_dirty. 

Those allocations were supposed to look the same way, let me fix my cache code and re-run
that on YARN.

> Ignore deleted file mapping from memory computation when smaps is enabled
> -------------------------------------------------------------------------
>
>                 Key: YARN-5551
>                 URL: https://issues.apache.org/jira/browse/YARN-5551
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>         Attachments: YARN-5551.branch-2.001.patch
>
>
> Currently deleted file mappings are also included in the memory computation when SMAP
is enabled. For e.g
> {noformat}
> 7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513                 /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185
(deleted)
> Size:                  8 kB
> Rss:                   4 kB
> Pss:                   2 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          4 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            4 kB
> Anonymous:             0 kB
> AnonHugePages:         0 kB
> Swap:                  0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> 7f6123f99000-7f6163f99000 rw-p 00000000 08:41 211419477                  /grid/4/hadoop/yarn/local/usercache/root/appcache/application_1466700718395_1249/container_e19_1466700718395_1249_01_000003/7389389356021597290.cache
(deleted)
> Size:            1048576 kB
> Rss:              637292 kB
> Pss:              637292 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:    637292 kB
> Referenced:       637292 kB
> Anonymous:        637292 kB
> AnonHugePages:         0 kB
> Swap:                  0 kB
> KernelPageSize:        4 kB
> {noformat}
> It would be good to exclude these from getSmapBasedRssMemorySize() computation.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message