hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1167) Make ProcfsBasedProcessTree collect rss memory information
Date Thu, 12 Nov 2009 06:47:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod K V updated MAPREDUCE-1167:

    Status: Open  (was: Patch Available)

The patch looks OK. Only minor comments:
 - ProcessTreeDump is printing Rss size in pages but the header reads RSSMEM_USAGE(BYTES).
This should be fixed.
 - In TestProcfsBasedProcessTree, Long.ParseLong(String) is used in many places. This is costly,
you can use the long value directly if long type is needed, otherwise Long.valueOf(long) if
Long type is. For example, see +408 after applying your patch.
 - TestProcfsBasedProcessTree failed. You need to modify the pattern at TestProcfsBasedProcessTree.java

bq. The reason for 3 is because getCumulativeVmem and getCumulativeRssmem should be consistent.
When /proc/ is not available, getCumulativeVmem will return 0 instead of throwing Exception.
I think this was wrongly done and should be changed. Will file a new issue. We can keep whatever
you've done for now.

> Make ProcfsBasedProcessTree collect rss memory information
> ----------------------------------------------------------
>                 Key: MAPREDUCE-1167
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1167
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>         Attachments: MAPREDUCE-1167-v2.patch, MAPREDUCE-1167-v3.patch, MAPREDUCE-1167-v4.patch,
> Right now ProcfsBasedProcess collects only virtual memory. We can make it collect rss
memory as well.
> Later we can use rss in TaskMemoryManagerThread to obtain better memory management.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message