hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization
Date Thu, 03 Aug 2017 00:09:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111957#comment-16111957

Haibo Chen commented on YARN-6668:

Thanks [~miklos.szegedi@cloudera.com] for the patch. A few comments:
1) The patch calls the LOG instance declared ResourceCalculatorProcessTree in the new CGroupsResourceCalculator
class. The log can be confusing. Can we use a new LOG instance instead?
2) In CGroupResourceCalculator.isAvailable(), we check ResourceHandlerModule.getCGroupsHandler()
== null to see if cgroup is enabled. Does a null cgroupHandler indicate we have both cpu
and memory cgroup enabled? If not, we need to check if both memory and cpu are enabled.
3) The reason why we need to assign different values to jiffLengthMs if the clock is SystemClock
is not quite clear to me. Can you please explain a little bit?
4) Is the process jiff too large to store in a long instead of BigInteger? Using BigInteger
if long is sufficient seems wasteful.
5) It seems like we update processTotalJiffies in readTotalProcessJiffies() every time before
we read processTotalJiffles. What do you think of having readTotalProcessJiffies return processTotalJiffies?
6) we can use try() statements to simplify and get rid of close() calls. 

> Use cgroup to get container resource utilization
> ------------------------------------------------
>                 Key: YARN-6668
>                 URL: https://issues.apache.org/jira/browse/YARN-6668
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Haibo Chen
>            Assignee: Miklos Szegedi
>         Attachments: YARN-6668.000.patch, YARN-6668.001.patch
> Container Monitor relies on proc file system to get container resource utilization, which
is not as efficient as reading cgroup accounting. We should in NM, when cgroup is enabled,
read cgroup stats instead. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message