hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12356) CPU usage statistics on Windows
Date Thu, 17 Dec 2015 21:04:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062809#comment-15062809

Wangda Tan commented on HADOOP-12356:

Hi [~elgoiri]/[~cnauroth],

Looked at patch and discussions:

bq. Inigo Goiri, my understanding of the resource monitoring code at this layer is that it
needs to track utilization as a percentage, not in units of vcores. I'd also be concerned
that changing this calculation could cause unusual effects during a rolling upgrade scenario,
when some NodeManagers are running the old calculation and some NodeManagers are running the
new calculation. I expect Wangda can give us a more definitive answer.
Since the ResourceUtilization is in 2.8.0 only,  I think the rolling upgrade is not an issue.

I'm wondering if we have different behaviors in SysInfoLinux and SysInfoWindows to get CPU
- Linux normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated
usage after normalization is 50%.
- Windows is not normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned
accumulated usage is 400%.

I think we should make sure the two SysInfo implementation has consistency behavior, maybe
we need add the behavior description to SysInfo#getCPUUsage java docs.

I'm fine with the change of NodeResourceMonitorImpl if SysInfoWindows/Linux both normalized
usage to 0-100%.

> CPU usage statistics on Windows
> -------------------------------
>                 Key: HADOOP-12356
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12356
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>         Environment: CPU: Intel Xeon
> OS: Windows server
>            Reporter: Yunqi Zhang
>            Assignee: Yunqi Zhang
>              Labels: easyfix, newbie, patch
>         Attachments: 0001-Correct-the-CPU-usage-calcualtion.patch, 0001-Correct-the-CPU-usage-calcualtion.patch,
>   Original Estimate: 2h
>  Remaining Estimate: 2h
> The CPU usage information on Windows is computed incorrectly. The proposed patch fixes
the issue, and unifies the the interface with Linux.

This message was sent by Atlassian JIRA

View raw message