Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Thu, 17 Dec 2015 21:04:46 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12858698.1440524133000.58915.1450386286692@Atlassian.JIRA>
In-Reply-To: <JIRA.12858698.1440524133000@Atlassian.JIRA>
References: <JIRA.12858698.1440524133000@Atlassian.JIRA>
 <JIRA.12858698.1440524133128@arcas>
Subject: [jira] [Commented] (HADOOP-12356) CPU usage statistics on Windows
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062809#comment-15062809 ] 

Wangda Tan commented on HADOOP-12356:
-------------------------------------

Hi [~elgoiri]/[~cnauroth],

Looked at patch and discussions:

bq. Inigo Goiri, my understanding of the resource monitoring code at this layer is that it needs to track utilization as a percentage, not in units of vcores. I'd also be concerned that changing this calculation could cause unusual effects during a rolling upgrade scenario, when some NodeManagers are running the old calculation and some NodeManagers are running the new calculation. I expect Wangda can give us a more definitive answer.
Since the ResourceUtilization is in 2.8.0 only,  I think the rolling upgrade is not an issue.

I'm wondering if we have different behaviors in SysInfoLinux and SysInfoWindows to get CPU usage:
- Linux normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated usage after normalization is 50%.
- Windows is not normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated usage is 400%.

I think we should make sure the two SysInfo implementation has consistency behavior, maybe we need add the behavior description to SysInfo#getCPUUsage java docs.

I'm fine with the change of NodeResourceMonitorImpl if SysInfoWindows/Linux both normalized usage to 0-100%.

> CPU usage statistics on Windows
> -------------------------------
>
>                 Key: HADOOP-12356
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12356
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>         Environment: CPU: Intel Xeon
> OS: Windows server
>            Reporter: Yunqi Zhang
>            Assignee: Yunqi Zhang
>              Labels: easyfix, newbie, patch
>         Attachments: 0001-Correct-the-CPU-usage-calcualtion.patch, 0001-Correct-the-CPU-usage-calcualtion.patch, HADOOP-12356-v3.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> The CPU usage information on Windows is computed incorrectly. The proposed patch fixes the issue, and unifies the the interface with Linux.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)