hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Grandl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3820) Collect disks usages on the node
Date Wed, 17 Jun 2015 20:36:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590533#comment-14590533
] 

Robert Grandl commented on YARN-3820:
-------------------------------------

Short description:
This JIRA collects bytes read/written from/to disks in Linux. 
Step 1: We exploit the "/proc/diskstats" counters, extract the number of sectors read/written
for every disk, and return the aggregation of these counters among all the disks. 

Step 2: To convert sectors into bytes, for every disk, we extract the sector size from "/sys/block/diskName/queue/hw_sector_size".

Step 3: Finally by multiplying the number of sectors from Step 1 with sector size from Step
2 we compute the number of bytes.

We tested the existence of these files in the following Linux kernel versions:
Linux 3.2.0
Linux 2.6.32
Linux 3.13.0

Also, doing further search on the web, it seems people are using/recommending these files
for extracting read/written disks counters


> Collect disks usages on the node
> --------------------------------
>
>                 Key: YARN-3820
>                 URL: https://issues.apache.org/jira/browse/YARN-3820
>             Project: Hadoop YARN
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Robert Grandl
>            Assignee: Robert Grandl
>              Labels: yarn-common, yarn-util
>         Attachments: YARN-3820-1.patch
>
>
> In this JIRA we propose to collect disks usages on a node. This JIRA is part of a larger
effort of monitoring resource usages on the nodes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message