hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kassiano josé matteussi (JIRA) <j...@apache.org>
Subject [jira] [Commented] (YARN-2139) [Umbrella] Support for Disk as a Resource in YARN
Date Thu, 04 Jun 2015 19:50:45 GMT

    [ https://issues.apache.org/jira/browse/YARN-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573468#comment-14573468

kassiano josé matteussi commented on YARN-2139:


I have studied resource management under Hadoop applications running wrapped in Linux containers
and I have faced troubles to restrict disk I/O with cgroups (bps_write, bps_read). 

Does anybody know if it is possible to do so?

I have heard that limiting I/O with cgroups is restricted to synchronous writing (SYNC) and
that is why it wouldn't work well with Hadoop + HDFS. Is this still true in more recent kernel

Best Regards,

> [Umbrella] Support for Disk as a Resource in YARN 
> --------------------------------------------------
>                 Key: YARN-2139
>                 URL: https://issues.apache.org/jira/browse/YARN-2139
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Wei Yan
>         Attachments: Disk_IO_Isolation_Scheduling_3.pdf, Disk_IO_Scheduling_Design_1.pdf,
Disk_IO_Scheduling_Design_2.pdf, YARN-2139-prototype-2.patch, YARN-2139-prototype.patch
> YARN should consider disk as another resource for (1) scheduling tasks on nodes, (2)
isolation at runtime, (3) spindle locality. 

This message was sent by Atlassian JIRA

View raw message