hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2139) Add support for disk IO isolation/scheduling for containers
Date Tue, 10 Jun 2014 19:59:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026910#comment-14026910

Steve Loughran commented on YARN-2139:

I have looked a bit at this

Cgroups can do IO throttling -but that's just local IO, and doesn't cover HDFS IO, as that's
happening in the HDFS process, and some of it can even be happening over the net on remote

you could do some co-operative throttling of IO clients in hdfs/fs library: the client itself
does some moving average throttling. 

Otherwise, you could perhaps include some IO requirements in a container request (light, medium,
heavy) and include that in placement

> Add support for disk IO isolation/scheduling for containers
> -----------------------------------------------------------
>                 Key: YARN-2139
>                 URL: https://issues.apache.org/jira/browse/YARN-2139
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Wei Yan
>            Assignee: Wei Yan

This message was sent by Atlassian JIRA

View raw message